limestone quarry

I think Japan had a lot of different quarries all over the country providing stones for the castles and other structures. They name the stone after the quarry so it’s difficult to know exactly which kind of stone it is geologically speaking, and I’m not sure what kind of stone you’re talking about.

Anyway, here’s a website for a granite quarry on  Shodoshima island which provided stones for the Osaka castle. http://www.japanvisitor.com/japan-city-guides/castle-stones

And you can visit the Oya stone quarry museum which provided stones for houses, temples, etc. in the area. https://planetyze.com/en/japan/tochigi/oya-history-museum