Where can I get my 500Yen deposit back for my pasmo?

Any of the train stations on the PASMO / SUICA network, which should be virtually all of them.

Shibuya Station Keio-Inokashira Line has done it for me with relatively painless paperwork.

Just keep in mind, although you can use it outside of Japan, you can only get your deposit back where you have issued it.  So if you leave the Greater Tōkyō Area, you won't be able to refund your card. Keep that in mind if you are in Kyoto or Osaka.