I think you're going to need some hidey-holes for him, even just a little clay pot or something like that. He'll be too stressed, otherwise. In fact, that might be why he won't eat... Any live plants like Java moss or Java fern would be good, too, they'll help keep it clean for the little guy.
Yeah, his fins can probably grow back.
I'd suggest getting a big tupperware, several gallons if you can, and using that as a QT tank. Not one that's been used before, it could have soap or cleaners in it. Some pet stores will sell plastic 'critter keepers', one of those would be good and they tend to be fairly cheap. A 10g starter kit from someplace like Petsmart should be pretty cheap, too, and that would be a perfect tank for a school of pygmy cories.