Sorry for your loss. Bettas always have a special place in their owner's hearts.
Have you kept shrimp before? A 2.5g is pretty small for a first attempt at shrimp, even RCS, IMHO. Did you cycle the betta's tank? Was the tank heated and filtered for the betta or did you just do a 100% WC? You will need to cycle the shrimp tank. Are you familiar with the nitrogen cycle?
I recommend the Penn Plax Small World filter for a tank that small. They're small and they work well, IME. Get two so you can change one cartridge without losing your cycle.
You can get a small bag of Tahitian Moon Sand at Petsmart or Petco's store brand of sand for really cheap. Go with either of those.
I hate marimo balls. They collect schmutz and also fall apart. h4n here has a great selection of mosses. I've bought at least two kinds from him and have been very pleased.
The more stuff you can put in a tank, the better. Shrimp eat mostly biofilm and you need surface area for that. Get a little piece of driftwood and two lava rocks; one for moss and one for just plain biofilm. Plus lava rocks are great places for the beneficial bacteria to live. And you can never have too much beneficial bacteria.
If you'd like to do more research on shrimp keeping, go to planetinverts.com. It's really comprehensive.
If you're looking for regular RCS and not the fire reds, find Epicfish. He sells RCS for $1 each.
-Lisa