Are you going to be running filtration? I'd assume the breakdown of the potting soil would release dangerous amounts of ammonia unless the tank was heavily planted from the start.
Also IME, bettas will eat shrimp. If they can't eat them whole, they will harrass and nip at them until they die from stress or jump out. I think decent plant cover might mitigate this though.
I think I'll pick up a tiny filter for it.
as far as cover goes, I plan on doing TONS of anubias (or similar depending on availability). I pretty much want the ground covered with nice hiding places.