Huh. I've seen a lot of people have no problems keeping shrimp and tetras with bettas. I guess maybe it depends on the fish? And I thought maybe that would be overstocking. I really would like variety though, so what do you suggest would be a good tank-mate for the tetras?
The tetras may nip at the betta's fins, particularly blind cave tetras, which tend to bite whatever they bump into.
Shrimp and bettas are tricky. Some people do keep shrimp in with bettas, and ghost shrimp are bigger than most. One the other hand, my officemate adds ghost shrimp to her betta sorority as feeders, and they tend to last about 5 minutes...
If you want to keep blind cave tetras (which are very cool fish!), I would probably recommend something relatively tough and not prone to fin nippage. How about albino cories? They would go well with your "theme", and they have spiny fins to discouraging "nibbling" from the tetras. You could also add another fast schooling fish (bloodfin tetras would work) but I think you generally get the nicest effect from one schooling fish species rather than multiple.
Try 8 blind cave tetras and 6 albino cories. This should give you a nice display if you pair it with some cave-like rocks, a black background, and a few small spotlights.
Here's some inspiration for you: