Start there. You need better filtration. Yes, shrimp can survive in filterless tanks. They can survive - at least for a while - in bags of water as they're shipped around the world. But that's not ideal.
Don't take this personally but as a word of experienced advice and opinion: if you aren't willing to squeeze out a sponge in old tank water (during water change day, for instance - so you don't kill beneficial bacteria) once a week, you probably don't need to be keeping an aquarium. I squeeze sponges & intakes out multiple times per week on some of my tanks. It's just part of the hobby and only takes a few seconds. And if you need more information about shrimp keeping and the need for good filtration, use the search function here on the forum, check the Shrimp section, check the Equipment section and read through some tank journals.
I get the feeling that you are
willing to use a sponge filter, though, so that's good. Get a sponge filter. They're super-cheap. And you maybe already have one? That's step one.
One of the biiiig bonus about sponge filters: They're magnets for shrimplets and are great for shrimplet feeding.
There are many variables and things we can't easily see or test for in keeping small tanks. So we do all we can to make homes for our critters as suitable and ideal as possible.
Step two: Mess with your tank less. Feed less. Just do the basics. Do small water changes once every week or two, as needed. Your situation will improve.
As for the bio filter, that pile, plus the surface area of the tank, is all I have. I have never seen ammonia nor nitrites after cycling, so I assume this is good enough. I would be ok with adding more bio-media elsewhere if you can explain why it would be beneficial.