The fish waste won't be enough, what you're seeing is normal, as your substrate is also a bio bed (grows bacteria) don't remove it just turn it over if it's that bothersome, could be diatoms which is brown algae, nothing to be alarmed about. How much plant food do you add weekly?
Sent from my SM-G930P using Tapatalk
Sent from my SM-G930P using Tapatalk