Generally, tap water has dissolved gasses in it due to it being under pressure in the pipes. This quickly de-gasses when exposed to normal room pressure. Nothing to worry about.
I use RO water for other reasons,(my tap water varies in chemistry from week to week). I can say that it is more hassle getting the minerals added back into the water and it is more expensive and a pain in the butt if you do not have your own RO system because you have to buy water and lug it home.