Silicone skins over in hours, but in thicker applications it does need anywhere from 24 hours to 7 days to fully cure. In a tiny application like yours, you probably wouldn't see any effects at all. The nasty smell is Acetic Acid. Breaking the skin of a thicker application before full cure and you release it.
PVC cement is a totally different animal. It basically "chemically melts" the two surfaces together and it's water tight in a matter of minutes. In situations where serious pressure is applied, like in household water supply, I'd give it the recommended cure time. But virtually no hobby application will ever see that kind pressure.