I would go ahead and use it, assuming you washed it well before using it. Iron pyrite is insoluble in water, so it shouldn't leach out anyway. But, I doubt it being iron pyrite. More likely it is mica. Mica, a silicate, is also non soluble in water, so it too shouldn't be harmful.