First of all, the vinegar test is not very reliable. It's better to use a stronger acid if you can, and scratch a bit of the rock up; and even then it's not always apparent. All it does is reveal whether there are carbonates in the rock (acid reacts with carbonates, releasing CO2). For example, dolomite is hard for a lot of people to identify on mineralogy tests and what not, because it rarely fizzes, even when using a stronger acid. I doubt it would fizz at all with vinegar. It can still be a useful test, just keep in mind that it's a test prone to false negatives.
Another test that can help is to scratch it with a piece of steel (a nail, knife point, etc.). If you can't scratch the rock, it's probably some sort of silicate (very gross generalization) and may possibly be safe (big if). If you can scratch the rock with steel, it's not necessarily unsafe, but most of the rocks that will affect pH/hardness are going to be on the softer side of the Moh's scale.
The first group looks like it might be some sort of breccia. It's going to be tricky to say much more then that, since it's composed of a bunch of smaller individual rocks and a matrix, all of which can have very different mineral contents.
And, I've got nothing on the second group...