Yes I agree with your assessment, except for AU. I thought that the plant selection was better there than at Hoffa's. Hoffa seems to carry lots of the common plants, but nothing to unusual, like AU.
All 4 of the stores you mentioned have just a few who know plants, if you happen to get them with your question then your experience is good. At least my exeperience with AU isthat they will get the plant guys for you.
Natural World in Fond du Lac is a nice place if, like me you only get to Milwaukee every few months.