I think anything that removes the glue will damage the plastic. Acetone (nail polish remover) works great on glass, but will damage plastic as you stated. Best bet may be chiseling it off with something like an x acto knife. You'll want a blade like the ones in the bottom left of the picture. The trick is to put the bevel side down and glide the blade across the surface, chipping away at the glue. Bevel side up and you will dig in and damage the plastic. With sharp blades and a light touch you should be able to remove the glue. Light sanding may or may not be necessary afterwards. Good luck!