you mean the PVA stuff, like elmer's glue?
I wouldn't. I think it's also not very useful for gluing stryofoam together, since it's hard for it to dry inbetwixt the pieces of styrofoam.
can you just stick it together with toothpicks? I think this is what I usually did, but then I also coated everything with epoxy/sand, so that served to hold everything together when it was completed.