In a classroom, I think that's covered by Fair Use no matter what the copyright, but check with the administration, since some get paranoid.
Otherwise, I'd recommend checking your local laws if you're not in the United States (definitions of copyright and public domain vary wildly--some countries don't recognize a public domain at all) and double-check the copyright renewals for your own personal CYA action; it'll make administrators feel much better if you have printouts and stuff documenting a copyright search, rather than, "these guys said it would be OK."
If all that's covered (or you don't have the energy to care--remember, Fair Use should be fine), then yes, you can treat the material as if you owned it...which you basically do, if you're a U.S. citizen.
One additional thing to consider is that many of these comics were, to put it charitably, created in a different era. Buck-toothed "Japs," lazy Mexicans, and Vaudevillian African-Americans could get you in far more trouble than copyright infringement, so make sure to watch out for that sort of thing, too.
