What about Women?
The only women who ever wore togas were girls(but this ended by the early Empire),
and disreputable women, which is to say prostitutes, and those found guilty of adultery.
Sorry... But it didn't stoppped noone!

Look at following pictures...