Should Gender-Based Pricing of Consumer Products be Illegal?

Why should I have to pay more for a white shirt (“blouse”) then a man? Or do I need to wear all men’s clothing? In the long run I lose either way.