It's funny to say this but putting metal foil around your mask prevents this.
EDIT:
Just FYI, the reason that works is the same reason reflective aluminum sheeting is used as insulation, polished metal surfaces reflect heat similar to how a normal mirror reflects visible light.
