Masks ARE akin to wheelchairs and prosthetics:

If you're using one and you don't need to, people are gonna give you funny looks. If you do need to, they will give you the space you need. Because people are gonna wonder WHY you're wearing the mask. Robbing the place? Hiding your ID? Stealing from other customers? Planning a terror attack? Have COVID or influenza? Even at the height of NPC panic-programming, they said masks prevent spread, not self-infection, implying if you're wearing one now, you have something nasty you could spread.

So criminal or petri-dish, either way, makes sense to give you wide berth.