I don't see why not. Unless the place was swarming with disease, well, a swab is basically just a snapshot of what was stuck in the nose at the time ... and one probably has plenty of coronaviruses come and go out of one nostril or another without it ever being able to gain a foothold and cause an infection. Just one of those "random chance" moments, like getting exactly 50:50 heads:tails.

And a virus infects by getting into the cells, it can't do very much just sitting around on top of them, except try to get in so it can cause infection, which means that it takes over the reproductive machinery of your cell to make more of its own self.