Because the idea that the hobby is defined by a tiny number of degenerate weirdos who obsess over the thing doesn't stand up to inspection. It's an ad-hoc rationalization to explain away Matt's initial thoughtless dismissal of something he's simply not interested in or had icky feels about. Who outside of aimless children and game developers define their personality around games? Even my friends who are still way too obsessed with consumer culture have jobs and families and go to church and do other stuff. People who don't have those things probably wish they did, and if they indulge too much in digital entertainment it's as a coping mechanism.