I assume most people don't live outside the city they work in. Within a city it's pretty cheap.

Between cities, especially the Shinkansen, is pretty expensive. I assume people who have to go from one city to the other have some sort of subscription. The Shinkansen seemed to be mostly work travel - you're only allowed to use the slowest of the three trains (which stops at all the stops) with the Rail Pass.