If I had to venture a guess, it's probably verisimilitude. Having retro-style graphics means that your brain fills in alot of gaps and sets low expectations, while a more 'modern' game with it's hyper-realistic fashion allows you to see the invisible walls much more easily.

Simply put, if a game mirrors reality in extreme detail yet doesn't allow you to do everything reality allows, you start to see the pixels wether you want to or no.

To use an odd example, it's probably one of the reasons The Long Dark, a walking survival simulator, has remained popular to this day despite being over ten years old - it's odd water-color graphic design allows for alot of forgiveness for something that tends to be very 'hard' in terms of realism in alot of ways while failing horribly in others.