Seems to be two separate problems

Japanese complaining about gaijins who took low level/undesirable temp jobs because of the slightly higher pay than their own countries, jobs that didn't require fluency in Japanese by the government and companies

Japanese won't speak Japanese to gaijins who are making an effort to learn Japanese to stay there long term because they don't speak fluent Japanese yet