Even though toxicity is unlikely if you keep your daily intake levels below 10,000 IU per day and avoid excessively high intakes of vitamin D supplements, experts recommend that people with typical vitamin D levels do not exceed 4,000 IU of vitamin D per day
You're right.
https://www.healthline.com/nutrition/vitamin-d-side-effects#1.-Elevated-blood-levels