This is not a particularly American problem.

The average IQ of blacks is much lower in Africa, mostly in the 60s, but in some countries in the 40s. In places where there is ample nutrition and education it's in the low 80s, whether that's Alabama, Stockholm, or Tokyo.

So no, there's nothing special about the slaves. Evolution takes much longer than a few generations, and e.g. recessive genes and epigenetics make sure that you can't just selects for traits in a few generations by artificial means.