I don't know. A plant is a plant. It should depend on what your intention in growing it is. I'd feel bad for someone's pet getting poisoned by a strange bush, but most wild animals know what to eat or not especially if it's a local variety.

if marijuana grows in my garden, I'm in trouble

I don't like the idea of the government telling me not to grow a plant, even if it's marijuana or coca leaves. Same for poison ivy. I can understand there being liability if it's an unfenced garden, with no warning signs. Otherwise assume I have landmines buried in my yard and stay the hell off my property.

Gardening laws make no sense at all.

Agreed, the fact that gardening laws exist is stupid.