That wasn't the Canadians. That was the British regulars (mostly English) who were quite battle hardened from the Napoleonic Wars.

It's interesting because the invasion fleet was no fucking joke. Those were crack troops, who were heavily decorated in combat, with absolutely excellent weapons and equipment. The fact that the American militias were able to forestall them and get them to withdraw is a credit to their marksmanship in identifying and targeting officers.