Being libertarian because you want to smoke pot versus not trusting a authority that has the power to ban it at the state/federal level. My aggravation since being a high schooler is how the former became the household definition, because the mode averaged person has an allergy to cynicism for some gods' forsaken reason. These naive idealists, coinciding with single-issue fuckwads, are so invirtuously useless for society that the proggie NPC has a point when he paints a libertarian as a social loser living in fantasies. Then we have anarcho-capitalism, a designation stemming from humans irresistible urge to subdivide into counterproductive tribes, to be divided and conquered by the outside culture. Any hypothetical minarchist state and ancapistan would be indistinguishable in practice.
Paleoconservative/paleolibertarian would be great brands if they weren't syllablistic vomit doomed to irrelevancy to the average Westerner. Would someone notable start a movement that isn't complete anathema to social marketing, but not so vague as to be amorphously inclusionary? I propose the axiomatic party. Principled yet without delusion, unique, and only slightly more grating to pronounce than "Democratic" or "Republican". I suggest taking a lesson out of Heinlein's History and Moral Philosophy class, sticking to uncomfortable, unambiguous social truths as a serious science. Of course the conditions in Starship Troopers were different, where hard men emerged out of truly hard times, and not bread and circuses limbo we're stuck in.
If my axiomatic ideal were attempted in my lif, neurotypes and grifters would just pervert the meaning like they did to liberalism. Principled will be confused with being a dense zealot. But really, libertarianism needs to rebrand.
You're trying to herd cats. Different libertarian groups aren't necessarily going to compromise on their principles to form a solid political party. Voting violates our Non-Aggression Principle by forcing our desires on opposing voters, after all (/s).
The major problem in libertarianism that might never be properly countered is in our opposition to state structures.
I don't think it's that. People are generally pretty skeptical of certain aspects of the state (see US approval ratings of the legislative branch), or at least their political rivals, but there are fears to be raised when you want to decrease funding or power of police, national security, schools, or welfare programs. There is fear in the unknown, an "untested" method without a presumed permanent structure that states have "given" for some generations at this point. There's a reason "muh roads" is a meme. People are accustomed to a central authority to take care of things.