You can be a reference even if you've never had a clearance. It just means someone trying to get a clearance put you down as someone they knew when they lived at a particular address.

I had someone put me as a reference, and I've never had a clearance (though he did ask me before using me as a reference, because they sometimes contact references...)