Not exactly a nervous habit, but I have slight hayfever year-round. I'm pretty sure that could be very annoying to someone else who was around me a lot. (It's also annoying to me because it kills my immune system.)

Also, I bite my nails. But I'm insidious; I don't do it all the time, I do it just enough that you'd feel really unreasonable if you called it an annoying habit of mine, because I hardly ever do it, but enough to be annoying as well. And I run my fingers through my hair when I'm nervous, or thinking, or feeling most emotions, really.

And my tastes in food are freakish, because I mix eating things no sane person would touch with being a picky eater about normal things.