One of the key characteristics of Asperger's — at least in most people I've encountered who have it, along with my own experience — is an inability to make "small talk". In my case, I actually find small talk actively uncomfortable, even cringeworthy at times.
What I mean by small talk is those conversations that people have for no other reason than to fill the silence. At my workplace, for example, every conference call tends to start with asinine conversations about the weather or what people are doing at the weekend or whatever. No-one really gives a toss what each other is saying; they're just saying things for the sake of it.
I think the reason this makes me uncomfortable is because it's communication without clear purpose; or, well, no. I know the purpose — to fill silence — but I don't understand the purpose. Consequently, in a situation that would generally demand small talk, I will either remain completely silent or, on a rare occasion where I have a bit of confidence, blurt out something awkwardly, inevitably fail to get the exact reaction I anticipated and promptly clam right up again.
As you might imagine, this makes me not a ton of fun to go out to restaurants with, to be waiting for a bus with or to have a long car journey with. Thankfully, most of the people with whom I would typically do those activities know me well enough to understand that if I'm quiet or silent, I'm not being cold or rude, I'm just not in a particular position to be able to adequately process the apparently meaningless conversation unfolding in front of me.
There's a self-confidence issue here, too. When conversation does start to take on a bit more structure and move past small talk, I find it quite difficult to spot cues and provide my own contribution to a discussion. In the time where I "wind up" to give a response — which is often quite a lengthy process in my head — I often second-guess the things I'm planning to say and end up talking myself out of saying anything at all, worrying about coming across as stupid, ill-informed or somehow inadequate.
All these reasons are why I tend to avoid one-on-one social situations, online voice chat and the like… and why it surprises (and pleases!) me that I'm able to cope with this enough to produce a podcast with a good friend on a semi-regular basis. That "good friend" bit is probably super-important.
It's not you, it's most definitely me!