Safety is an issue, but I most travel alone in Western Europe, which is about 20 times safer than my hometown.
The spouse hates to fly, so when I have time off work and some extra $ (I don't have hobbies, so I save a lot), i spend a week or two in Europe. I will either just stay in a great city (next up, Arles), or rent a car and drive around the countryside, staying BnBs and cheap hotels.
I haven't yet done what you mention- taking one of those caravan tour-- but I want to. I'd love the idea of someone else making all the arrangements and doing the driving!
I tend not to notice when people are dissing me, and generally don't care (I'll never see them again). But if anyone offends me, I try to answer back really brightly and happily, like I'm the luckiest person in the world to have this opportunity. "Oh, yeah, I love travelling alone! I just do exactly what I want without having to worry about anyone else."
I think if those critical people you meet would be really honest, about half would confess they envy you, that they're going crazy stuck with this spouse/companion/roommate all the time.