Having a T-Bag on the bike is a non-issue. Locals and travelers all use them; I had a mid-sized one on my Honda Shadow year 'round rather than saddlebags, and I've seen one guy use his to haul his groceries home.
What matters is the license plate and how far from home you are. People are curious about solo riders and stop to talk. I've camped and had people bring over breakfasts and, once, a full dinner, been offered cookies, pop, beer, coffee, advice, ride suggestions. I've fielded innumerable questions about riding and what bike I have, heard stories about trips they had taken years ago, and listened while they wished they were doing what I'm doing.
I don't get the same interest when riding with others. Perhaps a solo rider isn't intimidating whereas a group is. So load the T-Bag and go, I think you will be pleasantly pleased.