Well here is another follow up...cant believe this but here goes, lol.
I was noticing a strange, pulsing vibe again today and noticed my exhaust riding extra close to my swingarm cover lower bolt again. I thought that very odd, so as I was investigating it closer...my flashlight caught this:
So....my mystery noise, vibration, etc....was this offending, hidden rock the whole time!!
It was wedged in so tight that it was literally swinging my suspension pivot at the swingarm enough to put exhaust into the pivot bolt.
Took some doing, but I got it out finally.
You can see how badly it chafed my swingarm and frame.
Now, in my defense before everyone says "You should have easily seen that", yeah...I probably should have. But its one of those things, if you didn't hit it with the flashlight just right, you wouldn't have seen it.
That and I never would have believed a rock that big, could have wedged in that spot so hard for so many miles!
At least when I tore my whole primary apart, it made me finally fix a leak I had been procrastinating fixing for a long time, changed out my worn forward engine mount too, so it wasn't a total waste of time anyway. ;-)
Thanks again everyone for your patience with me and your help.
Hopefully someone else will learn from my experience here during their search!!