Someone mentioned the Clay Bar above. GREAT PRODUCT! If you have never used it is definitely worth a dry for some of that hard to remove buildup. I have used it on my car a number of times, thankfully have not had to use it on my bike yet, but it is always there. You spray a little spray wax on and them rub the clay bar hard over it, while continuously molding the clay when it gets too smooth.
All that said it sounds like FifthGear has experience with this so I would follow his advice, just wanted to throw my 2 cents in on the clay bar.