I think if you had the spacers on the wrong side, the brake caliper would not line up.
=================================================
Looking at the spacers from the back of the bike, the short spacer will be on the right, and the long on the left.