You are correct in that there are spacers on each side of the wheel but the difference in length of the spacers is so obvious that you would not be able to fit the rotors into the calipers if the spacers were switched.
BTW: I am probably the last one to know this but by putting a little Never Seize on the front and back axle while installing a wheel will prevent the rusting and harder-than-ought-to-be removal of the axle next time it's removed.