just because the axle fits the swing arm sans the wheel, it is not a certainty that it is correct (or incorrect). You CAN assemble everything except the swing arm and (hopefully) determine the issue.
Breaking the problem down there are only two possibilities: Incorrect parts, or incorrect assembly. (Of course, both could be present) Considering how this is looking, let's take the assembly issue first as it is the easiest to eliminate. Consider that the assembly - forgetting correct assembly for the moment - is just a stack of parts on the axle, and when they are all placed on the axle, the nut will thread down to the correct location for a cotter pin. If this can not be done, either one or more parts in the stack are too long, and therefore have been substituted for the correct parts, or the axle has similarly been swapped. If all the parts are correct, then there has to be a space between the parts somewhere. Otherwise there could be a problem with the assembly which would seem to be eliminated. The right side looks to be foolproof, chain adjuster rests against spacer, then brake backing plate, bearing race, spacer, bearing race, left side spacer, and chain adjuster. If all these parts rest against each other in the stack, then you can eliminate the assembly issue and focus on which parts are wrong. The only caveat is when one part slides inside another and comes to rest on a step (some rear axle assemblies have a stepped spacer that scopes inside the wheel bearing race and rests against the step) which can make for confusion in getting the parts put together correctly, but that does not appear to be the case here. Just make sure there is not some "invisible" gap where you can't see it - like the brake backing plate hanging up somehow and not allowing that cast-in spacer to contact the inner bearing race. Looks like this also is not the case, but you need to double check that ALL the components are tight together in the stack and not being artificially spaced apart somehow. Once you know there are no "mystery " gaps you can focus on the parts, and which one (s) are incorrect.
Swing arms do get bent from a variety of causes, idiot assembly not the least. Since it is looking like you have incorrect parts, no doubt you could see this happening! However, it is not uncommon for it to be a bit difficult to squeeze all the parts between the swing arms, but 1/4" is a lot. Regardless, the slotted plates that the axle runs through need to be perfectly parallel. If they are not you should be suspicious.
So if you have checked out the assembly and swingarm and feel confident that it is a parts issue, I suggest starting with a practical assessment. How might this happen? Well, there are a million possibilities, but maybe the most likely is a lost part. It is pretty hard to lose the axle, and the length of the one you have looks to fit your swingarm, so I would guess that is probably not it. Spacers are much easier to lose. Especially spacers that stay stuck to the hub when you take it off to change a tire because they are held by a seal. Change the tire and the spacer falls out and disappears. Or the right side one falls out when you pull the axle and rolls under the bench unnoticed. Regardless, maybe look there first as the easiest to get replaced with a seemingly right but not replacement. I don't have a 500/550 to inspect at the moment, but do have a cl450 and a 400f and your left side spacer looks long to me by comparison for what it is worth. If you can't find an easier way, you can clamp a straight edge to the countershaft sprocket and assemble the wheel. If the rear sprocket does not line up, you can look at the left side spacing for the problem.
