Got around to straightening the swing arm today. Went real well. I was initially going to straighten the swingarm off the bike, but couldn't come up with a good method given my limited tools and such. So I decided to to it on the bike. I first started by leveling the bike. My garage floor is not perfectly level (slopes to a drain in the middle of the floor) so I had to shim the center stand to make it perfect. With the bike level, it was pretty straightforward. I propped one arm up on a jack stand, heated the other arm with a torch and tapped it down until it was perfectly level with the other.
Another issue with the swingarm is the brake side shock mount was also bent outwards a bit causing the shock to be under a lateral stress. Used the torch again to help straighten them. You can see how much it was out in this pic. Shifter side was fine.
As a final check, put the rear wheel back on and was happy with the result. I know I took a before picture but can't find it for some reason. Here is the after, much better than before. Excuse the trolly jack lever handlebars, put them on to see how drag bars would feel