Right. A die grinder will carve it out pretty easily. FWIW, I have had similar bushings that had to be pressed out, and some that left the outer steel case in the casting after pushing the rubber and center spacer out. You have to split the outer casing with a die grinder to get it free in such a situation, no more of a PITA than if it were epoxied. I've seen the castings broken too from inept attempts to remove similar bushings as sometimes they are seriously hard to remove and require great care and proper back support to keep from doing damage. My point is, you may not be necessarily be saving that future owner anything by trying to restore the press fit assembly. You likely will also have to do some pretty delicate grinding/filing/shaping of the non-machined welded surface to make it reasonably invisible - welding in places like that usually gets a bit messy. It is also easy to lose the original center point after welding unless you plan ahead pretty carefully.
Don't think of epoxy as a lazy stop-gap solution (though it can be, of course!). It is just another tool in the engineers arsenal of weapons. Right tool for the right job is the rule, and structurally, this is an ideal application if you use the right material. As already noted, you trust your very life to epoxy every time you fly on a commercial jet.