If this is anything like mine (CB450 K6), then you'll want to adjust the brakes so it's engaging the brake lever enough to limit travel, but not so much that it actually engages the brake pad to the drum. Hope that makes sense. Essentially, from the natural stop of the brake lever (on the drum) to the point where it's actually engaging the pad to drum, there's a ton of travel. So take up some of this slack (this is where a "stop" for the arm on the rear set comes in) so the travel is minimized. You'll want to adjust it so you aren't causing friction on the pad/drum, however.
Hope this makes sense and applies to your issue! Let me know if I need to explain any of this in a different way.
Tim