I cut out the bearing support spacers with a mini-bandsaw. Used two drills to line up the new spacers with the bearing support and smoothed both parts together. I mounted the spacers to the bearing support with 2 small countersunk screws on the front. On the rear spacer there was not enough room so there I used 2 dowel pins.

After test-fitting on the bike I noticed the studs in the gearbox were now a bit too short and the clutch housing was hitting the bearing support. I countersunk the nuts and machined off a few millimeters from the bearing support, now everything fits correctly.