My current approach is to load the tileset.png which contains all the tiles. I then use Image.crop() for every tile I want to extract. I then upscale them using Image.resize() by some arbitrary factor, because if I would let the scene upscale the 8x8 textures to something like 64x64, the performance drops hard. Using BytesIO I convert them to ui.Image without saving them on disk and from there I can load them as a scene.Texture.
But thank you for the hint that I can load the image and scale them in one operation!
All I had to do to fix the original problem was converting the tileset.png to RGB mode.