Is it possible to use MetalKit with objc_util

[aloof]

Hi I’m trying to render a triangle using metal kit but I’m very new to working with the objc_util module and I’m not sure if I’m going about things correctly.

I understand how to bridge certain classes and call it’s methods but I’m not sure how to call methods that don’t belong to a class. For example, after creating a MTKView I need to set its device property using the method MTLCreateSystemDefaultDevice but I’m not sure which class this method comes from and how to get it in python.

What am I missing?? Please help, thanks in advance.

Side note, If anyone has some code examples they can point me towards to learn more about bridging existing APIs it would be much appreciated

[JonB]

In short, probably, though there might be better ways to do whatever you are trying -- for instance, scene has support for shaders, or @Cethric's wrappers for opengles. https://github.com/Cethric/OpenGLES-Pythonista

Cethric's code might be a good starting place to understand how the bridge works.

Usually you need to load the framework, using objc_util.load_framework then can access the ObjCClass's by name.

Anything listed as a func instead of a class or instance method in the apple docs is accessed via the obc_util.c, or just c if you import * from objc_util, which most people do.

Then, you have to define the argtypes and restype appropriately, which is usually the tricky part, or some have installed cffi in which case you can use headers directly.

I believe to create your device you would use

MTLCreateSystemDefaultDevice=c.MTLCreateSystemDefaultDevice
MTLCreateSystemDefaultDevice.argtypes=[]
MTLCreateSystemDefaultDevice.restype=c_void_p

default_device=MTLCreateSystemDefaultDevice()

To create an MTKView would be something like

MTKView=ObjCClass('MTKView')
myview=MTKView.alloc().init_with_frame_device_(CGRect((0,0),(w,h)), default_device)

Though often it is best to look at what completions are available in the console for MTKView after using .alloc()... Sometimes there are undocumented convienence init methods that, say, get the default device for you, etc. I have not tried.

I think you'd need to create your own delegate subclass, which is a whole other level of effort...also, I think metal might be tricky in terms of threads, etc.

[aloof]

This is perfect, I never knew there was another C variable from objc_util you can access. Thank you for the quick and in depth response, I’ll be giving this another go.

I’ve been playing around with scene the past few days but I wanted to try rendering something in 3d and possibly use different shader stages. I think scene only allows for fragment shader but I’m not sure. I’ll take a look at cethrics port, I think there might be some similarities.

[aloof]

@JonB Sorry for the noob questions but I’m looking into the c variable after calling load_framework(‘MetalKit’) for the functions, I was able to find and create a device using c.MTLCreateDefaultDevice
so I tried to do the same thing for the function MTLClearColorMake using

MTLClearColorMake = c.MTLClearColorMake
MTLClearColorMake.argtypes = [c_double, c_double, c_double, c_double]
MTLClearColorMake.restype = c_void_p

Colors = {
    'clear': MTLClearColorMake(0.2, 0.8, 0.44, 1.0)
}

But running the code gives an error saying that function symbol wasn’t found in the library.

Traceback (most recent call last):
  File "/private/var/mobile/Containers/Shared/AppGroup/14D63AD1-3BA3-4745-9D78-6A22517D275C/Pythonista3/Documents/test/metal/test.py", line 21, in <module>
    MTLClearColorMake = c.MTLClearColorMake
  File "/var/containers/Bundle/Application/687ADCDB-6C3D-44D7-91BB-9E86748ECFAE/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/ctypes/__init__.py", line 362, in __getattr__
    func = self.__getitem__(name)
  File "/var/containers/Bundle/Application/687ADCDB-6C3D-44D7-91BB-9E86748ECFAE/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/ctypes/__init__.py", line 367, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, MTLClearColorMake): symbol not found

Am I looking for the function in the wrong place? Also is it possible to read all the functions loaded into c in a list of strings or something? I tried printing out the c variable to learn more about it but I get a ‘None’ handle. Is this normal? Thanks again for all the help!

Edit: I tried using c.__dict__.keys() to get a list of everything available and MTLCreateSystemDefaultDevice was the only Metal related function in the list.

[JonB]

Not a noob question at all... some of those ColorMake functions are not functions at all, but are actually preprocessor functions, so they never appear in the symbol table...

When you look at the type definition of MTLClearColor, you see it is a struct containing red, blue, green, alpha, as doubles. You have to use a ctypes structure...

from objc_util import *
import ctypes
MTLCreateSystemDefaultDevice=c.MTLCreateSystemDefaultDevice
MTLCreateSystemDefaultDevice.argtypes=[]
MTLCreateSystemDefaultDevice.restype=c_void_p
load_framework('MetalKit')
default_device=ObjCInstance(MTLCreateSystemDefaultDevice())

MTKView=ObjCClass('MTKView')

class MTLClearColor(Structure):
    _fields_= [('red', ctypes.c_double),
                ('blue', ctypes.c_double),
                ('green', ctypes.c_double),
                ('alpha', ctypes.c_double)]

MTLClearColor(1,0,0,1)

[JonB]

There is not a good way to list all symbols — there will be others, but they are basically only known once accessed.

[ccc]

Is there a way to programmatically get a list of all symbols that are currently known?

[JonB]

@ccc actually, macholib allows one to parse the symbol table of sys.executable, however that only includes the statically combiled symbols -- objc runtime basically, and a few other things. however the frameworks symbols are all in the dyld shared cache /System/Library/Caches/com.apple.dyld. in principle it is possible to parse that, but i dont think macholib supports it.

[aloof]

@JonB Very helpful, I was looking for a way to implement structs. I tried using the MTLClearColor struct code and I get a weird error saying the argument I’m passing in is of an incorrect type

Traceback (most recent call last):
  File "/private/var/mobile/Containers/Shared/AppGroup/14D63AD1-3BA3-4745-9D78-6A22517D275C/Pythonista3/Documents/test/metal/test.py", line 32, in <module>
    mtlView.clearColor = Colors.clear
  File "/var/containers/Bundle/Application/687ADCDB-6C3D-44D7-91BB-9E86748ECFAE/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/objc_util.py", line 651, in __setattr__
    setter_method(value)
  File "/var/containers/Bundle/Application/687ADCDB-6C3D-44D7-91BB-9E86748ECFAE/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/objc_util.py", line 898, in __call__
    res = objc_msgSend(obj.ptr, sel(self.sel_name), *args)
ctypes.ArgumentError: argument 3: <class 'TypeError'>: expected __Structure instance instead of MTLClearColor

But when I use a normal tuple it works (I think...the error is gone) Any idea why this might be happening?

I think jumping into this without in depth knowledge of python, obj c, and metal may not have been wise lol. The last major hiccup I’m having is using the device we created earlier using MTLCreateSystemDefaultDevice. I’m following Ray Wenderlich’s tutorial on clearing the screen to a single color
https://youtu.be/Gqj2lP7qlAM
https://developer.apple.com/documentation/metal/basic_tasks_and_concepts/using_metal_to_draw_a_view_s_contents
I’m trying to use the device to create a command buffer but when trying to call the create function I get the error

Traceback (most recent call last):
  File "/private/var/mobile/Containers/Shared/AppGroup/14D63AD1-3BA3-4745-9D78-6A22517D275C/Pythonista3/Documents/test/metal/test.py", line 37, in <module>
    commandQueue = device.makeCommandQueue()
AttributeError: 'ObjCInstanceMethodProxy' object has no attribute 'makeCommandQueue'

Stackoverflow suggested I invoke the method proxy to get the underlying data which in this case turns out to be an int. I'm not sure why the device is an int instead of some object, is this a metal thing or some kind of issue from the way i used objc_util? It creates a new error when trying to call the createCommandBuffer function on the int (which makes sense bc its an int, why is it an int)

Traceback (most recent call last):
  File "/private/var/mobile/Containers/Shared/AppGroup/14D63AD1-3BA3-4745-9D78-6A22517D275C/Pythonista3/Documents/test/metal/test.py", line 37, in <module>
    commandQueue = device.makeCommandQueue()
  File "/var/containers/Bundle/Application/687ADCDB-6C3D-44D7-91BB-9E86748ECFAE/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/objc_util.py", line 798, in __call__
    method_name, kwarg_order = resolve_instance_method(obj, self.name, args, kwargs)
  File "/var/containers/Bundle/Application/687ADCDB-6C3D-44D7-91BB-9E86748ECFAE/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/objc_util.py", line 405, in resolve_instance_method
    raise AttributeError('No method found for %s' % (name,))
AttributeError: No method found for makeCommandQueue

I think this is the bare minimum code you’d need to clear the screen a given color, I’m including it so you can see the errors if you want to. Also any insight to whether I’m setting up the ui view correctly would be helpful. Currently I’m just adding my metal view as a subview to the ui view but I’ve seen later tutorials override the ui view’s layer property with a CAMetalLayer. Is there a preferred method? Also does it matter when I add my metal view as a subview, can I do it before creating the command queue?

Code:

from objc_util import *
import ui
import ctypes

load_framework('MetalKit')

MTKView = ObjCClass('MTKView')

MTLCreateSystemDefaultDevice = c.MTLCreateSystemDefaultDevice
MTLCreateSystemDefaultDevice.argtypes = []
MTLCreateSystemDefaultDevice.restype = c_void_p

class MTLClearColor(Structure):
    _fields_ = [
        ('red', ctypes.c_double),
        ('blue', ctypes.c_double),
        ('green', ctypes.c_double),
        ('alpha', ctypes.c_double)
    ]
    
class Colors():
    clear = MTLClearColor(0.2, 0.8, 0.44, 1.0)


main_view = ui.View()
w, h = ui.get_screen_size()
main_view.frame = (0,0,w,h)
main_view.name = 'Metal Demo'
view = ObjCInstance(main_view)

mtlView = MTKView.alloc().init()
# mtlView.clearColor = Colors.clear
mtlView.clearColor = (0.2, 0.8, 0.44, 1.0)
mtlView.device = MTLCreateSystemDefaultDevice()
device = mtlView.device()

commandQueue = device.makeCommandQueue()
commandBuffer = commandQueue.makeCommandBuffer()
commandEncoder = commandBuffer.makeRenderCommandEncoder(mtlView.currentRenderPassDescriptor)

commandEncoder.endEncoding()
commandBuffer.present(mtlView.currentDrawable)
commandBuffer.commit()

view.addSubview_(mtlView)

if __name__ == '__main__':
    main_view.present(hide_title_bar=True)

Thanks again for your help!

[JonB]

Passing structs gets annoying because you have to manually set the restype and argtypes of the objc method call to include your personal struct... But tuples to get auto converted I think.

Objc methods take an optional argtypes and restype arguments which let you override, if needed.

someobj.some_method_(some_struct, restype=None, argtypes=[MYSTRUCTCLASS])

When setting attributes that are structs, you need to use the hidden set method

someobj.setAttributeName_(value, restype=None, argtypes=[MYSTRUCTCLASS])

instead of

someobj.attributeName=value

I'll go over your code more later, but I notice in a few places you are missing the () after an attribute -- ObjCInstance attributes are really methods that return a value, so must be called.

The c functions returning int or c_void_p -- if you know the function is supposed to return an ObjC object, then you can wrap the return in ObjCInstance() to get the actual object. Most object method returns get wrapped automatically by the ObjcInstanceMethod call.

It is usually a good idea to run one command at a time, in the console, then you can use the auto completion to explore what you get back.

[aloof]

Thanks for your time @JonB, running things on the console was super useful, it actually shows you all the members and functions of the objc variables.

I'm still having issues with getting the device. I tried wrapping the return value ObjCInstance(mtlView.device()) but it still returns a number, specifically <b'__NSCFNumber': 137478144> Apple docs say MTLCreateSystemDefaultDevice should return an object that follows the MTLDevice protocol. So could this number be a pointer or a memory address for where the actual device object is located or am I not supposed to get a number?

Edit: Nvm, wrapping the int with ObjCInstance does do the trick. I'm getting the A9 gpu device. I think it didn't work earlier because the order of the code.

Update: After finding the device I was able to find the correct function names and I’m proud to say we got the screen clearing to a color lol. Thanks again for all the input

Updated code:

from objc_util import *
import ui
import ctypes

load_framework('MetalKit')

MTKView = ObjCClass('MTKView')

MTLCreateSystemDefaultDevice = c.MTLCreateSystemDefaultDevice
MTLCreateSystemDefaultDevice.argtypes = []
MTLCreateSystemDefaultDevice.restype = c_void_p

main_view = ui.View()
w, h = ui.get_screen_size()
main_view.name = 'Metal Demo'
view = ObjCInstance(main_view)

mtlView = MTKView.alloc().init()
mtlView.setFrame_(CGRect((0, 0), (w, h)))
mtlView.setClearColor_((0.2, 0.8, 0.44, 1.0))

mtlView.device = ObjCInstance(MTLCreateSystemDefaultDevice())
device = mtlView.device()

view.addSubview_(mtlView)

commandQueue = device.newCommandQueue()
commandBuffer = commandQueue.commandBuffer()
commandEncoder = commandBuffer.renderCommandEncoderWithDescriptor_(mtlView.currentRenderPassDescriptor())

commandEncoder.endEncoding()
commandBuffer.presentDrawable(mtlView.currentDrawable())
commandBuffer.commit()

if __name__ == '__main__':
    main_view.present(hide_title_bar=True)