CoreMIDI using objc_util questions

[wradcliffe]

With the beta version of Pythonista it looks like it might now be possible to get CoreMIDI access going.

I know that there are several issues surrounding this like needing to have the audio key in their UIBackgroundModes in order to use CoreMIDI’s MIDISourceCreate and MIDIDestinationCreate functions. These functions return kMIDINotPermitted (-10844) if the key is not set according to Apple docs.

Pythonista seems to have the ability to play background audio now - but does that mean it has the audio key set?

In looking at CoreMIDI I see that it is more of a "C" API. It does not have many Objects - only Functions that can be called. Can you show an example of how to access a framework like this? I have just tried:

CoreMIDI = NSBundle.bundleWithPath_('/System/Library/Frameworks/CoreMIDI.framework')
CoreMIDI.load()

Which seems to work and does not throw any errors. I think I would like to do something like this C code next:

MIDIClientCreate(CFSTR("simple core midi client"), NULL, NULL, &(_midiClient));

... but since this is straight C code there is no obvious NSObject to create with methods to call to do this using objc_util.

Please feel free to waive me off, if this is going to be a dead end for any reason.

[omz]

Pythonista seems to have the ability to play background audio now - but does that mean it has the audio key set?

Yes, the key is set in the Info.plist, so that should work.

In looking at CoreMIDI I see that it is more of a "C" API. It does not have many Objects - only Functions that can be called. Can you show an example of how to access a framework like this?

Basically, you'll mostly need to use "raw" ctypes. objc_util will make a few things slightly more convenient, but it won't help you very much, given that CoreMIDI is indeed a plain C framework.

Quite honestly, I don't know if this is going to be a dead end. I've never worked with CoreMIDI before, so there might be gotchas that I'm not aware of. It definitely looks like it's going to be quite a lot of work... Anyway, I've played around with it a little bit, so here' some code that might help you get started:

from objc_util import *

CoreMIDI = NSBundle.bundleWithPath_('/System/Library/Frameworks/CoreMIDI.framework')
CoreMIDI.load()

#NOTE: `c` is defined in objc_util as `ctypes.cdll.LoadLibrary(None)`
MIDIClientCreate = c.MIDIClientCreate
MIDIClientCreate.restype = c_int
MIDIClientCreate.argtypes = [c_void_p, c_void_p, c_void_p, c_void_p]

MIDIObjectGetProperties = c.MIDIObjectGetProperties
MIDIObjectGetProperties.restype = c_int
MIDIObjectGetProperties.argtypes = [c_void_p, c_void_p, c_bool]
#... You're probably going to need a lot more declarations like these.

# Create a client:
client = c_void_p()
MIDIClientCreate(ns('test client'), None, None, byref(client))

# ...then get its properties, which is a bit pointless,
# as the only property will be the name we just assigned, but it
# confirms that the client was successfully created.
props = c_void_p()
MIDIObjectGetProperties(client, byref(props), True)
# The properties are returned as a CFDictionaryRef, which is
# 'toll-free bridged' to NSDictionary, so we can wrap the pointer
# in an ObjCInstance for easy printing etc.
print ObjCInstance(props)

[JonB]

You would use more of a ctypes approach rather than objc_utils. That means you need to explicitly set the restype and argtypes for each function, and in many cases you will need to define the Structures for various return types.
You might be able to search for ctypes and some of the method names, to see if you find someone who has already done the hard work. A few of the beginning bits with loading libraries is not necessary, but I think the rest should basically work.

[wradcliffe]

@omz Thanks for the starter code and pointer in the right direction.

@JonB I took your advice and tracked down some existing code to help jumpstart the work.

https://github.com/pmuellr/webimidi

I have been able to setup a client and a source and send data to the source. I used a midi monitor program to read the source and verified that the midi events were flowing. I only used a very small part of the CoreMIDI API. Same as what Patrick Mueller supported in his project.

The one thing that makes this so difficult is that everything has to be rolled by hand. It is especially tough for "Core" stuff that is basically C API's and heavily uses CoreFoundation functions rather then Foundation objects.

I can't help noticing that the objc_util code is using the same methodology as the rubicon-objc project. That project seems to have support for CoreFoundation.

https://github.com/pybee/rubicon-objc/blob/master/rubicon/objc/core_foundation.py

Maybe porting that code across would make my life easier as I flesh this CoreMIDI support out.

I also can't help thinking that this would all be a lot simpler if there was a utility to read a framework header file and churn out all this tedious code automatically. Somebody must have thought of that already though - don't you think?

[omz]

@wradcliffe A lot of CoreFoundation types are actually "toll-free" bridged to Foundation classes, so you can use them interchangeably. You might have noticed for example that MIDIClientCreate() expects a CFStringRef, but I just passed ns('test client') which creates an NSString. This works with a lot of other types as well, e.g. CFArrayRef/NSArray, CFDictionaryRef/NSDictionary etc., and the ns() function can be used to create these objects quite easily.

[wradcliffe]

I have been doing a lot of investigation on how to avoid doing all this error prone hand coding of API's such as CoreMIDI. It turns out that there are several different projects out there that are trying to do this.

The one that is most up to date is cffi. In theory it should be possible to use cffi to "compile" any framework header file into ctype wrappers completely automatically.

The other project is PySDL which includes PyCParser. This one was written specifically to build and maintain SDL bindings by compiling SDL.h into ctype wrappers at runtime. This work has not been upgraded since 2011.

Both of these solutions require copies of the header files for the frameworks being accessed that match up with the library being loaded. I am not an XCode dev and don't have access to the header files needed. The header files are all copyrighted by Apple as well, so it is unclear how I would get copies.

I am not sure how to proceed and not clear why you are apparently steering clear of all of this with obj_util instead of leveraging this prior work. If the answer to that it is copyright/licensing issues then I DO understand. If this is the reason it means that if I go this route, I may not be able to share my work with anyone else or include it into a product of my own. It would only be for personal use and hacking fun.

[JonB]

most of the require headers can be found at the apple open source page, though it is not well reference by search engines. there are several github repos as well that contain framework headers.

here is a link to coremidi headers
http://denemo.org/~jjbenham/gub/downloads/darwin-sdk/darwin8-sdk-0.4/System/Library/Frameworks/CoreMIDI.framework/Headers/

i thought that cffi and the like require a c compiler in order to extract type info from precompiled or compiled code, rather than trying to parse C code directly (anyone who has tried to write their own simple c type parser soon realizes how painful this is, since you have to deal with basically two languages, c preprocessor language as well as c, and before you know it you basically have to write a complete c preprocessor to handle anything other than the simplest of headers.

I think your best bet is to pre create a complete set of python bindings, perhaps with one of the tools you mentioned above, as a class, then reuse those.

[dgelessus]

@wradcliffe To get the iOS header files you only need to install Xcode from the Mac App Store for free and poke around inside the app package a little. The license on the header files is Apple's open-source code license that allows (as far as I can tell) modification and redistribution.

@JonB When set up properly, cffi can run on Pythonista with ctypes. By default it tries to import a compiled C module and do its ffi-ing with that, but you can make it use ctypes instead:

import cffi
import cffi.backend_ctypes

ffi = cffi.FFI(backend=cffi.backend_ctypes.CTypesBackend())
# Done.

The C parsing is done entirely in Python using pycparser, which is a full C parser based on ply (Python lex and yacc). Though it does expect preprocessed C code - this means no comments, no includes, no defines, no if macros.

And I am still trying, and repeatedly failing, to write up some kind of basic C preprocessor in Python. Right now it can do #errors and #warnings, #define, #undef and expansion of macro constants, and if you're lucky it can even handle macro functions. That is, nested macro function calls will fail almost all the time.

[wradcliffe]

@JonB - as @dgelessus explains, there is a C parser written in pure Python in cffi as well as PyCParser.

@dgelessus - it is new info to me that cffi must have preprocessed headers. I have not located any examples showing how they process header files and was expecting to find a utility of some kind that did the preprocessing somewhere. So you are telling me that this does not exist?

PyCParser actually has the preprocessor but I don't know how good or general it is. It is obviously good enough to handle SDL.h and is designed specifically to handle header files that include other header files, macros, ifdefs, etc. I have started looking at the code and playing with it and it looks very impressive. I keep wondering why the author - Albert Zeyer - stopped working on this 3 years ago. Might be because of cffi. The amount of work that went into this in order to solve this use case is jaw dropping.

EDIT: Also - I don't own a Mac desktop of any kind so downloading and installing XCode is not an option for me right now. @JonB thanks much for the pointer to the headers.

EDIT: Albert implemented the CWrapper (ctypes wrapper) functionality only 3 years ago so the work is still fairly fresh and he is accepting and merging fixes to the codebase from contributors.

[dgelessus]

cffi comes with pycparser bundled, it doesn't have its own C parser. pycparser's level of understanding unpreprocessed C goes as far as "this line starts with a #, I'm not touching that" and "what is this /* doing here, this isn't valid syntax". On a normal computer you'd tell pycparser to run the code through cpp (the C preprocessor) before parsing it. That's not an option under Pythonista, for obvious reasons.

[wradcliffe]

@dgelessus - I found this reference on stackoverflow:

http://stackoverflow.com/questions/31795394/using-pythons-cffi-and-excluding-system-headers

Basically it says that there is no preprocessor and even using gcc to generate the cleaned up code is not recommended.

So this effectively takes cffi off the plate for me unless someone like @omz would commit to running some kind of useful preprocessing of the framework headers and supplying these for either download or as part of Pythonista. The good news is that this might effectively remove the copyright problem since these files would not be original Apple source code. May be a gray area but I think we could get away with it.

@JonB - the pointer to the headers you sent is pretty sketchy. File dates are 2006. I am hoping to find something more official and up to date. The Apple Open Source sites I looked at do not seem to provide usable access. Lots of tar balls to download but no way to see what is in them. Still searching for a stable reliable site that provides these headers.

[dgelessus]

gcc has a lot of non-standard syntax (like __attribute__(...)) which pycparser can't handle. I'm not sure about clang, but the Darwin header files look like they don't include any other special syntax when using clang. (I'm not sure, but wouldn't we have to (pretend to) use clang, because that's what @omz compiles Python(ista) with?)

In a normal use case (providing Python bindings for a cross-platform C library) including system headers would be a problem, because they would only work on the exact system where they come from. It's also a load of boilerplate code that isn't really important in most cases. In the case of Pythonista portability is less of an issue - we're all on ARM running Darwin with the same version of Pythonista. The only important differences are the bitness (__LP64__) and the iOS version.

Just because you don't have real header files doesn't mean you can't use cffi. It just means that instead of this ctypes code:

libc = ctypes.CDLL(None)
libc.strlen.argtypes = [ctypes.c_char_p]
libc.strlen.restype = ctypes.c_int

you'd write this cffi code:

ffi = cffi.FFI(...)
libc = ffi.dlopen(None)
ffi.cdef("""
int strlen(char *);
""")

[wradcliffe]

@dgelessus - you make many good points about cffi. I hope I did not imply that it was useless in general. Only for my particular use case.

I am not the only one with this case as you may have noticed @Cethric has bitten off doing OpenGLES bindings, which makes CoreMIDI look simple in comparison. He wrote a header file processor to generate a lot of the boilerplate: parseHeaders.py

I also want to clarify for anyone else reading through this thread that I WAS able to get CoreMIDI working using a limited subset of the API and it was not all that terrible to hand code it. I just wanted to go to the next step by implementing a full set of bindings, writing test cases, etc. and went down this rabbit hole. I also can't help noticing that there are a lot of forum support requests being caused by simple coding errors for these bindings with users asking @omz to debug or help write the code. I really wish to avoid using the forum to help find my typos or teach me Python syntax.

[Phuket2]

@wradcliffe , sorry I didn't quite get the last paragraph. Is omz making errors implementing the bindings for MIDI because of too many silly questions?

[wradcliffe]

@Phuket2 - I guess that last paragraph sounds like I am angry or pissed off but that is not the case. I was just noting that a lot of the forum postings asking for help with it by other users has to do with mistakes in coding. Some of it is getting the API wrong, some is lack of understanding of Python, some is an inability to debug the mistakes. I feel like this is a waste of his time more then anything and I personally don't want to contribute to that. I could have it all wrong and he may enjoy the interaction.

Also - CoreMIDI support has been a requested feature of Pythonista for awhile by a few users but has not been implemented for a variety of reasons. You can search for MIDI or CoreMIDI to see the history of the topic. As a relatively new user you should read through those threads as well as the discussions on getting Python3 implemented to see the kinds of constraints omz is under in offering this product.

[Phuket2]

@wradcliffe , you read me correctly :) we all get frustrated at times. But we all need distractions sometime. I think if some seemingly stupid questions are been answered by over qualified people, it could be just what they need to take a break. You never know. unfortunately, I ask my fair share of stupid questions! But I am getting better :)

Anyway, I am useless to your question here. However, funny enough , I use to work for Roland Australia (many years ago, D50 was released when I was working for them). Also, As far as I know, I had one of the first ever external Midi interfaces for a Macintosh. I got a prototype from a company in Brookvale, Sydney. It was a serial interface, I had a Mac Plus back then.
It didn't matter a great deal. Atari was so ahead of the game when it come to MIDI back then. They even had pressure response.
Oh, also while I was a Roland I used to eat lunch most days with a guy that was on the original fairlight team. Fun years!

I hope you get a resolution to your problem
Ian

[Webmaster4o]

This is MIDI as in my MIDI keyboard that I can use with GarageBand for iOS? I'd love to see this in pythonista. Or am I confused about what MIDI is...

[Phuket2]

Musical Instrument Digital Interface

[wradcliffe]

@Webmaster40 - yep - CoreMIDI is the framework used to interact with MIDI devices and applications. See: How To Use CoreMIDI

[wradcliffe]

@dgelessus - I think some readers of this thread may wonder why PyObjC could not be used. This package has the longest history and goes back to 1996. It was designed to use a c compiler to generate the metadata we are discussing. It has struggled with methods of storing the metadata (preprocessed headers and massaged headers) and used XML files and more recently python source code and JSON files for this purpose. The fact that the metadata was always getting out of date and the use of a compiler as part of the process seems to have steered people away from it. Here is a pointer to the preprocessing part of the project:

https://bitbucket.org/ronaldoussoren/objective.metadata

The project README says: "This project is hopelessly incomplete at the moment".

All in all, it looks like there are good reasons why everyone is still looking for a pure python solution to my use case and why cffi offers the current best set of tradeoffs.