Beta Build 160008
I thought I'd start a new thread for this build, the old one is getting a bit unwieldy...
As promised in the release notes, here are a few examples of what you can do with the
ctypesmodule... Be warned: This is pretty advanced stuff – it's very easy to shoot yourself in the foot with this, just like with plain old C... And again: I haven't been able to get
ctypesfully working in 64-bit mode, but the examples here should be relatively "future-proof" (I've been able to get them to work in a 64-bit build).
# coding: utf-8 # Some experimental `ctypes` demos for Pythonista on iOS from ctypes import c_void_p, c_char_p, c_double, c_float, cdll, util import os # Load Objective-C runtime: objc = cdll.LoadLibrary(util.find_library('objc')) objc.sel_getName.restype = c_char_p objc.sel_getName.argtypes = [c_void_p] objc.sel_registerName.restype = c_void_p objc.sel_registerName.argtypes = [c_char_p] objc.objc_getClass.argtypes = [c_char_p] objc.objc_getClass.restype = c_void_p # Some helper methods: def obj_to_str(obj): objc.objc_msgSend.argtypes = [c_void_p, c_void_p] objc.objc_msgSend.restype = c_void_p desc = objc.objc_msgSend(obj, objc.sel_registerName('description')) objc.objc_msgSend.argtypes = [c_void_p, c_void_p] objc.objc_msgSend.restype = c_char_p return objc.objc_msgSend(desc, objc.sel_registerName('UTF8String')) def msg(obj, restype, sel, argtypes=None, *args): if argtypes is None: argtypes =  objc.objc_msgSend.argtypes = [c_void_p, c_void_p] + argtypes objc.objc_msgSend.restype = restype res = objc.objc_msgSend(obj, objc.sel_registerName(sel), *args) return res def cls(cls_name): return objc.objc_getClass(cls_name) def nsstr(s): return msg(cls('NSString'), c_void_p, 'stringWithUTF8String:', [c_char_p], s) # Demo: def print_ipod_title(): MPMusicPlayerController = cls('MPMusicPlayerController') player = msg(MPMusicPlayerController, c_void_p, 'iPodMusicPlayer') item = msg(player, c_void_p, 'nowPlayingItem') if item: artist = msg(item, c_void_p, 'valueForProperty:', [c_void_p], nsstr('artist')) title = msg(item, c_void_p, 'valueForProperty:', [c_void_p], nsstr('title')) print 'Now Playing: %s -- %s' % (obj_to_str(artist), obj_to_str (title)) else: print 'iPod not playing' def set_screen_brightness(value): UIScreen = cls('UIScreen') main_screen = msg(UIScreen, c_void_p, 'mainScreen') msg(main_screen, None, 'setBrightness:', [c_float], value) def save_video(video_file): uikit = cdll.LoadLibrary(util.find_library('UIKit')) save_func = uikit.UISaveVideoAtPathToSavedPhotosAlbum save_func.argtypes = [c_void_p] * 4 save_func(nsstr(os.path.abspath(video_file)), None, None, None) def download_and_save_video(): # Download a short sample file from Apple: # http://support.apple.com/en-us/HT201549 if not os.path.exists('sample_mpeg4.mp4'): print 'Downloading test video...' import urllib, zipfile urllib.urlretrieve('http://a1408.g.akamai.net/5/1408/1388/2005110405/1a1a1ad948be278cff2d96046ad90768d848b41947aa1986/sample_mpeg4.mp4.zip', 'temp.zip') zipfile.ZipFile('temp.zip').extractall() os.remove('temp.zip') print 'Saving video...' save_video('sample_mpeg4.mp4') print 'Done' if __name__ == '__main__': from time import sleep print 'Printing currently playing music (iPod/Music app only)...' print_ipod_title() sleep(1) print 'Downloading a video, and saving it to the camera roll...' download_and_save_video() print 'Making screen very bright...' sleep(1) set_screen_brightness(1.0) print '...very dark...' sleep(1) set_screen_brightness(0.0) print '...and something in-between.' sleep(1) set_screen_brightness(0.5)
While probably not terribly useful, these are all things you couldn't do before with Pythonista. :)
As I said – don't get too excited, there are a lot of things you can't do with this... Because of the 64-bit issues, anything that would require a callback is impossible (including things like defining new ObjC classes etc.), and it's also not possible (to my knowledge) to use any block-based APIs, which includes a lot of stuff in the more recent iOS SDKs...
Very interesting indeed. I'm a bit curious as to what the 64-bit issue has to do with callbacks not being possible. I understand there are hurdles, but I thought they were a little different.
The obvious limitation on iOS is that you are not allowed to execute dynamically generated machine code. If that is how ctypes.CFUNCTYPE works, it cannot be used directly. But there are workarounds. For an arbitrary pure C callback, one may need to resort to tricks like defining lots of trampoline function slots.
ctypesis based on libffi, and from my (limited) understanding, this is how the ARM versions of the library work. ARM64 support has only been added to libffi quite recently, and in my testing, it always crashed when trying to use
CFUNCTYPEand friends... I haven't found a lot of information about this (there was a bug report on a mailing list somewhere that sounded related, but I can't find it anymore). It could very well be that I'm doing something wrong, but it also doesn't seem unlikely that the ARM64 support in libffi just isn't very robust yet.
But in most cases (all?) in Objective-C, that won't be needed.
As you know, often in Cocoa you don't specify a C function pointer at all, but instead an Objective-C object and a selector. That is easily mapped to Python. You just need to supply an Objective-C wrapper class. And blocks, as you mentioned, are just a special case of this. A tiny bit of googling should tell you how. (Or start here.)
Well, yes, when you write actual Objective-C, you'll see very few function pointers, but if you want to do things like creating a new Objective-C class dynamically at runtime, you actually do need them (for things like
It might very well be possible to work around these issues by generating wrapper classes/functions etc., but I haven't looked into that very much yet.
They have a module called rubicon for Java and Objective C. Rubicon-ObjC is a bridge between the Objective C libraries and Python. It has been tested on both desktop/laptops, and on iOS devices.
It is basically a very well fleshed out set of wrappers that looks very much like the code you are writing in your tests and demos but completely generalized. The amount of code is huge! It is handling all kinds of hairy stuff like object reference counting and being able to dynamically add methods to classes.
It enables you to use Python to instantiate objects defined in Objective C, use Python to invoke methods on objects defined in Objective C, and subclass and extend Objective C classes in Python. It also includes wrappers for some of the some key data types from the Core Foundation framework (e.g., NSString and NSObject).
There seems to be support both 32 and 64 bit usage that you can see in the "CoreFoundation" module.
Other goodies include templates for building app that include Python and bunch of visual tools like a debugger that look very interesting.
Seems to be very well supported and actively being worked on. I downloaded it and was able to use it to list classes methods and ivars so it looks like it works. I have not found an example of it being used for block code or even just delegates or callbacks.
This group looks to be doing a lot of interesting things that are very synergistic with Pythonista.
@omz - this is just a message of thanks for continuing to keep this project up to date.
I ran into a recent thread of discussion started by the pybee author Russell Keith-Magee at https://mail.python.org/pipermail/python-ideas/2014-October/029856.html where he proposes that Python be modified to formally support building it for IOS and Android. This thread goes into many of the tough decisions that currently need to be made to get a build of any kind done on these platforms, but IOS is the obvious standout for having the worst. Russell is just discussing how this could be made a standard and supported build for Python in general makes your head spin. Russell is coming at it with a "proof of concept" implementation and just wants to give it back to the community and it looks like a difficult and tedious effort to do that.
The main point we should all keep in mind is that base level of work on this is substantial and it just keeps getting harder - not easier. With 64-bit and Python 3 bearing down - it makes me wonder if there is something more the users of this product could be doing to lighten that load. I get the sense that there are a lot of old timers in this community that have spare time on our hands that might be put to use doing some of the more tedious aspects of building and testing.
UPDATE: Here is a good Landon Fuller rant on what is behind a lot of this: http://landonf.bikemonkey.org/code/ios/Radar_15800975_iOS_Frameworks.20140112.html
I see. I looked at the libffi source, and it seems they fixed something very close to this in the AArch64 port in a commit three weeks ago. They use a trampoline table as I suggested, but with PC-relative addressing and fancy virtual memory mirroring so there's no limit to how many they can have. (Although there's a bug in that they forgot to unlock the
ffi_trampoline_lockin case of failure, line 871 in
ffi.c.) So maybe this haven't been included in ctypes yet, causing your crashes.
However, as when using Objective-C APIs, the Python programmer indeed doesn't have to provide pure C functions in order to subclass Objective-C classes. The function you pass to class_addMethod() receives the selector
_cmdas well as the
selfpointer. So you can always add the same single function (at least one per value of the
typesparameter) that further dispatch to Python code based on
_cmd. Not super efficient, but it works in most cases.
There are many projects in many languages that do this. Others have mentioned some in Python that might do it this way. I myself am familiar with two in Java: the SWT Platform Interface for Cocoa from eclipse.org, and a vaguely similar thing for AWT in OpenJDK (since JDK 7).
And for blocks, what you actually pass is a pointer to a data structure (an Objective-C object) which (after the header including the function pointer) can contain anything you like. So it is even simpler to dispatch to the correct Python code.
So while this will require some work, it is definitely doable.
I've playing with the
ctypesmodule for a while now and have for instance been able to integrate a live feed from the camera in a
ui.View. I have also defined a custom Objective-C class that appears to work from the main interpreter thread. I can add an instance of that class as a delegate to a
AVCaptureMetadataOutput, to be called on a newly created serial dispatch queue when "metadata" is detected in the stream. However, when metadata, such as a bar code or a face, is actually detected, Pythonista crashes.
I've looked over the code many times and tried many variations, but I can't get it to work. I hoped there would be crash reports that I could look at and get hints from, but since iOS 8 they seem to no longer be accessible on the device, only from Xcode. But even from Xcode, there are no recent crash reports from Pythonista. Why is that? (I have crash reports from older Pythonista versions and recent from other apps.) Is there any way to enable them?
The only hint I've got is a few lines in the system log that I got when running my code while the iPad was tethered to Xcode. But it basically just said "segmentation fault", no further details. I've tried to send
retainto my delegate, but fear that maybe it is some Python object that gets garbage collected. (Comments in the PyBee Rubicon code suggest that CFUNCTYPE instances could be GC'd, but I don't quite see how that would happen in my code. That's about the only path I have left to explore.)
Has anyone else been able to get something similar to work? Or get crash reports? omz?
As a side note, I looked at the SWT PI Java code that I mentioned earlier, and it had comments suggesting that the Objective-C classes it defined only would work in the thread in which it was created (in this case the AppKit UI thread). But there was no explanation and can't really see why that would be the case.
@mteep As I said earlier,
CFUNCTYPEwill NOT work AT ALL in 64-bit builds, and I will probably move to 64-bit sooner than I had initially planned. I will continue looking into this because it's quite interesting, but honestly, it's also pretty much an 'edge use case'. It's nice that it works at all (to some degree), but it's not a feature I plan to be supporting "officially"...
As for crash reports, they've moved from the General section to Privacy/Usage & Diagnostics in iOS 8, took me a while to figure this out as well... You won't be able to get symbolicated crash reports anyway though, so it's probably not very useful to you.
@mteep wrote: "[I have] been able to integrate a live feed from the camera in a ui.View."
That sounds like code worth sharing. ;-)
Thanks, I now found the crash reports. But I find it slightly weird that they didn't show up in Xcode. Maybe it's because they end in
.ips.betainstead of just
.ips. Anyway, at least I now know which thread that crashed and why, in some sense. Then I can hopefully detect differences when I change stuff. (Interesting to note that it is a 64-bit address on an A8X, everything else is 32-bit, but maybe that's how it works.)
Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000040000004 Triggered by Thread: 5 Thread 5 name: Dispatch queue: com.apple.avfoundation.metadataoutput.objectqueue Thread 5 Crashed: 0 Pythonista 0x006915e7 0xea000 + 5928423 1 Pythonista 0x0069406c 0xea000 + 5939308 2 AVFoundation 0x218bb6f5 0x217b0000 + 1095413 3 CoreMedia 0x234822c1 0x23455000 + 185025 4 CoreMedia 0x23494713 0x23455000 + 259859 5 libdispatch.dylib 0x30dc224f 0x30db0000 + 74319
And yes, I am fully aware that you said CFUNCTYPE didn't work in the 64-bit builds. That's why I had to try it now while it is still 32-bit. (Otherwise, I couldn't compare face/barcode detections between newer/64-bit and older/32-bit devices.) My motivation is twofold.
First, this is functionality I've long wished for in Pythonista, but always assumed it would be out of scope. (Over a year ago, I prototyped something similar in Codea using a custom GLSL shader for bar code decoding, but due to the lack of a native UI I never polished it enough.)
Second, being able to use almost any iOS framework is of course very powerful, and before you added the
ctypesmodule the thought hadn't really crossed my mind that such capabilities could come to Pythonista. But now, I know it is possible, although I understand that it likely wouldn't be a high priority for you but rather an 'edge use case', as you wrote. If Pythonista has this capability, a lot of user requests for features could be solved in Python, and you wouldn't need to spend time on the edge cases (like perhaps, the
cbmodule, which I love too). I wanted to help make this a reality by both providing a usage example and a way in which it could be implemented.
As I wrote earlier, I think it is possible to enable access to a large part of the iOS frameworks without requiring a working CFUNCTYPE. I had hoped to be able to prototype such a solution that ultimately would be using a fixed set of predefined native dispatch functions. In the prototype however, the dispatch functions would need to be defined using CFUNCTYPE, while they work. (Alternatively, after seeing the
libffisource code and the recent ARM64 patch, it doesn't seem implausible that CFUNCTYPE soon could work in 64-bit.)
@ccc Yes, but I had hoped I could make it do a little more. It is rather similar to omz' AudioRecorder. I just added a
AVCaptureVideoPreviewLayeras a sub layer to the
ui.View, whose pointer I got from the undocumented
ui.View._objc_ptr. That part should work also in the 64-bit builds, as long as the pointer remains accessible.
Wow this is amazing. I am actually an iOS dev and would really love to actually write iOS specific code on iOS. I just hope Apple lets this through.
BTW is this a public beta? And if so, do you have any invites left?
@hvd, See https://omz-forums.appspot.com/pythonista/post/5792132698734592 for info on the beta.
I finally got the bar code detection to work. It was the Python method corresponding to the Objective-C method implementation (IMP) that got GC'd. Now I just have to clean the code up, a lot. In particular, I need to find a good place to dispose the Objective-C classes I registered. Otherwise, crashes are likely.
@mteep - very impressive. How did you manage to figure the crash issue out? Did you force GC manually or something like that? Also - did you make use of the PyBee projects helper classes?
@wradcliffe Thanks, but I can't really say it was a very structured approach. I started on two different approaches: rewriting using PyBee, and logging the address and memory contents of the
ctypesobjects to be able to compare with the register contents in the crash reports. But while doing that I came to realize that the way I had written the code, nothing in Python (nor in Objective-C) could actually hold on to the function trampoline slot. So I added a direct reference to the Python object which allocated it. Luckily it was that simple.
So I never completed the rewrite using PyBee, but that would most likely have eliminated the crashes too . Except that I could only have run the code once without restarting Pythonista, due to the Objective-C classes remaining registered. I currently deal with this by disposing the old class pair if I cannot allocate a new class pair, but it would be better to clean up before exiting.
In parallel, I also did a stripped down version without the metadata detection, in order to post it. However, when I added the ability to switch between the front and back cameras, it crashed again. But this does not seem to be a GC thing, rather that an
AVCaptureSessiononly should be manipulated from a single thread (or serial queue). I'm looking at Apple example code, but haven't yet had the time to re-create the exact scheduling of the code across threads/queues. A
ui.in_foregroundwould have made that easier.
@mteep - I think you are onto something that is going to be very valuable information going forward. If this has to do with a mechanism that is needed to make sure that an object is only manipulated from the same thread it was created on then a fix may be quite difficult to come up with.
The last time I tried to debug this kind of problem I gave up because adding the logging seemed to make problems go away or change to something else. I also found that I had to restart Pythonista for every single change I would make or repro was impossible. I did not have the crash report to look at so that could have yeilded a clue as well.
My only thought on this is that there should be some way to force GC related issues to occur by directly manipulating the gc system. Perhaps forcing gc to happen continuously could flush out any issues. Multithreading issues are much more difficult, in general, to debug - but there must be ways of forcing objects to be accessed from various threads in order to force these types of bugs to occur.
I am totally impressed with your tenacity and perseverance on this.
@omz - could you discuss what the background is with cffi and how it relates to the ctypes - ffi combination? I keep seeing references to it as an active project that makes writing the python code simpler somehow but does basically the same thing as ctypes and ffi in combination. Is this old stuff - or new? If new - maybe it solves the 64-bit issues already. Just curious.
UPDATE: I just noticed that cffi is included in Pythonista under pylib/site_packages along with a copy of pycparser. Interesting.
./pylib/site-packages/cffi/__init__.py ./pylib/site-packages/cffi/api.py ./pylib/site-packages/cffi/backend_ctypes.py ./pylib/site-packages/cffi/commontypes.py ./pylib/site-packages/cffi/cparser.py ./pylib/site-packages/cffi/ffiplatform.py ./pylib/site-packages/cffi/gc_weakref.py ./pylib/site-packages/cffi/lock.py ./pylib/site-packages/cffi/model.py ./pylib/site-packages/cffi/vengine_cpy.py ./pylib/site-packages/cffi/vengine_gen.py ./pylib/site-packages/cffi/verifier.py