Welcome!
This is the community forum for my apps Pythonista and Editorial.
For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.
Beta Build 160008
-
I've playing with the
ctypes
module for a while now and have for instance been able to integrate a live feed from the camera in aui.View
. I have also defined a custom Objective-C class that appears to work from the main interpreter thread. I can add an instance of that class as a delegate to aAVCaptureMetadataOutput
, to be called on a newly created serial dispatch queue when "metadata" is detected in the stream. However, when metadata, such as a bar code or a face, is actually detected, Pythonista crashes.I've looked over the code many times and tried many variations, but I can't get it to work. I hoped there would be crash reports that I could look at and get hints from, but since iOS 8 they seem to no longer be accessible on the device, only from Xcode. But even from Xcode, there are no recent crash reports from Pythonista. Why is that? (I have crash reports from older Pythonista versions and recent from other apps.) Is there any way to enable them?
The only hint I've got is a few lines in the system log that I got when running my code while the iPad was tethered to Xcode. But it basically just said "segmentation fault", no further details. I've tried to send
retain
to my delegate, but fear that maybe it is some Python object that gets garbage collected. (Comments in the PyBee Rubicon code suggest that CFUNCTYPE instances could be GC'd, but I don't quite see how that would happen in my code. That's about the only path I have left to explore.)Has anyone else been able to get something similar to work? Or get crash reports? omz?
As a side note, I looked at the SWT PI Java code that I mentioned earlier, and it had comments suggesting that the Objective-C classes it defined only would work in the thread in which it was created (in this case the AppKit UI thread). But there was no explanation and can't really see why that would be the case.
-
@mteep As I said earlier,
CFUNCTYPE
will NOT work AT ALL in 64-bit builds, and I will probably move to 64-bit sooner than I had initially planned. I will continue looking into this because it's quite interesting, but honestly, it's also pretty much an 'edge use case'. It's nice that it works at all (to some degree), but it's not a feature I plan to be supporting "officially"...As for crash reports, they've moved from the General section to Privacy/Usage & Diagnostics in iOS 8, took me a while to figure this out as well... You won't be able to get symbolicated crash reports anyway though, so it's probably not very useful to you.
-
@mteep wrote: "[I have] been able to integrate a live feed from the camera in a ui.View."
That sounds like code worth sharing. ;-)
-
@omz
Thanks, I now found the crash reports. But I find it slightly weird that they didn't show up in Xcode. Maybe it's because they end in.ips.beta
instead of just.ips
. Anyway, at least I now know which thread that crashed and why, in some sense. Then I can hopefully detect differences when I change stuff. (Interesting to note that it is a 64-bit address on an A8X, everything else is 32-bit, but maybe that's how it works.)Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000040000004 Triggered by Thread: 5 Thread 5 name: Dispatch queue: com.apple.avfoundation.metadataoutput.objectqueue Thread 5 Crashed: 0 Pythonista 0x006915e7 0xea000 + 5928423 1 Pythonista 0x0069406c 0xea000 + 5939308 2 AVFoundation 0x218bb6f5 0x217b0000 + 1095413 3 CoreMedia 0x234822c1 0x23455000 + 185025 4 CoreMedia 0x23494713 0x23455000 + 259859 5 libdispatch.dylib 0x30dc224f 0x30db0000 + 74319
And yes, I am fully aware that you said CFUNCTYPE didn't work in the 64-bit builds. That's why I had to try it now while it is still 32-bit. (Otherwise, I couldn't compare face/barcode detections between newer/64-bit and older/32-bit devices.) My motivation is twofold.
First, this is functionality I've long wished for in Pythonista, but always assumed it would be out of scope. (Over a year ago, I prototyped something similar in Codea using a custom GLSL shader for bar code decoding, but due to the lack of a native UI I never polished it enough.)
Second, being able to use almost any iOS framework is of course very powerful, and before you added the
ctypes
module the thought hadn't really crossed my mind that such capabilities could come to Pythonista. But now, I know it is possible, although I understand that it likely wouldn't be a high priority for you but rather an 'edge use case', as you wrote. If Pythonista has this capability, a lot of user requests for features could be solved in Python, and you wouldn't need to spend time on the edge cases (like perhaps, thecb
module, which I love too). I wanted to help make this a reality by both providing a usage example and a way in which it could be implemented.As I wrote earlier, I think it is possible to enable access to a large part of the iOS frameworks without requiring a working CFUNCTYPE. I had hoped to be able to prototype such a solution that ultimately would be using a fixed set of predefined native dispatch functions. In the prototype however, the dispatch functions would need to be defined using CFUNCTYPE, while they work. (Alternatively, after seeing the
libffi
source code and the recent ARM64 patch, it doesn't seem implausible that CFUNCTYPE soon could work in 64-bit.)@ccc Yes, but I had hoped I could make it do a little more. It is rather similar to omz' AudioRecorder. I just added a
AVCaptureVideoPreviewLayer
as a sub layer to theCALayer
of aui.View
, whose pointer I got from the undocumentedui.View._objc_ptr
. That part should work also in the 64-bit builds, as long as the pointer remains accessible. -
Wow this is amazing. I am actually an iOS dev and would really love to actually write iOS specific code on iOS. I just hope Apple lets this through.
BTW is this a public beta? And if so, do you have any invites left?
-
@hvd, See https://omz-forums.appspot.com/pythonista/post/5792132698734592 for info on the beta.
-
I finally got the bar code detection to work. It was the Python method corresponding to the Objective-C method implementation (IMP) that got GC'd. Now I just have to clean the code up, a lot. In particular, I need to find a good place to dispose the Objective-C classes I registered. Otherwise, crashes are likely.
-
@mteep - very impressive. How did you manage to figure the crash issue out? Did you force GC manually or something like that? Also - did you make use of the PyBee projects helper classes?
-
@wradcliffe Thanks, but I can't really say it was a very structured approach. I started on two different approaches: rewriting using PyBee, and logging the address and memory contents of the
ctypes
objects to be able to compare with the register contents in the crash reports. But while doing that I came to realize that the way I had written the code, nothing in Python (nor in Objective-C) could actually hold on to the function trampoline slot. So I added a direct reference to the Python object which allocated it. Luckily it was that simple.So I never completed the rewrite using PyBee, but that would most likely have eliminated the crashes too . Except that I could only have run the code once without restarting Pythonista, due to the Objective-C classes remaining registered. I currently deal with this by disposing the old class pair if I cannot allocate a new class pair, but it would be better to clean up before exiting.
In parallel, I also did a stripped down version without the metadata detection, in order to post it. However, when I added the ability to switch between the front and back cameras, it crashed again. But this does not seem to be a GC thing, rather that an
AVCaptureSession
only should be manipulated from a single thread (or serial queue). I'm looking at Apple example code, but haven't yet had the time to re-create the exact scheduling of the code across threads/queues. Aui.in_foreground
would have made that easier. -
@mteep - I think you are onto something that is going to be very valuable information going forward. If this has to do with a mechanism that is needed to make sure that an object is only manipulated from the same thread it was created on then a fix may be quite difficult to come up with.
The last time I tried to debug this kind of problem I gave up because adding the logging seemed to make problems go away or change to something else. I also found that I had to restart Pythonista for every single change I would make or repro was impossible. I did not have the crash report to look at so that could have yeilded a clue as well.
My only thought on this is that there should be some way to force GC related issues to occur by directly manipulating the gc system. Perhaps forcing gc to happen continuously could flush out any issues. Multithreading issues are much more difficult, in general, to debug - but there must be ways of forcing objects to be accessed from various threads in order to force these types of bugs to occur.
I am totally impressed with your tenacity and perseverance on this.
-
@omz - could you discuss what the background is with cffi and how it relates to the ctypes - ffi combination? I keep seeing references to it as an active project that makes writing the python code simpler somehow but does basically the same thing as ctypes and ffi in combination. Is this old stuff - or new? If new - maybe it solves the 64-bit issues already. Just curious.
UPDATE: I just noticed that cffi is included in Pythonista under pylib/site_packages along with a copy of pycparser. Interesting.
./pylib/site-packages/cffi/__init__.py ./pylib/site-packages/cffi/api.py ./pylib/site-packages/cffi/backend_ctypes.py ./pylib/site-packages/cffi/commontypes.py ./pylib/site-packages/cffi/cparser.py ./pylib/site-packages/cffi/ffiplatform.py ./pylib/site-packages/cffi/gc_weakref.py ./pylib/site-packages/cffi/lock.py ./pylib/site-packages/cffi/model.py ./pylib/site-packages/cffi/vengine_cpy.py ./pylib/site-packages/cffi/vengine_gen.py ./pylib/site-packages/cffi/verifier.py