Gestures for Scene
About a year ago I started looking for a solution to gesture recognition for a Pythonista Scene module. At that time, I was able to find the wonderful Gestures module for UI Views by @mikael but I was unable to use this in the Scene module (game). I was also unable to find anything that fit the need at the time, so I began crafting my own bit of code inspired by Gestures but for the Scene module using only Python code and the scene.touches interface (not hooking back into the objC code that the Gestures module uses).
Some of the history of this project can be found at this forum post:
But I am pleased to say that while the code hasn't progressed much since that post, I have now gotten the code broken into individual files (instead of the monolithic code blob posted previously) and put on GitHub here:
Because this continues to be developed completely in my free time, and is fairly low on my priority list at the moment, I am providing this on GitHub in hopes that someone with more free time will take an interest and help me bring this project to completion for others to freely use.
The current status of the project:
By running the code from the gesture_handler.py file, the user gets a Scene in which user touches can be visualized on-screen individually and data about the "gesture" (grouping and movement of the current touches) can be seen at the top of the screen.
Gesture data available:
State (of the recognizer internal state machine)
Number of Touches present
Duration of the "gesture" (seconds)
Translation of the "gesture" (pixels x, pixels y)
Rotation of the "gesture" (degrees)
Scale of the "gesture" (multiplier 1.0 = 100%)
Result of the Recognizer (was the "gesture" recognized, and if so what type)
There is also logging capability built in. To enable it, uncomment the code block near the top of the gesture_handler.py file. The logger dumps quite a bit of data (at least one message per update loop).
My intention for the project is to
(1) Finalize the functionality, making it similar to the Gestures module in that you have an object that, using hook functions placed inside the Scene module's touch_began(), touch_updated(), and touch_ended(), will analyze the touches and when a gesture is recognized will call a user defined callback which was setup on object creation along with the type of gesture to be recognized.
(2) Possibly move some of the recognizer's functionality onto another thread in order to improve performance (I have very little experience with multi-threaded programs in Python / Pythonista)
Currently, I am happy with the math which is analyzing the touches, but the schema for recognizing the "gesture" based on that data is not reliable. If someone would like to help troubleshoot what I have or develop a new / better schema for analyzing the data generated about a grouping of touches to recognize it as a gesture, that would be great!
@dcl: ”And the reason that your Gestures library is incompatible is that the objC calls that it is based on are dependent on a UI object to attach them to, of which there are natively none in a Scene application.”
scene.view. I think you should be able to attach ObjC gestures to that, but have not tried.
from scene import * import Gestures class MyScene (Scene): def setup(self): self.ship = SpriteNode('spc:PlayerShip1Orange') self.ship.position = self.size / 2 self.add_child(self.ship) Gestures.Gestures().add_pinch(self.view, self.handle_zoom) def handle_zoom(self, data): self.scale = data.scale run(MyScene())
@mikael, You’re great. Goes to show what I know... haha.
I’m going to play with this some to see what all I can do with it, but knowing this earlier (and shame on me for missing the fact that
Scene.viewthat could that could be attached to) would have saved me a great deal of work on what I have made to solve this issue, although making that code also taught me a lot, so I guess that’s a plus.
I’ll leave the code on GitHub if anyone is interested in looking at it. I feel like there are some good parts to it, and certainly some parts I am not happy with. Hopefully someone can find some use for it, and I may end up combining parts of it with Gestures to get a final result I am happy with using in a
@mikael Notably, for whatever reason, Pythonista is giving me continuous
NameError: ObjCInstance is not definedwhen I run the scene script, but not when I run directly from Gestures (your demo code). The code still runs, Pythonista just complains about the
NameErrorthe whole time. I’m going to look into this, but any advice is welcome.
Thanks for your input and your help!
@dcl It's like your "from objc_util import *" was after your use of ObjCInstance...
You might try instantating the gestures object in
_init_rather than setup. Setup happens in a strange thread land, though likely view does not yet exist during init... It might be cleaner to use a sceneview, since that is what
rundoes under the hood -- creates a sceneview then presents it .
from objc_util import *is part of the Gestures.py file which was obtained as-is from GitHub and the call is at the top of the file, before any of the code. So unless the files are getting included weird as a part of the Scene setup, I am not sure what might cause that behavior
@JonB Thanks for the tip! I tried instantiating the gestures object in the
__init__()function of the
setup()that seems to work. The key being that you MUST call the
Scene.__init__()function inside the
MyScene.__init__()function, else the script will throw exceptions of missing members.
The revised code is below:
from scene import * import Gestures class MyScene (Scene): def __init__(self, **kwargs): Scene.__init__(self, **kwargs) self.GO = Gestures.Gestures() def setup(self): self.ship = SpriteNode('spc:PlayerShip1Orange') self.ship.last_scale = self.ship.scale self.ship.position = self.size / 2 self.add_child(self.ship) GO.add_pinch(self.view, self.handle_zoom) def handle_zoom(self, data): self.ship.scale = self.ship.last_scale * data.scale if data.state is self.Gest.ENDED: self.ship.last_scale = self.ship.scale run(MyScene())
Having now seen the light by having tested the
code with a
Scene, I am convinced that this is the best way to implement what I am trying to do in my
Sceneapplications / scripts. However I am curious if more advanced gesture recognition is possible.
I read here about subclassing the objective C
UIGestureRecognizerclass to implement more advanced or customized gesture recognition.
@mikael or @JonB is it possible to subclass
UIGestureRecognizerinside Pythonista using
objc_util? I read the Pythonista documentation and from my understanding it is possible to subclass Objective C classes in Pythonista, but since the Apple help page specifically mentions having to include this
#import "UIGestureRecognizerSubclass.h"before doing the subclassing, I’m skeptical it can be done in Pythonista. And if it is possible, I am still yet unclear on how to properly subclass an Objective C class with Python syntax.
Yes, you can subclass using objc_util.
Basically, you create method names as defs (take the objc selector and replace colona with underscores), then use create_objc_class. You would tell it the superclass, and any protocols, and the list of methods. Sometimes you must specify the argtypes and restype, but when overriding existing methods or implementing a protocol, that is not needed because objc_util can figure it out from the name and runtime introspection.
There seem to be a lot of methods you need to implement. The main ones of course are the equivalent if touch_began/moves/ended.
If you need gestures that play nice with other recognizers like pan, pinch, etc, this might make sense. But I'm not sure if there would be much performance boost, compared to implement your own with the scene touch methods. Read up on the basic state machine idea, and it gives a good indication of how you could frame such a state machine in python as well...
from objc_util import * import ui def touchesBegan_withEvent_(_self,_cmd,touch,event): print('TOUCH ********\n', ObjCInstance(touch)) print('EVENT ********\n', ObjCInstance(event)) MyGestureRecognizer=create_objc_class('MyGestureRecognizer',ObjCClass('UIGestureRecognizer'), methods=[touchesBegan_withEvent_]) g=MyGestureRecognizer.new() v=ui.View() v.objc_instance.gestureRecognizers=ns([g]) v.touch_enabled=True v.present('sheet')
here is how you would get started.. you then have to figure out which methods you want to implement, and implement those.