Pythonista 1.6 Beta
So, with Apple's new TestFlight, I can finally have a lot more beta testers, and won't have to worry about device limits so much.
If you'd like to play with the next version of Pythonista before it hits the App Store, please send me an email with your Apple ID, and I'll see what I can do. I don't plan to use up all the slots yet (I might need some later), but if you've been posting here, there's a very good chance that I can give you access to the beta.
As for what to expect, here's a list of the new features in the current beta:
- Improved support for the iPhone 6 and 6 Plus screen sizes
- Removing files from the library moves them to a trash folder instead of deleting them immediately.
- URLs in console output are 'linkified' automatically
remindersmodule for accessing the iOS Reminders database (read/write)
cbmodule for connecting to Bluetooth LE peripherals (experimental)
- New dialogs module as an easy-to-use alternative to the ui module when you just need simple data entry. This also contains functions to import files from the iCloud Drive document picker (iOS 8 only). – note: this is broken in the current beta, but will be fixed in the next few days
For what it's worth, I have seen Pythonista crash when editing text files that have non native line endings. I have downloaded a lot of code from various projects on Github and via simple copy paste and occasionally run into source code in the editor that looks like it has an extra space on the ends of lines. I think this means that the lines are cr-lf terminated. Usually you can just clean the text up manually by deleting the extra "spaces" at the ends of the lines. However, doing this with some files can cause Pythonista to crash as you backup (delete) one of these characters. The crashes for these files seem more frequent on blank lines (no text other then the single extra "space"). I avoid these crashes these days largely by avoiding copy pasting code into Pythonista from random websites. I wish I had a sample to provide, but could not find one today.
On deleting Carriage Returns causing crashes: http://omz-forums.appspot.com/pythonista/post/5289645013204992 and http://omz-forums.appspot.com/pythonista/post/6431833046646784
The following extra modules in Pythonista are not current:
| module | local | PyPI | | name | version | version | | ------------- | -------- | ---------- | | bottle | 0.12.5 | 0.12.8 | | cffi | 0.8.6 | 0.9.2 | | Crypto | 2.6 | 2.6.1 | | distutils | 2.7.4 | | | ecdsa | 0.11 | 0.13 | | html2text | 2014.4.5 | 2015.2.18 | | httplib2 | 0.8 | 0.9 | | jedi | 0.7.0 | 0.8.1 | | jinja2 | 2.7 | 2.7.3 | | markdown | 2.2.0 | 2.6.1 | | markdown2 | 2.2.1 | 2.3.0 | | matplotlib | 1.3.1 | 1.4.2 | | mpmath | 0.18 | 0.19 | | numpy | 1.8.0 | 1.9.1 | | paramiko | 1.13.0 | 1.15.2 | | parsedatetime | 1.3 | 1.4 | | pyflakes | 0.7.3 | 0.8.1 | | pygments | 1.6 | 2.0.2 | | pyparsing | 2.0.1 | 2.0.3 | | PyPDF2 | 1.22 | 1.24 | | pytz | 2013b | 2014.10 | | requests | 2.2.1 | 2.6.0 | | simpy | 3.0.2 | 3.0.6 | | six | 1.6.1 | 1.9.0 | | sqlalchemy | 0.9.7 | 0.9.8 | | sympy | 0.7.4.1 | 0.7.6 | | werkzeug | 0.9.4 | 0.10.1 | | xmltodict | 0.8.7 | 0.9.2 | | yaml | 3.09 | 3.11 | | ------------- | -------- | ---------- |
Also, PIL could be updated via Pillow but this would require major surgery.
@the_buch Can we have a link to that workflow?
@omz latest beta has fixed my crashing issues. I have not had a crash since updating. Thanks!
@techteej I can confirm that the Export - Send Email... does not work on iPad. It does work on iPhone though. After "Creating Archive..." the email dialog shows up on iPhone, but it does not on iPad. This is 1.6 (160007).
A possible backward compatibility issue. A number of existing scripts that have been written assume that the default bound/frame of ui.View is something large. It now defaults to something relatively small like 100x100. Net result is that the ui of some existing scripts comes up in a tiny box and unusable. This change in default may be a fix to some other problem so this is just an FYI. I have had to fix several scripts to set a proper frame in the init section of their derived view class.
It might be a good idea to add a sample that shows best practice for setting sizes of views based on the devices orientation and ui.get_screen_size() and uses flex attributes.
Also - this is for apps that generate their ui programmatically, not via pui files.
UPDATE: The 'sheet' presentation option has been changed in 1.6 to allow its frame to be user controlled. The default seems to be 100x100 which is different then what it was in past releases.
Email export indeed seems to be broken on iPad. Should be relatively easy to fix.
A possible backward compatibility issue. A number of existing scripts that have been written assume that the default bound/frame of ui.View is something large. It now defaults to something relatively small like 100x100.
The default frame/size of
ui.Viewactually hasn't changed, but the 'sheet' presentation mode now uses the current size of a view instead of a fixed (larger) size. This is basically a new feature of iOS 8, and while I could effectively disable this, it can be quite useful. The alternative would be to actually change the default frame of new views (which has always been 100x100 for most view types), but this could break other things...
@omz As for the ui.View thing, putting this in the release notes would help a lot of people, even though it is not a change in the actual app.
Having the cb module for BLE is very cool, but I'd like to get the manufacturer specific data while scanning. And the RSSI, and set the scan option to YES. This is for generic advertising BLE peripherals. Also, support for iBeacons would be great -- it is a separate CoreLocation API on IOS.
I second @bvwelch's requests. The support for RSSI and a look at iBeacons is going to be required for many appplications. This allows you to get data without having to connect. The other item I would like is providing the peripheral to all the callbacks. This becomes important as soon as you need to support more then one peripheral. It also becomes a performance issue when you have to somehow match data coming out of a characteristic with the peripheral sending the data. It is tricky to do this on the python side since the callbacks are running in multiple threads.
Also - the CentralManager supports state preservation and restoration but you have to ask for it and supply a key to the init procedure. This would be extremely useful for writing a real world app that can be terminated and restarted in a variety of ways. Right now you have to do a complete rescan every time you run your script which can take some time.
Is it too late to request TouchID support? At the minimum to replace
keychain.master_password, but ideally to be able to use within a script.
Is it too late to request TouchID support? At the minimum to replace keychain.master_password, but ideally to be able to use within a script.
Not sure yet, it's definitely on the roadmap, but might not make it into 1.6.
Found a bug that will cause the console to become completely useless.
I have a program that redirects the
sys.stdoutstream to a
ui.TextFieldvia the following kind of class:
class OutputField (object): def __init__(self, textfield): self.out = textfield def write(self, s): self.out.text += s
sys.stdout = OutputField(v['textfield1'])and now the console will not do anything; not even autocomplete code. I found a quick fix on my iPhone 6 by simply uninstalling and reinstalling the beta, but I have enough projects on my iPad 3 to make that too impractical at the moment.
If anyone has an idea to fix this, that would be greatly appreciated. I gave tried
sys.stdout = sys.__stdout__, which is supposed to contain the origin system output, but it did nothing.
This also effects other projects as well; as nothing will print to the console.
After searching frantically around the web, I finally tried restarting Pythonista (I didn't immediately do this at first because it didn't work the first time I tried on my iPhone). Anyways, I found out that saving a reference to stdout is always a good idea.
It's late, I hope this edit helps people in the future.
Just wondering, what happens when the beta 'expires'? Does it revert back to 1.5?
@blmacbeth, this is due to the unusual way that Pythonista uses to print output to the console. What is stored in
sys.__stdout__is the default Python output stream (a file object pointing to file descriptor 1, which is Unix
stdout) but (presumably) because of iOS limitations that goes into nowhere.
Instead Pythonista has a "secret" module named
_outputcapturewith a few built-in functions to handle text I/O through the Pythonista console.
sys.stdoutis also replaced with a
writemethod encodes the string and shows it using
_outputcapture. Because the
StdoutCatcherclass isn't stored anywhere by default, it is lost when you change
sys.stdoutand it is hard to restore afterwards.
The reason why the interactive code completion stops working as well is simple - whenever Pythonista needs a list of possible completions, it creates and runs a function named
_pythonista_complete_line, which uses some standard Python module to do the completion and prints all possible results to
sys.stdout, where they are caught by Pythonista and displayed in the completion list. (Yes, it is possible to monkey-patch
sys.stdoutand hack the code completion mechanism.)
Here's an emergency script that you can run in case the in/out/err streams get lost. It will replace them with objects practically identical to Pythonista's.
import importcompletion as _importcompletion import _outputcapture if ( sys.stdin.__class__.__name__ != "StdinCatcher" or sys.stdout.__class__.__name__ != "StdoutCatcher" or sys.stderr.__class__.__name__ != "StderrCatcher" ): _outputcapture.CaptureStdout(b"I'm alive.\n") class StdinCatcher(object): def __init__(self): self.encoding = "utf8" def read(self, limit=-1): return _outputcapture.ReadStdin(limit) def readline(self): return _outputcapture.ReadStdin() sys.stdin = StdinCatcher() _outputcapture.CaptureStdout(b"Rebuilt StdinCatcher and sys.stdin...\n") class StdoutCatcher(object): def __init__(self): self.encoding = "utf8" def flush(self): pass def write(self, s): if isinstance(s, str): _outputcapture.CaptureStdout(s) elif isinstance(s, unicode): _outputcapture.CaptureStdout(s.encode("utf8")) def writelines(self, lines): for line in lines: self.write(line + "\n") sys.stdout = StdoutCatcher() _outputcapture.CaptureStdout(b"Rebuilt StdoutCatcher and sys.stdout...\n") class StderrCatcher(object): def __init__(self): self.encoding = "utf8" def flush(self): pass def write(self, s): if isinstance(s, str): _outputcapture.CaptureStderr(s) elif isinstance(s, unicode): _outputcapture.CaptureStderr(s.encode("utf8")) def writelines(self, lines): for line in lines: self.write(line + "\n") sys.stderr = StderrCatcher() _outputcapture.CaptureStdout(b"Rebuilt StderrCatcher and sys.stderr...\n")
@dgelessus Thank you for that excellent explanation. Where did you find all of that out? Is it hidden deep within the documentation, or have you looked through some source code? Just wondering.
As far as I know none of this is documented anywhere, but Python has great introspection capabilites and an interactive prompt. With a little brute-forcing I managed to figure out much of the iOS/Python interaction internals.
_outputcapturemodule is in the list of
sys.builtin_module_namesand the syntax of its functions is (relatively) easy to guess. It is always imported (because it is necessary for console IO), but it is hidden from the autocompletion list. If you want to experiment with it, use
import _outputcapture as outputcaptureto import it under a name that isn't hidden from autocompletion.
Pythonista's custom in/out/err stream replacements are normal Python classes/objects, and although there is no source code for them, they can be decompiled using the
dismodule. Python bytecode is not too hard to read in its decompiled form, so it was relatively easy to translate it back into Python source code.
Figuring out the interactive autocompletion was a little more challenging. At first I only noticed that when typing a single
_into the console without hitting Return, one of the completions is
_pythonista_complete_line. However when trying to use that function it isn't there - it is only created when Pythonista needs a line completion, and once the results are returned it is immediately deleted. It still lists itself in the completion list though, because it exists in the main namespace until it is fully executed once.
Later I found out how I could get a reference to the actual function object - I (accidentally) shadowed the built-in
settype with another function, which caused the line completion function to raise exceptions every time it ran. (This made the interactive prompt completely unusable, so from then on I had to use short Python scripts as a console replacement.) Because the exceptions occurred inside of
_pythonista_complete_line, their tracebacks had a reference to the function, which I could
dis-assemble to reconstruct the code:
def pythonista_complete_line(): import string import sys from rlcompleter import Completer completions = _importcompletion.complete("current input text", "text") if completions is not None and len(completions) > 0 and completions != "importcompletion": sys.stdout.write(string.join(completions, "|")) else: completer = Completer() completions =  exclusions = set(["___omz_complete___(", "_importcompletion", "_outputcapture"]) for i in range(1000): c = completer.complete("current input text") if not c: break if c not in exclusions: completions.append(c) if len(completions) > 0: sys.stdout.write(string.join(completions, "|")) else: sys.stdout.write("|")
Nice detective work there, @dgelessus! :)
Omg, thank you! I had discovered the function, but was never able to figure out the interface, or hoe it got access to the current input text line. Will this work for allowing completions within Cmd loops? Or ... Ok, your
current input textIs really a placeholder for some inaccessible string?
I would LOVE to implement a history system usable within pdb.pm() for example.