Beta Status Update

omz

Unfortunately, I've recently been unable to provide a new beta version of Pythonista 1.6. This is partly my own fault, and partly due to unexpected difficulties with TestFlight.

First off, I've been working on two different branches of Pythonista for quite some time, one being the current (expired) 1.6 beta, a relatively minor update with new Python modules but not much else, the other a more significant update with a major UI refresh and quite a few other new features that I'm pretty excited about (more on that soon).

This approach has sometimes made it difficult to keep both versions in a working state. When I changed something in version B, I sometimes ended up accidentally breaking stuff in version A... So a couple of weeks ago, I decided to ditch the current beta branch, and exclusively focus on the new version instead, but I have to admit that I vastly underestimated the amount of work that this would take – a lot of basic things were still missing in the new branch, and even though it had some nice new features, it would have been a step down in terms of stability and basic functionality that you'd expect, even if you were coming from 1.5.

Right now, it's still a bit rough around the edges, but definitely usable and an improvement over the current version. Unfortunately, I haven't been able to get TestFlight to work with it yet. I continue running into server-side issues with builds not processing properly after I upload them. In order to make some new features work, I had to change the App ID (for code-signing), and it seems that this is a scenario that just doesn't work very well with TestFlight – it's difficult to go back now because that would mean sacrificing new functionality that I spent quite a lot of time on... At first, the only reason for the new App ID requirement was the document picker support in the new dialogs module (requires iCloud entitlements), but the new version also contains an app extension (for running scripts from the share sheet in other apps), and I've realized now that this also doesn't work with the old App ID, and while the document picker support wasn't that important to me, the app extension is a major feature that I really want to ship.

If you do have the current (expired) beta installed and would like to get your data out of it, here are two options you can use:

Option 1 (if you have a Mac with Xcode):

Connect your device via USB
Start Xcode and select Devices from the Window menu, then select the connected device in the sidebar
Under Installed Apps, select Pythonista
Click on the "Gear" icon and select Download Container...

The downloaded container is a package (i.e. folder), so you can view the individual files in Finder by selecting Show Package Contents from the context menu.

Option 2 (if you're on a PC or don't want to install Xcode):

Make an unencrypted(!) backup of your device using iTunes
Download the iBackup Viewer tool (Mac or Windows) – (the free version will do, if you have the paid version, you could also encrypt your backup)
Go to "(your device)/Applications" (this may take a while to load)
Select Pythonista in the list of apps, then click the "gear" button and select Save all Files.... I would recommend that you create a new folder for this because the app will create multiple folders while extracting the backup files.

Edit: You can also try downgrading by downloading 1.5 from the App Store. Apparently, this didn't result in data loss for some people, but I'm not quite sure if this always works.

That's all I have for now. I'm very sorry this took so long. When I know more, I'll post it here.

omz

Here are a couple of screenshots of what the beta currently looks like (click on thumbnails for full-size versions):

The app extension, running in Safari:

wradcliffe

OK - my heart is fibrulating. Just seeing the MIDI support in there is enough, but the UI and game development tools look awesome. This is going to be a major upgrade. Thanks so much for getting some of these features in there. On the MIDI side of things - make sure to expose that crazy MIDI - BLE dialog box API. It is needed to write apps that enable BLE MIDI via CoreMIDI. You probably already know that - but just in case :-)

One dumb question. I can't find a way to reinstall 1.5 over the expired beta on my iPad. Is that something that has to be done from iTunes on my PC? A pointer on how this can be done would be helpful as I wait patiently for this next incredible upgrade.

dgelessus

Well this is some great news! I certainly wasn't expecting this. MIDI library, visual scene designer, more themes, more syntax highlighting support, app extension... and are those input history buttons on the interactive prompt? It's great to see what you've been secretly working on all the time. Hopefully you'll be able to resolve your problems with TestFlight etc. soon, good luck with that. Can't use Pythonista in the meantime then, but my data is safe and backed up, so I'll be patiently waiting for now. In any case, glad to hear that you're alright. Willkommen zurück Ole :)

briarfox

@omz Thank you for updating us. Looking forward to the new build! It looks awesome. Game Dev tools look like a lot of fun. In the end it is we that signed up to test builds, and unfortunately that sometimes means there will be issues. Keep up the great work!

omz

@wradcliffe, @dgelessus

I'm sorry if those screenshots gave you the wrong impression about MIDI support. There's a tiny bit of that, but probably not what you actually want. There's very basic playback support for .mid files (with a built-in soundbank), and I've included the pure-Python midiutil module for creating MIDI files (which actually works in 1.5 as well if you'd download it manually), and that's it, nothing related to CoreMIDI, really, and I don't think I'll get to that for this update, just too much other stuff I'll need to get done still, and this is an area I know very little about.

It can still be fun to generate melodies algorithmically this way, and you might be able to do more with the results by exporting a .mid file to another app, but you won't be able to use Pythonista as some sort of MIDI controller or anything like that.

wradcliffe

@omz - no worries. The MIDI file playback support should still be quite useful. If there is a way to install or supply your own soundbank via soundfonts it would be even more interesting.

zencuke

@omz thanks for the update. As usual I'm just waiting to get BLE. It sounds like there is a lot of work left to get a release done so I shouldn't expect anything soon. Should I switch to the updated snapshot? Someone said In a different thread that there was a 1.6 snapshot but I haven't tried it. Will that give me back Pythonista BLE?

ccc

@omz Awesome as always! Congratulations and good luck in your efforts with TestFlight and App IDs.

Above you wrote: You can also try downgrading by downloading 1.5 from the App Store.

That will only work for those who have not yet upgraded to iOS 8.3. On iOS 8.3 devices the workarounds that you mention above should allow folks downgrade from Pythonista v1.6 beta to v1.5 without losing their existing scripts.

Gerzer

Wow, this is freaking amazing. Tabs in the editor?! Drag-and-drop scene builder??!! iOS 8 extension???!!!!! I don’t know what to say. I personally don’t really care how long it takes for a new beta, it’s already obvious the wait will be well worth it. I don’t know about anyone else, but I say @omz take your time and do whatever is needed because what you’ve just shown us is fantastic beyond words.

PK

When will release 1.6 on App Store?
Function wonderful, I want out of use, but suffer from the beta is not desirable.

zencuke

&Omz I vote with @gerzer. The beta is really to help you by letting us help test. The goal is your standard high quality release. Only work on the beta if it helps you get there.

athros216

omz: Beautiful work! I can't wait to play with it, it looks pretty amazing!

tomkirn

Hi omz,

Option 1 to get my data out of the device isn't working. With iOS 8.3 and Xcode 6.3.1 I got following error:

mobile_house_arrest[5688] <Error>: handle_vending: Only containers for xcode-installed apps (apps with validated profiles) can be queried

wradcliffe

Could someone on this list please clarify how the downgrade from 1.6 beta to 1.5 is accomplished specifically? Going to the AppStore and selecting Pythonista only shows the "Open" option and not "Update". TestFlight only shows an option to Unsubscribe. What's the trick to getting an "Update" to 1.5 accomplished?

ccc

@wradcliffe, You have done the iOS upgrade to v8.3? If so, then the App Store downgrade is not longer an option for you. Look at the last paragraph om OMZ's post at the very top of this thread for options.

polymerchm

@OMZ: I "downgraded" before 8.3 so I lost nothing. the xcode trick does not wok in that situaton. I will try the DropBox solution.

omz

I've made some progress... A new beta should be up on TestFlight now. As I've mentioned in the release notes, it is NOT possible to install it on top of the App Store version, and I recommend that you make a backup of your files first and delete the old version before starting the download. I've made a simple backup script in case you don't have your own way of doing this: PythonistaBackup.py

Here are the release notes (also contained in the app):

New Features

Refreshed look and feel with new and improved color themes, refined toolbar icons, additional editor font choices, and a new app icon.

Selecting a different theme in the settings changes the look of the app much more profoundly than before, especially with one of the new dark themes.
App extension + appex module: This is basically a mini version of Pythonista that you can use in many system and third-party apps via the standard iOS share sheet. With the appex module, you can get the data that was passed to the share sheet (e.g. text, images, URLs) and use it in your scripts. The extension contains a basic editor (not all features of the main app, but good enough for minor changes) and an interactive console for ad-hoc experimentation. Please note that if you want to make a script (or other file) accessible from the app extension, it has to be in the special Extension folder. There are a couple of templates for extension scripts and the Extension folder also contains a few sample scripts by default.
Tabbed editor: You can now have multiple files open at once and quickly switch between them. On the iPad, there's a traditional tab bar interface (only shown when there are at least two tabs), on the iPhone, tabs are shown in a thumbnail grid when you tap the "tabs" button in the toolbar. If you have an iPhone 6 or 6 Plus, there's an option in the settings to use a tab bar instead (this doesn't work well on smaller screens, so the option isn't available on other iPhones or the iPod touch).
Significantly improved support for non-Python files: All files are now opened in a tab, so that all types of files can be renamed (it's also possible to change the file extension for all file types, which can be useful e.g. if you want to view a .pyui file as text). Apart from that, you can show a QuickLook preview for lots of files, open them in another app, extract zip archives, and play audio files.
The editor has basic syntax highlighting support for HTML, CSS, JavaScript (useful for bottle, Flask etc.), Markdown (which you see right now), and TaskPaper (if you want to keep a todo list next to your script).
New sk (SpriteKit) module: This is basically a wrapper around Apple's SpriteKit framework for creating 2D games. The API is very close to the original, but function names are often shorter than the 'original' (making it less tedious to write code with the on-screen keyboard) and I've simplified a couple of things to make it more suitable for short scripts (e.g. a few things that require setting up delegates in SpriteKit can be done by just overriding a method in sk.Scene).

While I don't plan to remove the scene module, this is intended to be a replacement for it, featuring significantly better performance, a more consistent, object-oriented API, support for advanced particle effects, filters, custom shaders and more.

There's also a new visual editor for setting up scenes without writing code. Like with the UI editor, these files are just JSON under the hood (you can rename a .pysk file to .json or .txt if you like). The sk.load() function for loading these scenes doesn't do any magic, and it's actually written in pure Python. While the scene editor doesn't do anything you couldn't do in code, it can often be easier to set things up when you see the results directly, especially for particle systems (sk.EmitterNode).
Commands in the console are syntax-highlighted now. The history UI has also changed a little and works better with Bluetooth keyboards (up/down arrow keys), and the code completion is a little smarter about when to add a space after you select a suggestion.
New setting to show the standard library in the file browser (you can't edit these files, but it can be quite interesting to read the source code nevertheless)
New "traceback navigator": When an exception occurs, you can tap on the error marker in the title bar to view the traceback in a much more readable format than before. You can also tap on individual lines to view them in the editor (even if they're in different files)
Improved sharing in the editor and file browser – instead of the custom 'Actions' menu, the editor now uses the standard iOS share sheet. You can still add custom scripts to this sheet in the settings (Editor Actions). You can also invoke a share sheet in the file browser (Edit mode), and customize this sheet via the File Actions setting. This is useful for scripts that operate on a list of files (e.g. to create a zip archive, upload them somewhere...) because the selected files are passed to the script (as part of sys.argv) automatically.
Improved asset/color picker and new built-in images and sound effects for games (mostly Public Domain, thanks to Kenney)
Improved in-editor search, including find/replace (currently only on iPad)
New setting to use "true" division in the console (this is basically equivalent to from __future__ import division in a script, you might want to turn this on if you often use the console as a calculator)
New setting for using a more compact keyboard (one row instead of two, iPad only)
New Indent/dedent menu items in the editor (copy/paste menu) – note: dedenting currently doesn't work very well if you use soft tabs/spaces
New "Highlight all" menu item in the editor (copy/paste menu) – this can be very useful for highlighting all occurrences of a variable/function name etc.
Improved "new file" menu with more templates and the ability to import photos and other saved images from the camera roll
New sound.MIDIPlayer class (plus midiutil module, currently both undocumented, but there's a "Random MIDI Melody" sample script)
Color and image previews in the code editor (when the cursor is inside a string that contains a built-in image asset name or a hex color, a preview is shown as an overlay)
Lots of new sample code (now contained in a dedicated Examples folder, so you have more room for your own scripts in the main library folder) – I've also made some minor changes to existing sample scripts. A lot of this is still very much work-in-progress (especially the larger sk examples "Platformer Game", "Word Game").

Notes and Known Issues

iOS 8.0 or later is required now
This version is built as a 64-bit app. Not only is this a requirement for submitting to the App Store very soon (starting in June), but app extensions require a 64-bit slice to work at all. Hopefully you won't notice this much, but it's possible that there are bugs related to this that I haven't caught yet. Unfortunately, as I've already discussed in the forum, this also means that a few things in the ctypes module cause crashes. I still haven't found a way around this, but all the ctypes-related sample code I've shared so far continues to work in this version.
Built-in images and sounds are currently not available in the app extension
The editor and console sometimes don't adjust their sizes correctly after the keyboard is hidden
The title of the editor and the error indicator are often positioned incorrectly after a view without status bar (e.g. an sk.View) was presented
The 'sidebar' presentation style (ui module) is currently not supported
The dialogs.import/export_file functions (from previous betas) are not available in this build (might come back, but I can't promise anything right now)
The console currently doesn't have its own Run button
Renaming a file that is open in multiple tabs may result in duplicates
On iOS 8.3, the extended keyboard doesn't respond to touches if they occur too quickly after a regular key was pressed. This appears to be an intentional change by Apple (probably to avoid mistakes when typing quickly), and it doesn't look like there's anything I can do about it.
Automatic indentation doesn't work in some cases (e.g. when splitting a line)
Cmd+left/right-arrow (Bluetooth keyboard) doesn't work correctly in some cases (workaround: press the arrow key without Cmd once before using the shortcut)
When selecting from the outline popup the editor sometimes scrolls too far
Selecting text in non-editable (i.e. standard library) files doesn't work very well
The new color picker doesn't support alpha values yet
Setting the target_node attribute of an sk.EmitterNode can lead to crashes (usually not immediately because this happens during garbage collection)
The keyboard is sometimes dismissed when it shouldn't be (usually after a popover is dismissed when any kind of modal view was presented before)
The documentation is incomplete (especially for the new sk module)
The modal and stop_when_done parameters for webbrowser.open() are currently ignored
The zoom setting in the sk scene editor doesn't work very well
Changing the editor font size can be slow

Cethric

It is looking amazing @omz, keep up the great work.

What do I need to do or what should I do to get access to the Test Flight Beta Version.

briarfox

@omz You have been busy! Thank you for your continued support in pythonista, Please give us a way to donate to you or an optional in app. With as much time that I have spent in pythonista, I feel guilty buying the app on sale and would like to support your continued development.

Looking forward to playing with the new beta!