Welcome!
This is the community forum for my apps Pythonista and Editorial.
For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.
StaSh - Shell Like an Expert in Pythonista
-
[EDIT]: 2016-05-26
Inspired by shellista and its variants, StaSh is a serious attempt to implement a Bash-like shell for Pythonista.
Since its initial release, valuable contributions and advices have been received constantly from the Pythonista community. The two most popular utilities are
pip
(authored by @briarfox) andgit
(authored by @jsbain). Remarkable contributions are also made by
@dgelessus, @pudquick, @oefe, @cclauss, @georg.viehoever, @BBOOXX and @bennr01.StaSh stands for Pythonista Shell. While Sta may not be the best abbreviation for Pythonista, it forms a concise and meaningful word with the following Sh part. So the name StaSh was chose to indicate it is a confined environment and great treasures may be found within.
Installation
StaSh can be easily installed via one line of python command (courtesy of @whitone).
import requests as r; exec(r.get('http://bit.ly/get-stash').text)
Simply copy the above line, paste into Pythonista interactive prompt and execute. It installs StaSh as a Python module under the
site-packages
folder (~/Documents/site-packages/stash
) and copies a launching script,~/Documents/launch_stash.py
for easy access.If you have previous versions of StaSh installed (e.g. v0.4.x), You may need to restart Pythonista BEFORE the installation.
If you have a GitHub tool available in Pythonista, such as gitview or gitrepo, you can choose to directly clone or download the repository.
StaSh requires Pythonista v2.0 as the new ObjC feature is heavily used. For older Pythonista 1.5 compatible version, please refer to the v0.4 branch.
Upgrade
Once StaSh is installed, it can be easily updated by running the
selfupdate
command from within the shell.selfupdate
defaults to themaster
branch. To update from a different branch, e.g.dev
, useselfupdate dev
.- By default,
selfupdate
compares local and remote versions and only performs update if newer version is found. You can however force the update without version checking viaselfupdate -f
. - To check for newer version without actually install it, use
selfupdate -n
. selfupdate
manages StaSh installation folder and may delete files in the process. It is therefore recommended to not place your own scripts under$STASH_ROOT/bin
. Instead, save your own scripts in~/Documents/bin
or customise the locations with theBIN_PATH
environment variable.- You may need to restart Pythonista after the update for changes to take full effects.
selfupdate cannot be used for version 0.4.x and under. A fresh installation is needed.
Notable Features
StaSh has a pile of features that are expected from a real shell. These
features are what really set the difference from shellista.-
Panel UI program that is completely event driven
- No blocking thread, builtin interactive prompt is accessible at all time
- Consistent look and feel as a proper PC terminal
- Almost all scripts can be called from within StaSh, including programs using UI and Scene packages.
- Attributed text (color and style) support
- Multiple sessions are possible by opening additional Panel tabs
- Being a pure UI program, it is possible to launch and forget. The program
stays active indefinitely. Non-UI scripts can only run for 10 minutes in background. But StaSh can stay up forever (till memory runs out due to other Apps). You can just launch StaSh to run a few commands and leave it.
It will still be there for you when you return later.
-
Comprehensive command line parsing and handling using pyparsing
- Environmental variables, e.g
echo $HOME
,NAME=value
- Aliases, e.g.
alias l1='ls -1'
- Single and double quotes behave like Bash, e.g.
"*"
means literal*
,"$HOME"
expands while'$HOME'
does not. - Backslash escaping, e.g.
ls My\ Script.py
- Glob, e.g.
ls ~/*.py
- Backtick quotes for subprocess, e.g.
touch `ls *.py`
- Pipes to chain commands, e.g.
find . -name "*.txt" | grep interesting
- IO redirect (actually just Output redirect), e.g.
ls *.py > py_files.txt
. Input redirect can be achieved by using pipes.- It is possible to redirect to the Pythonista builtin console,
e.g.ls > &3
- It is possible to redirect to the Pythonista builtin console,
- Bang(!) to search command history, e.g.
ls -1
,!l
. Bang commands like!!
and!-1
also works.
- Environmental variables, e.g
-
Smart auto-completion just as expected
- One UI button,
Tab
, is provided to enable command line auto-completion. - It is smart to complete either commands or files based on the cursor position
- It also completes environment variables and aliases.
- It also features a sub-command auto-completion system. For an example, type
git sta
and pressTab
. It will auto-completes togit status
. You can easily add your own sub-commands completion via JSON files.
- One UI button,
-
Thread management allows multiple commands running in parallel
- One foreground jobs and unlimited number of background jobs can run simultaneously.
- A foreground job can be stopped by pressing the CC button or Ctrl-C on an external keyboard.
- A background job is issued by appending an ampersand character (
&
) at the end of a normal command, e.g.httpserver &
. It can be terminated by thekill
command using its job ID. - A few utilities are provided for thread management.
jobs
to list current running background jobs.kill
to kill a running job.fg
to bring background jobs to foregroundCZ
button (Ctrl-Z) to send a foreground job to background
-
Command line history management. Three UI buttons are provided to navigate through the history.
-
On-screen virtual keys - an extra row of keys on top of the on-screen keyboard to provide control functions and easier access to symbols
- Virtual keys for control functions including:
- Tab - command line auto-completion
- CC (Ctrl-C) - terminate the running job
- CD (Ctrl-D) - end of Input
- CU (Ctrl-U) - kill line
- CZ (Ctrl-Z) - Send current running foreground job to background
- KB - show/hide keyboard
- H - display a popup window to show command history
- Up - recall the previous command in history
- Dn - recall the next command in history
- Customisable virtual keys for commonly used symbols, e.g.
~/.-*|>
.- The Symbols can be customized via the
VK_SYMBOLS
option in stash config file (default is.stash_config
).
- The Symbols can be customized via the
- Virtual keys for control functions including:
-
Swipe on the virtual key row to position cursor (similar to what Pythonista builtin editor offers)
-
External keyboard support
- Tab key for auto-completion
- Up (↑) / Down (↓) for navigating through command history
- Ctrl-A and Ctrl-E to jump to the beginning and end of the input line, respectively
- Ctrl-U to erase the input line
- Ctrl-W to erase one word before cursor
- Ctrl-L to clear the screen
-
You can run (almost) any regular python scripts from within StaSh
- There is no need to customize them for the shell. If it can be executed by a python interpreter via
python your_script.py
, you can just call it from within StaSh by just typingyour_script
- The shell object is made available to scripts being called. This enables a range of complex interactions between the shell and called scripts. For an example, the running script can use the shell object to execute more commands, e.g.
_stash('pwd')
.
- There is no need to customize them for the shell. If it can be executed by a python interpreter via
-
You can give it a resource file, similar to
.bashrc
, to customize its behaviour. Like the Bash resource file, aliases, environment variables can be set here. The default resource file is.stashrc
under StaSh installation root (i.e.~/Documents/site-packages/stash
).- The prompt is customizable with the
PROMPT
environment variable.\w
- current working directory with HOME folder abbreviated as~
\W
- last path component of current working directory- All other strings are displayed literally
- The default setting is
PROMPT='[\W]$ '
- The prompt is customizable with the
-
Easy self update to keep update with the development by running a single
selfupdate
command from within the shell. -
The UI can be configured via configuration file to customize its font size and color. The default config file is
.stash_config
orstash.cfg
under StaSh installation root.
Usage
The usage of StaSh is in principle similar to Bash. A few things to note are:
-
The search paths for executable scripts is set via an environment variable called
BIN_PATH
asPATH
is used by the system. The defaultBIN_PATH
is~/Documents/bin:$STASH_ROOT/bin
. -
The executable files are either Python scripts or StaSh scripts. The type of script is determined by looking at the file extensions ".py" and ".sh". A file without extension is considered as a shell script.
- When invoking a script, you can omit the extension, StaSh will try find the file with one of the extensions. For an example, StaSh interprets the command
selfupdate
and find the fileselfupdate.py
to execute. - Files without extension won't show up as an auto-completion possibility.
- When invoking a script, you can omit the extension, StaSh will try find the file with one of the extensions. For an example, StaSh interprets the command
-
Commands can only be written in a single line. No line continuation is possible. However, multiple commands can be written in a single line by separating them with semicolons, e.g.
ls -1 > file_list; cat file_list
. -
There are many Python scripts provided along with StaSh. These scripts range from performing regular shell tasks to advanced utilities like
ssh
andgit
. Note the scripts are by no means complete when compared to a real Linux shell. The collection will be gradually expanded should the need arise. It is also expected and appreciated that the community would come up with more scripts.alias.py
- Define or print aliasescat.py
- Print contents of filecd.py
- Change current directoryclear.py
- Clear consolecp.py
- Copy filecrypt.py
- File encryption using AES in CBC modecurl.py
- Transfer from an URLcut.py
- Cut out selection portions of each line of a filedu.py
- Summarize disk usage of the set of FILEs, recursively for directoriesecho.py
- Output text to consoleedit.py
- Open any text type files in Pythonista editorfind.py
- Powerful file searching toolfg.py
- Bring a background job to foreground- `gci.py - Interface to Python's built-in garbage collector
git.py
- Git client ported from shellistagrep.py
- search contents of file(s)httpserver.py
- A simple HTTP server with upload function (ripped from https://gist.github.com/UniIsland/3346170)jobs.py
- List all jobs that are currently runningkill.py
- Terminate a running jobls.py
- List filesmail.py
- Send emails with optional file attachmentman.py
- Show help message (docstring) of a given commandmc.py
- Easily work with multiple filesystems (e.g. local and FTP) synchronously.md5sum.py
- Print or check MD5 checksumsmkdir.py
- Create directorymv.py
- Move fileopenin.py
- Show the open in dialog to open a file in external apps.pbcopy.py
- Copy to iOS clipboardpbpaste.py
- Paste from iOS clipboardpip.py
- Search, download, install, update and uninstall pure Python packages from PyPI.printenv.py
- List environment variablesprinthex.py
- Print hexadecimal dump of the given filepwd.py
- Print current directorypython.py
- Run python scripts or modulesquicklook.py
- iOS quick look for files of known typesrm.py
- delete (remove) filescp.py
- Copy files from/to remote servers.selfupdate.py
- Update StaSh from its GitHub reposha1sum.py
- Print of check SHA1 checksumssha256sum.py
- Print of check SHA256 checksumssort.py
- Sort a list, also see uniquesource.py
- Evaluate a script in the current environmentssh.py
- SSH client to either execute a command or spawn an interactive session on remote servers. pyte is used for terminal emulation and gives the command the feel of a full-fledged SSH client.ssh-keygen.py
- Generate RSA/DSA SSH Keys.stashconf.py
- Change StaSh configuration on the flytar.py
- Manipulate archive filestouch.py
- Update timestamp of the given file or create it if not existuniq.py
- Remove duplicates from list, also see sortunzip.py
- Unzip file, also see zipversion.py
- Show StaSh installation and version informationwc.py
- Line, word, character countingwget.py
- get data from the netwhich.py
- Find the exact path to a command scriptwol.py
- Wake on LAN using MAC address for launching a sleeping systemxargs.py
- Command constructing and executing utilityzip.py
- Zip file, also see unzip
Acknowledgements
- Pythonista is a wonderful piece of software.
- StaSh is inspired by
shellista and its variants, including ShellistaExt and ShellistaUI. - The UI part of StaSh has its root from ShellistaUI.
- Many of the command scripts, e.g.
ls.py
,cp.py
,mv.py
, are taken from ShellistaExt with some modifications.
Known Issues
- Pickled objects are not restored correctly and generate
AttributeError
as if the class definition cannot be found.
Contributing
- Check any open issues or open a new issue to start discussions about your ideas of features and/or bugs
- Fork the repository, make changes, and send pull requests
- Please send pull requests to the dev branch instead of master
-
Well this is awkward, I've been working on basically the same thing, except with no UI and less functionality... Nicely done though, I have yet to play around with it some more, but having working pipes and some of the more advanced shell features sounds useful. And it looks like you figured out how to reliably scroll down the console output, the method I used in ShellistaUI always started to have problems after a hundred lines or so.
Regarding executable files, have you thought of using shebangs?
#!/bin/sh # ^ that is a shebang, it tells the shell which executable to use to run this file. # Normally the file must have the executable bit set for the shell to look at the shebang.
(I'll see about making a pull request so I can merge some of my code into your repo - your core shell is definitely better than mine, and since mine also loads commands from Python scripts the systems should be compatible.)
-
Woah! Thanks.
-
This is very nice. Thanks so much for contributing this.
You mention in your README that scripts can interact with the shell. How does that work? Sounds like it is intended to include scripts running a UI. I notice that when I open the history UI and then type a new command, the UI does not update. Could it? Is clicking on an item in the history list and example of script communication back to the shell?
-
My first attempt was trying to work from shellista. Then I realized it is impossible to achieve what I want without re-writing the whole thing. ShellistaUI points the right direction by employs the UI system. But it still depends on shellista and has many the same limitations. So I decided to start over as a new project.
Please be my guest to add your code to it. Sorry if it means double effort for you.The code was compiled in past 3 weeks. It is not clean or elegant. There sure is much room for improvements. In fact I wasn't entirely happy with the pyparsing bit and may have some major overhaul when I have the time.
I thought about shebang as well. I didn't implement it mainly because I am too lazy to do it. The parser in its current state does not parse comments, i.e.
#
. Also I am not sure how the performance would like if the Shell needs to read the first line of every file to tell the type. In Pythonista's controlled environment, I decided to take the lazy way and rely on file extensions. But, in fact, a file without extension will still run as StaSh script, e.gscript
will run if a file calledscript
is found. It will just not show up in the auto-completion possibilities when your typescri
and pressTab
.On a side note, I feel my development workflow is somewhat awkward. I effectively use GitHub as a staging area. Once I made changes on my computer, I push to GitHub and retrieve them using
selfupdate
on my phone. Please let me know if you have a better way to streamline the whole workflow. -
I have to admit I did most of the development on a Phone and haven't thought about updating the history UI in real-time while typing commands. It is definitely a nice feature and I'll work on it when time permits.
However, it is worth to note that the history UI is NOT an external script. It is an inherent part of the Shell. Yes it runs in its own thread. So it does communicate back to the Shell. But it is not exactly the interaction between Shell and an external script that mentioned in the README.
The interaction between shell and running external script are mainly for two type of things:
- Allow a running script access shell's status and act accordingly.
I also thought about (not implemented) a callback system that the running script can register to. For an example, a running script can register for being called when the shell's input textfield returns or a button tapped. One use of this feature is to allow running thread being terminated from the shell gracefully. But it does require the script to be written specifically to utilize this feature. For regular script, StaSh relies on
Thread._Thread__stop()
to terminate the running thread. It often works but not guaranteed.- Allow a running script to ask the shell to run another shell command
StaSh does not have a full set of shell language. It does not have keywords and program control constructs, such as
if
,for
. Hence a StaSh script can only do limited things (basically just a list of commands to be executed from the top to bottom. But with this feature, it is possible to write a Python script that acts like a real Shell script. Here is a contrived example:_stash = globals()['_stash'] # get the shell object # Commands to get a sequence of files from an url cmd_base = r'''wget http://www.test.com/file_%d.txt -o saved_%d.txt''' for i in range(10): cmd = cmd_base % (i, i) worker = _stash.runtime.run(cmd, add_to_history=False, reset_inp=False) if worker.isAlive(): # wait for the thread to finish as StaSh does not allow parallel threads pass
I confess it is rather a convoluted way to interact with the Shell. But it is available if you really wants it.
-
@ywangd this looks fantastic! My question may sound totally stupid to you, but i have no idea of what a 'shell' is, so could you put explainations (or a link to some) about the global context of your addon? You are giving a lot of explainations on the details, but not the global picture. Thanks!
-
Few comments
-
awesome work!
-
the slight flashing due to the multiple scrolls can be eliminated by using an ui.animation(). This probably deserves some more testing. I think the animate must force the drawing queue to finish before calling the animate function. Thus content_size gets properly refreshed the first time, without having to keep trying to scroll.
def _scroll_to_end(self): # small animation to force proper loading of content_size def ani(): self.out.content_offset = (0, self.out.content_size[1] - self.out.height) ui.animate(ani,0.01)
EDIT. Never mind, while this works for some cases, an empty line seems to not scroll to the end using this method. Oh well.
-
maybe consider adding some additional useful keys to virtual keyboard? On ios I find it particularly annoying to get to
_*|~>
, basically anything on the symbol page. -
the gist install script didn't install all of the scripts. I had to run the selfupdate script to get things like ls, etc. also, the gist installed to Documents/stash/bin, but the BIN_PATH pointed to /Documents/bin.
-
-
-
I just had some test using ui.animate. As you found out, it does not in all cases. I really like the effect that it scrolls really smoothly. Pity it still is not perfect.
-
Adding more buttons for symbols is a good idea. I am still clumsy with the ui system. But this addition should be really handy for the shell.
-
The gist script was intended to just install the minimal set of files so that
selfupdate
can run. This is because the gist script use a very primitive approach to issue an URL request for every single file (so it does not rely on unzip). So I decided to have a two stage installation. This is mentioned in my original post. Maybe I didn't emphasize it enough and it could cause confusions. -
The
BIN_PATH
is set to~/Documents/stash/bin:~/Documents/bin
if you typeecho $BIN_PATH
. However, due to a bug, the path~/Documents/bin
actually does Not work. SoBIN_PATH
is just~/Documents/stash/bin
in its fully expanded form. I have fixed the bug along with some other fixes. You canselfupdate
to get the latest changes.
Thanks for the comments.
-
-
A shell is a command line interpreter that sits between the end-user and operating system. It is a traditional text based user interface.
Here is the wiki page about the Unix Shell. Also a wiki page about Bash, often called the Shell as it is so influential (for people like C Shell, please, I am not trying to start a religious war here).
Graphic interfaces are so popular and dominant in current world. But a shell is still a very valuable tool, especially at performing administrative works. Basically if a program often needs to be called with very different arguments and it can benefit when used with other programs, a shell comes in handy.
Please note that I am talking about real Shells, e.g. Bash. I am not claiming that StaSh is a real shell. It just models after Bash and provide some opportunities for like-minded people. Personally I wanted a shell because I often need to run some CLI (command line interface) python tools, which could be cumbersome to run using action menus (because of constantly changing arguments).
-
@ywangd Very very nice. Thank you. I really like the console look. Would you mind If I borrowed some of the code? I am working on a console that connects to a Raspberry Pi via bluetooth.
Are you planning to add ssh access?
-
I've had a chance to play with it some. I'm really liking the layout. I do like how easy it is to add Python scripts. IT would be great to see a git , ssh client/server, Python interactive interpreter. I've added a small script that allows running of Python modules and scripts(main use is to call SimpleHTTPServer, Pylint, nose.
I haven't looked through the code to deeply but what would be the best way to take over the UI input for a Python script such as an ssh client.
With a few additions I will be using this instead of shellista. Great Job.
-
@ywangd thanks a lot for your explanations.
-
Thanks for your comments. Please feel free to use any code from StaSh. The UI layout has its root from ShellistaUI. So credits also goes to ShellistaUI (I believe @dgelessus is the original author of ShellistaUI). I also borrowed code from your ShellistaExt for various commands such as
ls
,cp
etc. It would be even more appreciated if you could share your changes or even send pull requests.-
StaSh is more of a framework that manages calls to python scripts. So I am hoping that the community would provide the scripts. In principle, if a script can be executed on a PC, it would also be usable from within StaSh. I/O should be taken care of automatically in most cases.
-
For an example, your
ssh
code (https://github.com/briarfox/ShellistaExt/blob/master/ShellistaExt/plugins/extensions/ssh/ssh.py) is pretty immediately usable in StaSh. Just place it into theBIN_PATH
, e.g.~/Documents/stash/bin
and just typessh
in Stash. You'll see the command gets executed. If you set proper host/user/pass in__main__
, it would connect to the host and all I/O is handled by StaSh. Apparently, for the script to be more useful, you'll need some arguments handling in__main__
, a primitive approach could be as follows (add to the end ofssh.py
):
if __name__ == '__main__': if len(sys.argv) == 1: print 'Usage: connect hostname user password' sys.exit(0) cmd = sys.argv[1] params = ' '.join(sys.argv[2:]) if cmd == 'connect': ssh = SSH() ssh.connect(params) else: print 'NYI'
-
Pythonista builtin interactive prompt is available even when StaSh is running. Just tap on the first button of the segmentation control on the top-left side. Sure you can add your own interactive interpreter, but the builtin one just has more features such as code completion etc.
-
I haven't thought about running Python modules. I would like to see SimpleHTTPServer gets run in StaSh. Please let me know your changes if it is OK with you.
-
I haven't had my hands on a Raspberry Pi yet. But I am a Lego Mindstorms fan. I'd love to know how your code works and see if it is adaptable to Mindstorms.
-
-
@ywangd I do like the drag and drop with stash. I'm still working on the running of modules. Code works great until I drop it in StaSh. I keep getting a package set to non string Error. I need to track down the cause. I'm hoping to run it on its own thread so the command:
<pre>
python -m SimpleHTTPServer &
</pre>
Will run it as a thread and allow you to load a client to work with it. It's a simple script that uses the module runpy and passes the args. I'll submit a pull request when I get the bugs out.edit Fixed
-
@ywangd Would you mind checking out this script? It works when ran my itself. But ShaSh when trying:
<pre>
python -m SimpleHTTPServer &
</pre>
It throws a "package set to non string"
python.pyAs for the question on the Rpi and BTLE, I have a bluetooth adapter on the Rx and Tx pins of the RPi. I'm just reading and sending data with the pythonista cb beta module. I'll post it up for you.
-
This error is tricky. It is due to the
module_name
variable being unicode type instead of an plain ASCII str. So once you addmodule_name = str(module_name)
into the code, everything works.
This error is probably due to the fact that Python 2.x and its libraries are not fully unicode compliant. In fact, if you type
runpy.run_module(u'SimpleHTTPServer', run_name='__main__')
into a PC version python interpreter, it reports exactly the same error.When the script runs by itself,
sys.argv
is set to plain strings. But it is set to the same strings of unicode version when executed by StaSh. I am not entirely clear about the reason. Anyway, the fix is easy. I'll also see if it is worthwhile to always convert all arguments tostr
before calling an external script. -
@ywangd, the
ui
module usesunicode
strings exclusively. Since the command input field is just a regular text field, it also returns its contents asunicode
, and since there's no reason for Python to coerce down tostr
it doesn't. Also instead of a plainstr()
conversion it would be better to use theencode()
method ofunicode
objects, that way the string is guaranteed to be properly encoded as UTF-8. -
@dgelessus Thanks for the explaination. That makes sense.
I am still thinking about whether it is really necessary to always encode unicode arguments to str before executing a command.
-
@ywangd, it isn't really necessary, because you'll almost never need to pass non-ASCII characters to a command. But when you do try, you'll very likely run into problems. I'm not sure what exactly happens when you do
str(u"üñîçø∂ε")
.encode
is also guaranteed to work in future Python versions. Python 3 (if/when Pythonista supports it) is much more strict when it comes to byte data and unicode text.I checked, and Python 2.7 on my Raspberry Pi encodes runtime args as UTF-8
str
objects. It is possible that the encoding is OS-dependent, but UTF-8 is usually a safe choice. Python 3 of course uses proper Unicodestr
objects instead.PS:
You're lucky,
str
encodesunicode
strings as UTF-8 anyway.encode
still is safer ;)