Welcome!
This is the community forum for my apps Pythonista and Editorial.
For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.
StaSh - Shell Like an Expert in Pythonista
-
My first attempt was trying to work from shellista. Then I realized it is impossible to achieve what I want without re-writing the whole thing. ShellistaUI points the right direction by employs the UI system. But it still depends on shellista and has many the same limitations. So I decided to start over as a new project.
Please be my guest to add your code to it. Sorry if it means double effort for you.The code was compiled in past 3 weeks. It is not clean or elegant. There sure is much room for improvements. In fact I wasn't entirely happy with the pyparsing bit and may have some major overhaul when I have the time.
I thought about shebang as well. I didn't implement it mainly because I am too lazy to do it. The parser in its current state does not parse comments, i.e.
#
. Also I am not sure how the performance would like if the Shell needs to read the first line of every file to tell the type. In Pythonista's controlled environment, I decided to take the lazy way and rely on file extensions. But, in fact, a file without extension will still run as StaSh script, e.gscript
will run if a file calledscript
is found. It will just not show up in the auto-completion possibilities when your typescri
and pressTab
.On a side note, I feel my development workflow is somewhat awkward. I effectively use GitHub as a staging area. Once I made changes on my computer, I push to GitHub and retrieve them using
selfupdate
on my phone. Please let me know if you have a better way to streamline the whole workflow. -
I have to admit I did most of the development on a Phone and haven't thought about updating the history UI in real-time while typing commands. It is definitely a nice feature and I'll work on it when time permits.
However, it is worth to note that the history UI is NOT an external script. It is an inherent part of the Shell. Yes it runs in its own thread. So it does communicate back to the Shell. But it is not exactly the interaction between Shell and an external script that mentioned in the README.
The interaction between shell and running external script are mainly for two type of things:
- Allow a running script access shell's status and act accordingly.
I also thought about (not implemented) a callback system that the running script can register to. For an example, a running script can register for being called when the shell's input textfield returns or a button tapped. One use of this feature is to allow running thread being terminated from the shell gracefully. But it does require the script to be written specifically to utilize this feature. For regular script, StaSh relies on
Thread._Thread__stop()
to terminate the running thread. It often works but not guaranteed.- Allow a running script to ask the shell to run another shell command
StaSh does not have a full set of shell language. It does not have keywords and program control constructs, such as
if
,for
. Hence a StaSh script can only do limited things (basically just a list of commands to be executed from the top to bottom. But with this feature, it is possible to write a Python script that acts like a real Shell script. Here is a contrived example:_stash = globals()['_stash'] # get the shell object # Commands to get a sequence of files from an url cmd_base = r'''wget http://www.test.com/file_%d.txt -o saved_%d.txt''' for i in range(10): cmd = cmd_base % (i, i) worker = _stash.runtime.run(cmd, add_to_history=False, reset_inp=False) if worker.isAlive(): # wait for the thread to finish as StaSh does not allow parallel threads pass
I confess it is rather a convoluted way to interact with the Shell. But it is available if you really wants it.
-
@ywangd this looks fantastic! My question may sound totally stupid to you, but i have no idea of what a 'shell' is, so could you put explainations (or a link to some) about the global context of your addon? You are giving a lot of explainations on the details, but not the global picture. Thanks!
-
Few comments
-
awesome work!
-
the slight flashing due to the multiple scrolls can be eliminated by using an ui.animation(). This probably deserves some more testing. I think the animate must force the drawing queue to finish before calling the animate function. Thus content_size gets properly refreshed the first time, without having to keep trying to scroll.
def _scroll_to_end(self): # small animation to force proper loading of content_size def ani(): self.out.content_offset = (0, self.out.content_size[1] - self.out.height) ui.animate(ani,0.01)
EDIT. Never mind, while this works for some cases, an empty line seems to not scroll to the end using this method. Oh well.
-
maybe consider adding some additional useful keys to virtual keyboard? On ios I find it particularly annoying to get to
_*|~>
, basically anything on the symbol page. -
the gist install script didn't install all of the scripts. I had to run the selfupdate script to get things like ls, etc. also, the gist installed to Documents/stash/bin, but the BIN_PATH pointed to /Documents/bin.
-
-
-
I just had some test using ui.animate. As you found out, it does not in all cases. I really like the effect that it scrolls really smoothly. Pity it still is not perfect.
-
Adding more buttons for symbols is a good idea. I am still clumsy with the ui system. But this addition should be really handy for the shell.
-
The gist script was intended to just install the minimal set of files so that
selfupdate
can run. This is because the gist script use a very primitive approach to issue an URL request for every single file (so it does not rely on unzip). So I decided to have a two stage installation. This is mentioned in my original post. Maybe I didn't emphasize it enough and it could cause confusions. -
The
BIN_PATH
is set to~/Documents/stash/bin:~/Documents/bin
if you typeecho $BIN_PATH
. However, due to a bug, the path~/Documents/bin
actually does Not work. SoBIN_PATH
is just~/Documents/stash/bin
in its fully expanded form. I have fixed the bug along with some other fixes. You canselfupdate
to get the latest changes.
Thanks for the comments.
-
-
A shell is a command line interpreter that sits between the end-user and operating system. It is a traditional text based user interface.
Here is the wiki page about the Unix Shell. Also a wiki page about Bash, often called the Shell as it is so influential (for people like C Shell, please, I am not trying to start a religious war here).
Graphic interfaces are so popular and dominant in current world. But a shell is still a very valuable tool, especially at performing administrative works. Basically if a program often needs to be called with very different arguments and it can benefit when used with other programs, a shell comes in handy.
Please note that I am talking about real Shells, e.g. Bash. I am not claiming that StaSh is a real shell. It just models after Bash and provide some opportunities for like-minded people. Personally I wanted a shell because I often need to run some CLI (command line interface) python tools, which could be cumbersome to run using action menus (because of constantly changing arguments).
-
@ywangd Very very nice. Thank you. I really like the console look. Would you mind If I borrowed some of the code? I am working on a console that connects to a Raspberry Pi via bluetooth.
Are you planning to add ssh access?
-
I've had a chance to play with it some. I'm really liking the layout. I do like how easy it is to add Python scripts. IT would be great to see a git , ssh client/server, Python interactive interpreter. I've added a small script that allows running of Python modules and scripts(main use is to call SimpleHTTPServer, Pylint, nose.
I haven't looked through the code to deeply but what would be the best way to take over the UI input for a Python script such as an ssh client.
With a few additions I will be using this instead of shellista. Great Job.
-
@ywangd thanks a lot for your explanations.
-
Thanks for your comments. Please feel free to use any code from StaSh. The UI layout has its root from ShellistaUI. So credits also goes to ShellistaUI (I believe @dgelessus is the original author of ShellistaUI). I also borrowed code from your ShellistaExt for various commands such as
ls
,cp
etc. It would be even more appreciated if you could share your changes or even send pull requests.-
StaSh is more of a framework that manages calls to python scripts. So I am hoping that the community would provide the scripts. In principle, if a script can be executed on a PC, it would also be usable from within StaSh. I/O should be taken care of automatically in most cases.
-
For an example, your
ssh
code (https://github.com/briarfox/ShellistaExt/blob/master/ShellistaExt/plugins/extensions/ssh/ssh.py) is pretty immediately usable in StaSh. Just place it into theBIN_PATH
, e.g.~/Documents/stash/bin
and just typessh
in Stash. You'll see the command gets executed. If you set proper host/user/pass in__main__
, it would connect to the host and all I/O is handled by StaSh. Apparently, for the script to be more useful, you'll need some arguments handling in__main__
, a primitive approach could be as follows (add to the end ofssh.py
):
if __name__ == '__main__': if len(sys.argv) == 1: print 'Usage: connect hostname user password' sys.exit(0) cmd = sys.argv[1] params = ' '.join(sys.argv[2:]) if cmd == 'connect': ssh = SSH() ssh.connect(params) else: print 'NYI'
-
Pythonista builtin interactive prompt is available even when StaSh is running. Just tap on the first button of the segmentation control on the top-left side. Sure you can add your own interactive interpreter, but the builtin one just has more features such as code completion etc.
-
I haven't thought about running Python modules. I would like to see SimpleHTTPServer gets run in StaSh. Please let me know your changes if it is OK with you.
-
I haven't had my hands on a Raspberry Pi yet. But I am a Lego Mindstorms fan. I'd love to know how your code works and see if it is adaptable to Mindstorms.
-
-
@ywangd I do like the drag and drop with stash. I'm still working on the running of modules. Code works great until I drop it in StaSh. I keep getting a package set to non string Error. I need to track down the cause. I'm hoping to run it on its own thread so the command:
<pre>
python -m SimpleHTTPServer &
</pre>
Will run it as a thread and allow you to load a client to work with it. It's a simple script that uses the module runpy and passes the args. I'll submit a pull request when I get the bugs out.edit Fixed
-
@ywangd Would you mind checking out this script? It works when ran my itself. But ShaSh when trying:
<pre>
python -m SimpleHTTPServer &
</pre>
It throws a "package set to non string"
python.pyAs for the question on the Rpi and BTLE, I have a bluetooth adapter on the Rx and Tx pins of the RPi. I'm just reading and sending data with the pythonista cb beta module. I'll post it up for you.
-
This error is tricky. It is due to the
module_name
variable being unicode type instead of an plain ASCII str. So once you addmodule_name = str(module_name)
into the code, everything works.
This error is probably due to the fact that Python 2.x and its libraries are not fully unicode compliant. In fact, if you type
runpy.run_module(u'SimpleHTTPServer', run_name='__main__')
into a PC version python interpreter, it reports exactly the same error.When the script runs by itself,
sys.argv
is set to plain strings. But it is set to the same strings of unicode version when executed by StaSh. I am not entirely clear about the reason. Anyway, the fix is easy. I'll also see if it is worthwhile to always convert all arguments tostr
before calling an external script. -
@ywangd, the
ui
module usesunicode
strings exclusively. Since the command input field is just a regular text field, it also returns its contents asunicode
, and since there's no reason for Python to coerce down tostr
it doesn't. Also instead of a plainstr()
conversion it would be better to use theencode()
method ofunicode
objects, that way the string is guaranteed to be properly encoded as UTF-8. -
@dgelessus Thanks for the explaination. That makes sense.
I am still thinking about whether it is really necessary to always encode unicode arguments to str before executing a command.
-
@ywangd, it isn't really necessary, because you'll almost never need to pass non-ASCII characters to a command. But when you do try, you'll very likely run into problems. I'm not sure what exactly happens when you do
str(u"üñîçø∂ε")
.encode
is also guaranteed to work in future Python versions. Python 3 (if/when Pythonista supports it) is much more strict when it comes to byte data and unicode text.I checked, and Python 2.7 on my Raspberry Pi encodes runtime args as UTF-8
str
objects. It is possible that the encoding is OS-dependent, but UTF-8 is usually a safe choice. Python 3 of course uses proper Unicodestr
objects instead.PS:
You're lucky,
str
encodesunicode
strings as UTF-8 anyway.encode
still is safer ;) -
@ywand Thank you very much, can't believe I missed that. Spend hours looking for that problem.
It seems that '&' is not passed by the input. I was using that to flag its own thread. I added it to _word_chars and it now passes &. Is it held for a reason?
@dgelessus Thanks for the explanation
-
This is a very nice piece of work and quite useful. I use a BT keyboard on my iPad and I am wondering if there is any way that the special keys (e.g., Tab, Up, Ctrl-C) could be made to work from the keyboard?
-
I just submited a pull request.
I've added:
- man.py - lists programs in /bin and displays the docstring.
- edit.py - opens a file in the pythonista editor
- python.py - allows execution of .py files and modules. Basicly allows the -m flag with python. You can run SimpleHTTPServer, pylint, unittests etc.
- added '&' to _word_chars in stash.py - I needed this to allow running a python script in it's own thread emulating bash &.
-
@briarfox - what are the references to Shash and "drag and drop" talking about in your prior posts? Sounds like some other shell where you drag and drop scripts into a window or something. Did you mean StaSh and dnd means something other then what I think it does?