StaSh - Shell Like an Expert in Pythonista

JonB

Few comments

awesome work!
the slight flashing due to the multiple scrolls can be eliminated by using an ui.animation(). This probably deserves some more testing. I think the animate must force the drawing queue to finish before calling the animate function. Thus content_size gets properly refreshed the first time, without having to keep trying to scroll.

    def _scroll_to_end(self):
        # small animation to force proper loading of content_size
        def ani():
            self.out.content_offset = (0, self.out.content_size[1] - self.out.height)
        ui.animate(ani,0.01)

EDIT. Never mind, while this works for some cases, an empty line seems to not scroll to the end using this method. Oh well.

maybe consider adding some additional useful keys to virtual keyboard? On ios I find it particularly annoying to get to _*|~>, basically anything on the symbol page.
the gist install script didn't install all of the scripts. I had to run the selfupdate script to get things like ls, etc. also, the gist installed to Documents/stash/bin, but the BIN_PATH pointed to /Documents/bin.

ywangd

@JonB

I just had some test using ui.animate. As you found out, it does not in all cases. I really like the effect that it scrolls really smoothly. Pity it still is not perfect.
Adding more buttons for symbols is a good idea. I am still clumsy with the ui system. But this addition should be really handy for the shell.
The gist script was intended to just install the minimal set of files so that selfupdate can run. This is because the gist script use a very primitive approach to issue an URL request for every single file (so it does not rely on unzip). So I decided to have a two stage installation. This is mentioned in my original post. Maybe I didn't emphasize it enough and it could cause confusions.
The BIN_PATH is set to ~/Documents/stash/bin:~/Documents/bin if you type echo $BIN_PATH. However, due to a bug, the path ~/Documents/bin actually does Not work. So BIN_PATH is just ~/Documents/stash/bin in its fully expanded form. I have fixed the bug along with some other fixes. You can selfupdate to get the latest changes.

Thanks for the comments.

ywangd

@jmv38

A shell is a command line interpreter that sits between the end-user and operating system. It is a traditional text based user interface.

Here is the wiki page about the Unix Shell. Also a wiki page about Bash, often called the Shell as it is so influential (for people like C Shell, please, I am not trying to start a religious war here).

Graphic interfaces are so popular and dominant in current world. But a shell is still a very valuable tool, especially at performing administrative works. Basically if a program often needs to be called with very different arguments and it can benefit when used with other programs, a shell comes in handy.

Please note that I am talking about real Shells, e.g. Bash. I am not claiming that StaSh is a real shell. It just models after Bash and provide some opportunities for like-minded people. Personally I wanted a shell because I often need to run some CLI (command line interface) python tools, which could be cumbersome to run using action menus (because of constantly changing arguments).

briarfox

@ywangd Very very nice. Thank you. I really like the console look. Would you mind If I borrowed some of the code? I am working on a console that connects to a Raspberry Pi via bluetooth.

Are you planning to add ssh access?

briarfox

I've had a chance to play with it some. I'm really liking the layout. I do like how easy it is to add Python scripts. IT would be great to see a git , ssh client/server, Python interactive interpreter. I've added a small script that allows running of Python modules and scripts(main use is to call SimpleHTTPServer, Pylint, nose.

I haven't looked through the code to deeply but what would be the best way to take over the UI input for a Python script such as an ssh client.

With a few additions I will be using this instead of shellista. Great Job.

jmv38

@ywangd thanks a lot for your explanations.

ywangd

@briarfox

Thanks for your comments. Please feel free to use any code from StaSh. The UI layout has its root from ShellistaUI. So credits also goes to ShellistaUI (I believe @dgelessus is the original author of ShellistaUI). I also borrowed code from your ShellistaExt for various commands such as ls, cp etc. It would be even more appreciated if you could share your changes or even send pull requests.

StaSh is more of a framework that manages calls to python scripts. So I am hoping that the community would provide the scripts. In principle, if a script can be executed on a PC, it would also be usable from within StaSh. I/O should be taken care of automatically in most cases.
For an example, your ssh code (https://github.com/briarfox/ShellistaExt/blob/master/ShellistaExt/plugins/extensions/ssh/ssh.py) is pretty immediately usable in StaSh. Just place it into the BIN_PATH, e.g. ~/Documents/stash/bin and just type ssh in Stash. You'll see the command gets executed. If you set proper host/user/pass in __main__, it would connect to the host and all I/O is handled by StaSh. Apparently, for the script to be more useful, you'll need some arguments handling in __main__, a primitive approach could be as follows (add to the end of ssh.py):

if __name__ == '__main__':
    if len(sys.argv) == 1:
        print 'Usage: connect hostname user password'
        sys.exit(0)
    cmd = sys.argv[1]
    params = ' '.join(sys.argv[2:])
    if cmd == 'connect':
        ssh = SSH()
        ssh.connect(params)
    else:
        print 'NYI'

Pythonista builtin interactive prompt is available even when StaSh is running. Just tap on the first button of the segmentation control on the top-left side. Sure you can add your own interactive interpreter, but the builtin one just has more features such as code completion etc.
I haven't thought about running Python modules. I would like to see SimpleHTTPServer gets run in StaSh. Please let me know your changes if it is OK with you.
I haven't had my hands on a Raspberry Pi yet. But I am a Lego Mindstorms fan. I'd love to know how your code works and see if it is adaptable to Mindstorms.

briarfox

@ywangd I do like the drag and drop with stash. I'm still working on the running of modules. Code works great until I drop it in StaSh. I keep getting a package set to non string Error. I need to track down the cause. I'm hoping to run it on its own thread so the command:
<pre>
python -m SimpleHTTPServer &
</pre>
Will run it as a thread and allow you to load a client to work with it. It's a simple script that uses the module runpy and passes the args. I'll submit a pull request when I get the bugs out.

edit Fixed

briarfox

@ywangd Would you mind checking out this script? It works when ran my itself. But ShaSh when trying:
<pre>
python -m SimpleHTTPServer &
</pre>
It throws a "package set to non string"
python.py

As for the question on the Rpi and BTLE, I have a bluetooth adapter on the Rx and Tx pins of the RPi. I'm just reading and sending data with the pythonista cb beta module. I'll post it up for you.

ywangd

@briarfox

This error is tricky. It is due to the module_name variable being unicode type instead of an plain ASCII str. So once you add

module_name = str(module_name)

into the code, everything works.

This error is probably due to the fact that Python 2.x and its libraries are not fully unicode compliant. In fact, if you type runpy.run_module(u'SimpleHTTPServer', run_name='__main__') into a PC version python interpreter, it reports exactly the same error.

When the script runs by itself, sys.argv is set to plain strings. But it is set to the same strings of unicode version when executed by StaSh. I am not entirely clear about the reason. Anyway, the fix is easy. I'll also see if it is worthwhile to always convert all arguments to str before calling an external script.

dgelessus

@ywangd, the ui module uses unicode strings exclusively. Since the command input field is just a regular text field, it also returns its contents as unicode, and since there's no reason for Python to coerce down to str it doesn't. Also instead of a plain str() conversion it would be better to use the encode() method of unicode objects, that way the string is guaranteed to be properly encoded as UTF-8.

ywangd

@dgelessus Thanks for the explaination. That makes sense.

I am still thinking about whether it is really necessary to always encode unicode arguments to str before executing a command.

dgelessus

@ywangd, it isn't really necessary, because you'll almost never need to pass non-ASCII characters to a command. But when you do try, you'll very likely run into problems. I'm not sure what exactly happens when you do str(u"üñîçø∂ε"). encode is also guaranteed to work in future Python versions. Python 3 (if/when Pythonista supports it) is much more strict when it comes to byte data and unicode text.

I checked, and Python 2.7 on my Raspberry Pi encodes runtime args as UTF-8 str objects. It is possible that the encoding is OS-dependent, but UTF-8 is usually a safe choice. Python 3 of course uses proper Unicode str objects instead.

PS:

You're lucky, str encodes unicode strings as UTF-8 anyway. encode still is safer ;)

briarfox

@ywand Thank you very much, can't believe I missed that. Spend hours looking for that problem.

It seems that '&' is not passed by the input. I was using that to flag its own thread. I added it to _word_chars and it now passes &. Is it held for a reason?

@dgelessus Thanks for the explanation

ihf

This is a very nice piece of work and quite useful. I use a BT keyboard on my iPad and I am wondering if there is any way that the special keys (e.g., Tab, Up, Ctrl-C) could be made to work from the keyboard?

briarfox

I just submited a pull request.

I've added:

man.py - lists programs in /bin and displays the docstring.
edit.py - opens a file in the pythonista editor
python.py - allows execution of .py files and modules. Basicly allows the -m flag with python. You can run SimpleHTTPServer, pylint, unittests etc.
added '&' to _word_chars in stash.py - I needed this to allow running a python script in it's own thread emulating bash &.

wradcliffe

@briarfox - what are the references to Shash and "drag and drop" talking about in your prior posts? Sounds like some other shell where you drag and drop scripts into a window or something. Did you mean StaSh and dnd means something other then what I think it does?

briarfox

@wradcliffe Sorry I was being a bit dyslexic. By Dnd I was referring to adding scripts to StaSh/bin. As in it's easy to extend. Probably not the correct use of the term.

ywangd

@briarfox

Thanks a lot for your pull request. I made some changes to python.py after the merge. In short, the script now takes a flag -b to indicate running as background thread instead of &. I used argparse module for argument processing. It even provides free help message if you type python -h.

I personally think the & character shall be handled by the shell instead of the script gets called. Because otherwise every script has to check for & to decide if it runs in background. This task should be handled by shell/OS instead of individual script.

My current preference is to not allow background jobs (commands end with a &). This is mainly because it does not seem possible to end a thread gracefully from another (main) thread. Thread._Thread__stop() can be used and often works, but it does not perform any housekeeping tasks (e.g. try: ... finally: block is not honoured). It just ends the thread abruptly and leaves a somewhat messy state, for example, the socket port will not be released if a http server thread is ended this way. Of course, an external can still spawn threads on its own and StaSh has no control of it since it is not really an OS.

The shell in fact should have reported parsing error when sees a &. It was actually a bug that causes the shell ignore the character instead of erring out (it is now fixed). This character is Not part of _word_char which is consistent as a real shell. It is a punctuator, similar to ;, to separate commands. I am still debating whether background job shall be directly allowed by StaSh. But as for now, I prefer to leave it out.

I hope these changes are OK with you. Thanks!

ywangd

@ihf

Sorry but I never had a chance to use an external keyboard. So I won't be of much help on this. Hopefully someone else would come up with a solution. Otherwise it'll have to wait till I can get a keyboard ...