StaSh - Shell Like an Expert in Pythonista

[briarfox]

@ywangd Very very nice. Thank you. I really like the console look. Would you mind If I borrowed some of the code? I am working on a console that connects to a Raspberry Pi via bluetooth.

Are you planning to add ssh access?

[briarfox]

I've had a chance to play with it some. I'm really liking the layout. I do like how easy it is to add Python scripts. IT would be great to see a git , ssh client/server, Python interactive interpreter. I've added a small script that allows running of Python modules and scripts(main use is to call SimpleHTTPServer, Pylint, nose.

I haven't looked through the code to deeply but what would be the best way to take over the UI input for a Python script such as an ssh client.

With a few additions I will be using this instead of shellista. Great Job.

[jmv38]

@ywangd thanks a lot for your explanations.

[ywangd]

@briarfox

Thanks for your comments. Please feel free to use any code from StaSh. The UI layout has its root from ShellistaUI. So credits also goes to ShellistaUI (I believe @dgelessus is the original author of ShellistaUI). I also borrowed code from your ShellistaExt for various commands such as ls, cp etc. It would be even more appreciated if you could share your changes or even send pull requests.

• StaSh is more of a framework that manages calls to python scripts. So I am hoping that the community would provide the scripts. In principle, if a script can be executed on a PC, it would also be usable from within StaSh. I/O should be taken care of automatically in most cases.

• For an example, your ssh code (https://github.com/briarfox/ShellistaExt/blob/master/ShellistaExt/plugins/extensions/ssh/ssh.py) is pretty immediately usable in StaSh. Just place it into the BIN_PATH, e.g. ~/Documents/stash/bin and just type ssh in Stash. You'll see the command gets executed. If you set proper host/user/pass in __main__, it would connect to the host and all I/O is handled by StaSh. Apparently, for the script to be more useful, you'll need some arguments handling in __main__, a primitive approach could be as follows (add to the end of ssh.py):

if __name__ == '__main__':
    if len(sys.argv) == 1:
        print 'Usage: connect hostname user password'
        sys.exit(0)
    cmd = sys.argv[1]
    params = ' '.join(sys.argv[2:])
    if cmd == 'connect':
        ssh = SSH()
        ssh.connect(params)
    else:
        print 'NYI'

• Pythonista builtin interactive prompt is available even when StaSh is running. Just tap on the first button of the segmentation control on the top-left side. Sure you can add your own interactive interpreter, but the builtin one just has more features such as code completion etc.

• I haven't thought about running Python modules. I would like to see SimpleHTTPServer gets run in StaSh. Please let me know your changes if it is OK with you.

• I haven't had my hands on a Raspberry Pi yet. But I am a Lego Mindstorms fan. I'd love to know how your code works and see if it is adaptable to Mindstorms.

[briarfox]

@ywangd I do like the drag and drop with stash. I'm still working on the running of modules. Code works great until I drop it in StaSh. I keep getting a package set to non string Error. I need to track down the cause. I'm hoping to run it on its own thread so the command:
<pre>
python -m SimpleHTTPServer &
</pre>
Will run it as a thread and allow you to load a client to work with it. It's a simple script that uses the module runpy and passes the args. I'll submit a pull request when I get the bugs out.

edit Fixed

[briarfox]

@ywangd Would you mind checking out this script? It works when ran my itself. But ShaSh when trying:
<pre>
python -m SimpleHTTPServer &
</pre>
It throws a "package set to non string"
python.py

As for the question on the Rpi and BTLE, I have a bluetooth adapter on the Rx and Tx pins of the RPi. I'm just reading and sending data with the pythonista cb beta module. I'll post it up for you.

[ywangd]

@briarfox

This error is tricky. It is due to the module_name variable being unicode type instead of an plain ASCII str. So once you add

module_name = str(module_name)

into the code, everything works.

This error is probably due to the fact that Python 2.x and its libraries are not fully unicode compliant. In fact, if you type runpy.run_module(u'SimpleHTTPServer', run_name='__main__') into a PC version python interpreter, it reports exactly the same error.

When the script runs by itself, sys.argv is set to plain strings. But it is set to the same strings of unicode version when executed by StaSh. I am not entirely clear about the reason. Anyway, the fix is easy. I'll also see if it is worthwhile to always convert all arguments to str before calling an external script.

[dgelessus]

@ywangd, the ui module uses unicode strings exclusively. Since the command input field is just a regular text field, it also returns its contents as unicode, and since there's no reason for Python to coerce down to str it doesn't. Also instead of a plain str() conversion it would be better to use the encode() method of unicode objects, that way the string is guaranteed to be properly encoded as UTF-8.

[ywangd]

@dgelessus Thanks for the explaination. That makes sense.

I am still thinking about whether it is really necessary to always encode unicode arguments to str before executing a command.

[dgelessus]

@ywangd, it isn't really necessary, because you'll almost never need to pass non-ASCII characters to a command. But when you do try, you'll very likely run into problems. I'm not sure what exactly happens when you do str(u"üñîçø∂ε"). encode is also guaranteed to work in future Python versions. Python 3 (if/when Pythonista supports it) is much more strict when it comes to byte data and unicode text.

I checked, and Python 2.7 on my Raspberry Pi encodes runtime args as UTF-8 str objects. It is possible that the encoding is OS-dependent, but UTF-8 is usually a safe choice. Python 3 of course uses proper Unicode str objects instead.

PS:

You're lucky, str encodes unicode strings as UTF-8 anyway. encode still is safer ;)

[briarfox]

@ywand Thank you very much, can't believe I missed that. Spend hours looking for that problem.

It seems that '&' is not passed by the input. I was using that to flag its own thread. I added it to _word_chars and it now passes &. Is it held for a reason?

@dgelessus Thanks for the explanation

[ihf]

This is a very nice piece of work and quite useful. I use a BT keyboard on my iPad and I am wondering if there is any way that the special keys (e.g., Tab, Up, Ctrl-C) could be made to work from the keyboard?

[briarfox]

I just submited a pull request.

I've added:

• man.py - lists programs in /bin and displays the docstring.
• edit.py - opens a file in the pythonista editor
• python.py - allows execution of .py files and modules. Basicly allows the -m flag with python. You can run SimpleHTTPServer, pylint, unittests etc.
• added '&' to _word_chars in stash.py - I needed this to allow running a python script in it's own thread emulating bash &.

[wradcliffe]

@briarfox - what are the references to Shash and "drag and drop" talking about in your prior posts? Sounds like some other shell where you drag and drop scripts into a window or something. Did you mean StaSh and dnd means something other then what I think it does?

[briarfox]

@wradcliffe Sorry I was being a bit dyslexic. By Dnd I was referring to adding scripts to StaSh/bin. As in it's easy to extend. Probably not the correct use of the term.

[ywangd]

@briarfox

Thanks a lot for your pull request. I made some changes to python.py after the merge. In short, the script now takes a flag -b to indicate running as background thread instead of &. I used argparse module for argument processing. It even provides free help message if you type python -h.

I personally think the & character shall be handled by the shell instead of the script gets called. Because otherwise every script has to check for & to decide if it runs in background. This task should be handled by shell/OS instead of individual script.

My current preference is to not allow background jobs (commands end with a &). This is mainly because it does not seem possible to end a thread gracefully from another (main) thread. Thread._Thread__stop() can be used and often works, but it does not perform any housekeeping tasks (e.g. try: ... finally: block is not honoured). It just ends the thread abruptly and leaves a somewhat messy state, for example, the socket port will not be released if a http server thread is ended this way. Of course, an external can still spawn threads on its own and StaSh has no control of it since it is not really an OS.

The shell in fact should have reported parsing error when sees a &. It was actually a bug that causes the shell ignore the character instead of erring out (it is now fixed). This character is Not part of _word_char which is consistent as a real shell. It is a punctuator, similar to ;, to separate commands. I am still debating whether background job shall be directly allowed by StaSh. But as for now, I prefer to leave it out.

I hope these changes are OK with you. Thanks!

[ywangd]

@ihf

Sorry but I never had a chance to use an external keyboard. So I won't be of much help on this. Hopefully someone else would come up with a solution. Otherwise it'll have to wait till I can get a keyboard ...

[dgelessus]

@ihf re. keyboard, that is possible in a very limited way. See KeyboardControl.py, it uses a delegate and a specifically filled text field to deduce the user's keystrokes. It has issues with some of the more unusual combinations and I doubt it would be possible to integrate it in StaSh. The only thing I could think of would be to detect when the cursor jumps to the beginning of the text field (that is what the up arrow does when there's no line above the current one) and then show the previous history item in the text field. Tab might also work, because there is no other way (except copy and paste) to insert a tab character or multiple spaces at once.

[briarfox]

Just submitted a pull request with:

• ssh-keygen - Generates rsa/dsa keys for scp and ssh
• scp - Secure copy. Gets and Put files/dirs onto or from a remote.

[briarfox]

I'm working on an ssh script and would like some feedback. It currently only works for linux. You should use ssh-keygen to create a key pair. Add the .pub to your known_hosts on your server. You can also use the user password if you do not want to use a rsa/dsa key.

I've used this modules with scp to make all my commits and pushes to the stash repo. It's nice to have full git access from Pythonista.

<pre>
ssh client for stash. ssh looks for a valid key generated buy ssh-keygen in .ssh.
You can open an intereactive shell by not passing a command. If a command is passed,
the single command is ran with output then ssh exits.

(exit) command will exit shell.

usage: ssh [-h] [--password PASSWORD] [-p PORT] host [command]

positional arguments:
host host ex. user@host.com
command Command to send as a quoted string

optional arguments:
-h, --help show this help message and exit
--password PASSWORD Password for rsa/dsa key or password login
-p PORT, --port PORT port for ssh default: 22
</pre>
ssh.py