Library/module for Editorial via Pythonista?

jsamlarose47

Hi. I'm experimenting with logging items to a Google spreadsheet from Editorial. I'd like to play with a Google Sheets API library— either Gspread or Pygsheets, but as far as I understand it, I won't be able to install those libraries in Editorial (is that right?) so I'm wondering whether there's a way of doing that setting up the necessary modules in Pythonista and then writing an Editorial workflow that somehow calls on them? Is that even possible?

omz

The general possibility of installing pure Python packages is the same in Pythonista and Editorial, though it may be slightly more convenient in Pythonista because there are some existing (third-party) tools around that. However, there's usually not much more to installing a module than downloading a zip/tar.gz file from somewhere, and copying some files from it. You can do that in a couple lines of Python.

jsamlarose47

Thank you. This is the point at which I declare my love for Editorial. ;)

I've been using Editorial as a text editor and taskpaper client for a while now, but I'm just now starting to write my own workflows, so I'm starting to learn Python.

Does this look about right?

import workflow
import urllib
import zipfile
import os

url = 'https://github.com/nithinmurali/pygsheets/archive/master.zip'

print "downloading..."
urllib.urlretrieve(url, 'pygsheets.zip')

zip_ref = zipfile.ZipFile('pygsheets.zip', 'r')
zip_ref.extractall()
zip_ref.close()

os.system("pygsheets-master/setup.py")

dgelessus

@jsamlarose47 os.system is not allowed on iOS because of the sandbox restrictions. It's not possible to run the setup.py files either, also because of the sandbox, and because Editorial/Pythonista's Python installation is laid out differently than on a normal computer.

To install a library in Editorial, you first need to download and unzip it (which the first part of your script does already). Then you need to look at the extracted files and find the library's module or package, which is usually a folder or a Python script with the same name as the library. Move that file/folder into the site-packages folder (I'm not sure if Editorial has one by default - if not, create one under "Local Files"). Once that's done, you should be able to import the module in the Python prompt or in a workflow.

jsamlarose47

Got it. Thanks!

Downloaded the zip, extracted, and moved files from a folder titled "pygsheets" into "site-packages/pygsheets". Lots of fun learning my way around shutil... ;)

Tried to import pygsheets into a workflow, but I'm getting an import error (no module of that name). Reckon I've hit a wall with my current level of Python understanding in knowing where the module is and where it needs to be for my workflow to access it...

ccc

In the pygsheets setup.py file, it says it needs google-api-python-client and enum so you will need to repeat the process with those two. Unfortunately google-api-python-client says it needs httplib2 (already in Editorial), oauth2client, six (already in Editorial), and uritemplate. :-(

You might be better off to go back and try with gspread instead because it has a simpler dependency graph.

jsamlarose47

Ah— understood, and good to bear dependencies like that in mind for future reference.

I've now got the gspread folder (contents of the folder in gspread-master) in site-packages/gspread. Still getting an import error (no module named gspread)...

ccc

You want the path to be: site-packages/gspread/{files} and not site-packages/gspread/gspread/{files}.

jsamlarose47

@ccc sorry— I realise that the way I phrased that was unclear. I meant that I have all of the files from gspread-master/gspread/ in site-packages/gspread/ (as you're saying it should be).

Admittedly, my first use of shutil.move ended up depositing pygsheets files into the folder above site-package— the less said about that, the better... :) That said, I spent a little time tidying up my handling of paths, and I'm pretty sure I had everything in the right place before my final tests of pygsheets, and now gspread. Anything else it might be?

Thanks for all the assistance thus far...

omz

@jsamlarose47 Could you try restarting Editorial? Not 100% sure why right now, but it seems to help.

ccc

So... In site-packages/gspread you have these files? https://github.com/burnash/gspread/tree/master/gspread

You probably figured this part out already but if not...

If you are used to using the unix command line then you can vaguely simulate that with https://github.com/cclauss/Ten-lines-or-less/blob/master/cd_ls_pwd.py

For example, in Editor Python Console:

>>> print(os.getcwd())      # pwd in unix
/private/var/mobile/Containers/Data/Application/{UUID}/Documents
>>> print(os.listdir('.'))  # ls in unix
['dear_Sally.pdf', ..., ...]
>>> os.chdir('..')          # cd in unix
>>> print(os.getcwd())
/private/var/mobile/Containers/Data/Application/{UUID}
>>> print(os.listdir('.'))

ccc

See: https://forum.omz-software.com/topic/1438/own-python-modules

http://www.editorial-workflows.com/workflow/5841416039170048/ypL54IX63JY and http://www.editorial-workflows.com/workflow/5218285272432640/U8H7QaIxaJI might help yo get more Python functionality working in Editorial

jsamlarose47

@ccc Yep!
print os.listdir("site-packages/gspread") returns:
['__init__.py', 'client.py', 'exceptions.py', 'gspread', 'httpsession.py', 'models.py', 'ns.py', 'urls.py', 'utils.py']

print os.getcwd() returns:
/private/var/mobile/Containers/Data/Application/{UUID}/Library/Application Support/Commands

Presuming I'm in the right place there...

@olemoritz restarted, no joy...

And of course, the longer this continues the more I think I've just made a rudimentary error somewhere along the way... ;)

ccc

Can you please try putting the code in ~/Documents/site-packages/?

dgelessus

@jsamlarose47 A short explanation about the ~ part:

(If you've used a Unix shell before, you probably know this part already) ~ means "the user's home directory". That's where on most systems you have folders like "Documents", "Downloads", "Desktop", "Music", etc.
On iOS, you don't have a "home directory", because every app runs in its own sandbox. So on iOS, the "home directory" is the app's user data folder, which is the /private/var/mobile/Containers/Data/Application/{UUID} folder that you see in many paths.
Unlike a shell, Python doesn't automatically understand the ~ shortcut. You need to manually use os.path.expanduser(path) to replace the ~ with the actual home directory path.

Long story short, to move your module files to the correct location, this command should work:

shutil.move(os.path.expanduser("~/Library/Application Support/Commands/site-packages/gspread"), os.path.expanduser("~/Documents/site-packages/gspread"))

jsamlarose47

Done. Looks like this is the fix for me. Haven't yet fully tested a connection to a spreadsheet, but at least I'm not getting any errors when trying to call gspread modules. Excellent. Thanks!

jsamlarose47

Epilogue!

A couple of questions for me to wrap this up:

1: Just so I've got this down for future reference, should my original script have been something more like the following?

#coding: utf-8
import workflow
import urllib
import zipfile
import shutil
import os

# get the files
url = 'https://github.com/burnash/gspread/archive/master.zip'

print "downloading..."
urllib.urlretrieve(url, 'gspread.zip')

# expand the zip
zip_ref = zipfile.ZipFile('gspread.zip', 'r')
zip_ref.extractall()
zip_ref.close()

# move the required folder to the appropriate location
shutil.move(os.path.expanduser("~/Library/Application Support/Commands/gspread-master/gspread"), os.path.expanduser("~/Documents/site-packages/gspread"))

Do I need to make the folder in ~/Documents/site-packages/ before I attempt to move something to it?

2: Is it easier to do this kind of work with Pythonista (stash/pip)? And if so, are modules installed via Pythonista also available to Editorial— does Pythonista also look at ~/Documents/site-packages/ or do they function independently?

Thanks again, all!

ccc

#coding: utf-8

import os
import shutil
import urllib
import workflow
import zipfile

# get the files
url = 'https://github.com/burnash/gspread/archive/master.zip'
pkg_name = url.split('/')[4]  # gspread
zip_name = pkg_name + '.zip'
print("downloading...")
urllib.urlretrieve(url, zip_name)
print("extracting...")
with zipfile.ZipFile(zip_name) as in_file:
    in_file.extractall()
srce_dir = os.path.join(os.getcwdu(), pkg_name + '-master', pkg_name)
dest_dir = '~/Documents/site-packages'
if not os.path.exists(dest_dir):
	  os.makedirs(dest_dir)
print('moving {} to {}...'.format(pkg_name, dest_dir))
shutil.move(srce_dir, dest_dir)
print('done.')