Pythonista 3 + StaSh + Plotly... I'm Close to a meltdown.

hecate

Hi everyone!

I have tried to install Plotly in Pythonista and I keep getting "cannot import name"-errors.

In a final try I made a fresh install of Phytonista 3 and gave it a new go!
Same procedure on iphone 6SE (12.1.2), ipad mini 4 (12.1.1) and ipad air 2 (12.1.1).

TLDR;

Pythonista + StaSh: different folder-structure on 12.1.2 vs 12.1.1.
Pythonista + StaSh + Plotly: same folder-structure for 12.1.2 and 12.1.1.
Empty local folders (/bin /fsi /man /patches). Is that normal?
PIP: I can upgrade on one ipad but the other. --upgrade is an unrecognized argument.
Same error on all installations for line 1 in script: import plotly

Everything crashes. 😿

(I have managed to install qhue (https://github.com/quentinsf/qhue)[https://github.com/quentinsf/qhue] with StaSh, and use it, without any problems. )

Installing Stash

In Console:
import requests as r; exec(r.get('https://bit.ly/get-stash').text)
Restarted Pythonista
Ran the file launch_stash.py
Restarted Pythonista
Ran the file launch_stash.py again. I get a shell.

The folders on iPhone

Is this how the folder-structure should be built? With a folder for stash_extensions with 4 empty folders? They remain empty, even after I have installed plotly.

SCRIPT LIBRARY

    |-- This iPhone
        |
	|-- [Examples]
	|-- [stash_extensions]
	     |-- [bin] - empty
	     |-- [fsi] - empty
	     |-- [man] - empty
	     |-- [patches] - empty

|-- ICloud

|-- Python Modules
	|
	|-- Standard Library 2..
	|-- Standard Library 3..
	|-- [site-packages] (empty)
	|-- [site-packages-2] (empty)
	|-- [site-packages-3] (empty)
		
|-- File Templates
	|
	|-- Readme.md

Installing plotly

1. Ran the pip install plotly command in the shell. Took half forever.
Got a warning right in the beginning saying "cannot find dependencies".. this is the result:

[...]
Running handler 'console_scripts installer'...
No entry_points.txt found, skipping.
Cleaning up...
Package installed: plotly
[~/Documents]$

[site-packages-3] now have several folders. It seems they are placed in root, there is no folder specific to plotly.

As a plot-twist: The iPads don't get the empty folders inside [This iPad] when installing StaSh. Instead the folders appeared after I installed plotly. They also remain empty.

I get the same error on the ipads as the iPhone when running the same script.
I tried from both iCloud and This iPhone/iPad.

2. I run a script get stuck on line 1.

import plotly.

ImportError 
cannot import name 'exceptions'
plotly.py <module> (Line 30)

Print Traceback

Traceback (most recent call last):
File "/private/var/mobile/Library/Mobile Documents/iCloud~com~omz-software~Pythonista3/Documents/Temperature/plotly_entropia.py", line 1, in <module>
import plotly

File "/private/var/mobile/Containers/Shared/AppGroup/63784CB9-E25E-4FFB-A7F4-D38A7B74A24B/Pythonista3/Documents/site-packages-3/plotly/__init__.py", line 10, in <module>
from . plotly import (

File "/private/var/mobile/Containers/Shared/AppGroup/63784CB9-E25E-4FFB-A7F4-D38A7B74A24B/Pythonista3/Documents/site-packages-3/plotly/plotly.py", line 30, in <module>
from plotly import exceptions, files, session, tools, utils

ImportError: cannot import name 'exceptions'

Not working.

Going bananas

1. But I read that an update could help!
plotly.py issue 104

However - as the mess that I am - I did an upgrade of StaSh and not of plotly.

selfupgrade -f , on iphone, made Pythonista crash.
And as it keeps crashing I will probably have to start over from scratch there...

2 Instead! I move to the ipad air 2!

pip install plotly --upgrade
As stated on https://plot.ly/python/getting-started/, did not work.

pip install --upgrade plotly
>> No download available for upgrade: 0.0.1..

3. The last straw: change an url in /bin/
From topic/4733/stash-install:


The PyPi API was changed. The new link is 'https://pypi.org/pypi/{}/json'. Thus, the simplest way to fix this is to find the pip.py (~/Modules/site-packages/stash/bin/pip.py) and change the old link(http://pypi.python.org/pypi/{}/json) to 'https://pypi.org/pypi/{}/json'

I tried that and restarted the app.
Now also Pythonista on ipad air 2 is crashing...

4. Trying out the upgrade of step 8 on the iPad mini 4

Either way I formatted it:

>> usage: pip.py [-h] [--verbose] [-6] sub-command ...
>> pip.py: error: unrecognized arguments: --upgrade

I need to take a walk now.

Any help would be greatly appreciated! 💟

bennr01

Hi @hecate,
I can confirm that this is currently broken.
However, there is a workaround (though this was tested using py2, not py3):

1. fixing your install
When trying to reproduce this issue, my StaSh install broke, so i assume this is the same for you.

delete everything in site-packages-3 (or 2, if you use py2)
force-quit pythonista and restart.
Run pip uninstall plotly . Pythonista crashed for me during this step. If it does, just repeat this.

2. install plotly
Go to site-packages/stash/bin/pip.py and replace in PyPIRepository.install() wheel_priority=True with wheel_priority=False. This line should be somewhere around 1052.
Then run pip install plotly. If you get an error telling you that plotly is already installed, run pip uninstall plotly. Then repeat.

Plotly should now be installed. I still got some import errors regarding decorators, but it should now be possible to simply install this package/module.

So, now the other issues:

Empty local folders (/bin /fsi /man /patches). Is that normal?

Depends. If you mean the directory /stash_extensions/bin, then yes. This directory is used for extensions, like external commands installed by pip.
If you mean directories inside site-packages/stash/, then no.

PIP: I can upgrade on one ipad but the other. --upgrade is an unrecognized argument.

In StaSh, there is a pip update command. I am surprised that pip install --upgrade worked.
Maybe one of your devices uses an old version? If you want to check, try running version. The latest version is 0.7.1.

TLDR;
It seems like there is a bug in the wheel installation handling. Sorry.

Edit/Update:
I found the bug and fixed it, but the fix is not yet in the main StaSh repo. To install the fixed version, simply run selfupdate -f bennr01:wheelfix. Please note that you should you should still remove the wrongly installed directories in site-packages-3. If you delete everything in site-packages-3, then you should also delete site-packages-3/.pypi_packages (e.g. rm $HOME/site-packages-3/.pypi_packages).

ihf

Just curious, but does Dash run under Pythonista or are there incompatibilities?

hecate

@bennr01

Thank you kindly for your detailed and very helpful (and very fast!!) answer! I have more busy days now that vacation is over..

I did a reinstall of both Pythonista and StaSh
On my iphone this far, and updated per your instructions to get the fixed version. It doesn't seem to work still, and I wonder if there could be something with my code. I thought maybe it would be easier to find the problem if you have the same as me.

import plotly 
plotly.tools.set_credentials_file(username='[username]', api_key='[token]')

import plotly.plotly as py
from plotly.graph_objs import *

data = Data([ Scatter(x=[1, 2], y=[3, 4]) ])

plot_url = py.plot(data, filename='my plot')

I'ts an example I found in a tutorial and I have replaced username and token for my own when I run it.

This generated the following traceback:

Traceback (most recent call last):
  File "/private/var/mobile/Library/Mobile Documents/iCloud~com~omz-software~Pythonista3/Documents/Temperature/plotly_entropia.py", line 1, in <module>
    import plotly
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/__init__.py", line 31, in <module>
    from plotly import (plotly, dashboard_objs, graph_objs, grid_objs, tools,
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/plotly/__init__.py", line 10, in <module>
    from . plotly import (
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/plotly/plotly.py", line 31, in <module>
    from plotly.api import v1, v2
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/api/v1/__init__.py", line 3, in <module>
    from plotly.api.v1.clientresp import clientresp
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/api/v1/clientresp.py", line 9, in <module>
    from plotly.api.v1.utils import request
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/api/v1/utils.py", line 9, in <module>
    from plotly.api.v2.utils import should_retry
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/api/v2/__init__.py", line 3, in <module>
    from plotly.api.v2 import (dash_apps, dashboards, files, folders, grids,
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/api/v2/dash_apps.py", line 6, in <module>
    from plotly.api.v2.utils import build_url, request
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/plotly/api/v2/utils.py", line 127, in <module>
    stop_max_delay=180000, retry_on_exception=should_retry)
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/retrying.py", line 47, in wrap
    @six.wraps(f)
AttributeError: module 'six' has no attribute 'wraps

If I run my darksky-app, (that I for sure can have broken myself!) it doesn't seem to get 'core'?

The traceback

**the Traceback (most recent call last):
  File "/private/var/mobile/Library/Mobile Documents/iCloud~com~omz-software~Pythonista3/Documents/weather_bagarmossen.py", line 4, in <module>
    from darksky import forecast
  File "/private/var/mobile/Containers/Shared/AppGroup/B45A435E-6A06-4C49-A432-C693B398B81F/Pythonista3/Documents/site-packages-3/darksky/__init__.py", line 1, in <module>
    from core import *
ModuleNotFoundError: No module named 'core'

The folders
On the phone I still get a fsi, man, patches in stash_extensions that are empty... binis also empty.

I seem to be running the last version of stash in either case :)
I will try this on the ipads asap and come back with the results!

JonB

After getting the six wraps error, try

import six
print(six.__file__)
print (sys.path)

It seems possible that either pip tried to install six (and failed), or you have a file six.py oor folder named six in your Documents folder, which gets precedence on sys.path.

For tour darksky problem, again, try printing sys.path and see if that explains anything.

In the pythonista environment, whenever you hit "play", the path of the open script gets inserted at the start of sys.path. Meaning any folders under Documents (in your example) get precedence. One would expect for darksky you should have a core.py (or folder called core that includes an __init__.py ) within site-packages-3, site-packages-3/darksky, or Documents.

Do you know if you are running stash in the 2.7 or 3.6 enviroment?

hecate

@ihf
Hi! maybe this qustion wasn't meant for me but I would love it to work in either case. :)

hecate

@JonB

import six
print(six.__file__)
print (sys.path)

Gave this response:

>>> import six
>>> print(six.__file__)
... 
/var/containers/Bundle/Application/A2676392-B9EC-4C7E-94C4-E1AA04500EDF/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/six.py
>>> print (sys.path)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'sys' is not defined

In site-packages-3/darksky/ there is a core.py and init.py.

When I run versionin StaSh:

StaSh v0.7.1
Pythonista 3.2 (320000)
iOS 12.1.2 (64-bit iPhone8,4)
Python 3.6.1
root: ~/Documents/site-packages/stash
core.py: 2019-01-06 15:13:31
SELFUPDATE_TARGET: master
BIN_PATH:
  ~/Documents/bin
  ~/Documents/stash_extensions/bin
  ~/Documents/site-packages/stash/bin

I did upgrade per instructions of @bennr01 : selfupdate -f bennr01:wheelfix

DarkSky worked when I installed this before, without the wheelfix-upgrade.

bennr01

@hecate

On the phone I still get a fsi, man, patches in stash_extensions that are empty... bin is also empty.

Do not worry about these folders. They are meant for extensions and are empty by default. In fact, stash_extensions/bin/ is the only one of these folders which is actively used (pip installs commands here).

I'ts an example I found in a tutorial and I have replaced username and token for my own when I run it.

I just tested this example on both py2 and py3 and it seems to work (I get some warnings that plotly.graph_objs.Data is deprecated and also an error message that there is no account for [username] (obviously), but it imported and executed fine). I had to install decorator and retrying and also restart pythonista a few times.

This generated the following traceback:

Have you tried restarting pythonista (double tap home, the force-quit the app and start again)? Pythonista caches imports, which sometimes leads to weird issues and old imports. You should always restart pythonista if you installed a package to fix an 'ImportError` (or something related).

If I run my darksky-app, (that I for sure can have broken myself!) it doesn't seem to get 'core'?

Same as above (btw, import works fine for me, but i have not tested actual functionality).
BTW, you should avoid from module import *-style imports, as they may lead to import-related problems. A better import would be from darksky.core import *.
If you are interested in an explanation: As @JonB said, sys.path is modified when a script is executed. This means that python search for imports in a different order depending on the executed script and your CWD. For example, if you would execute/import darksky/__init__.py from within the site-packages/stash-repository, import core would import stash.core instead of darksky.core. This would lead to problems or unexpected behavior.

Python 3.6.1

The py3 version of StaSh is still a bit experimental, so there may be more errors compared to py2. That being said, everything above should work fine (i tested your example on both py2 and py3 and darksky on py2) .

I did upgrade per instructions of @bennr01 : selfupdate -f bennr01:wheelfix
DarkSky worked when I installed this before, without the wheelfix-upgrade.

Is there a core.py-file in site-packages-3/darksky/? If there is, then you are probably experiencing some of the CWD-related problems mentioned above. I can confirm that the no module named 'core' error exists depending on the way you import/launch __init__.py You should either try it with from darksky.core import * instead or try to install and run with py2 (py2 has some different import behaviors and from darkskye import forecast works fine). Also, restart after any changes to files in a site-packages* directory, or your change will not take any effect.

hecate

@bennr01 said:

Do not worry about these folders. They are meant for extensions and are empty by default.

Ok!

I just tested this example on both py2 and py3 and it seems to work

Wow. I cannot get it to work. :/

Have you tried restarting pythonista (double tap home, the force-quit the app and start again)?

I have restarted the app countless of times, exactly that way. Last try of installing Plotly, Pythonista couldn't reopen again, it crashed while loading. But hey, I didn't have to restarting it?

Same as above (btw, import works fine for me, but i have not tested actual functionality).

Wow again. I will have to give this another go!

BTW, you should avoid from module import *-style imports, as they may lead to import-related problems. A better import would be from darksky.core import *.

I changed this part of the code, restarted again and again but it still won't run. I tried to remove darksky from Python Modules/site-packages-3 where I removed the darksky-folder. I did about 10 restarts of the app and then tried to reinstall through StaSh; "Error: Package is already installed"
I did a reinstall of pythonista but I don't feel like trying to get this Plotly to work anymore. I would be so happy if the scripts I had could just work as they used too..

Is there a core.py-file in site-packages-3/darksky/? If there is, then you are probably experiencing some of the CWD-related problems mentioned above.

Yep there was before I reinstalled it all over again. The change to from darksky.core import * didn't make any difference unfortunatly. And thank you and @JonB for explaining how the sys.path works!

I can confirm that the no module named 'core' error exists depending on the way you import/launch __init__.py You should either try it with from darksky.core import * instead or try to install and run with py2 (py2 has some different import behaviors and from darkskye import forecast works fine). Also, restart after any changes to files in a site-packages* directory, or your change will not take any effect.

I tried installing darksky with StaSh running python 2.7. It was installed. Restarted the app a couple of times. Ran the script with python 2.7, I get an error with the syntax, fair enough, though it doesn't print the traceback, I get an empty console. Regardless if I use from darksky import forecast or from darksky.core install * I get the same result.

I guess that's as far as I can manage to get with this.. can I use a js-based graph-tool in Pythonista? If you have any suggestions for this I will gladly listen.

Thank you again @bennr01 and @JonB for all your help!!

JonB

Sorry, is darksky your own repo? And you installed with pip? Can you provide the pip commands you used?

hecate

This post is deleted!

hecate

@JonB
Thank you!!
I actually wrote an answer along "I used pip install darkskyand lalala", the second I clicked [Submit] I realized I have been writing darksky instead of darkskylib... installing the ****** wrong library. 😳 😫 🙁

The library: https://github.com/lukaskubis/darkskylib
It's not my own, and it actually doesn't say how to install it in the readme. I have just assumed..

So I found the reason, I thought..! On the ipad air 2, that had a clean install of pythonista...

I installed stash, restarted app.
pip install darkskylib, restarted app.
pip install requests, restarted app.
Ran the script: "Cannot find module for chardet.

Which I thought was strange but I realized I didn't update stash this time around. So I did selfupdate -f bennr01:wheelfix. Tried again, same thing.

I removed all the files I could find for darkskylib.
I restarted pythonista.
Ran pip install darkskylib again, it said it was already installed.

Did a full reinstall of the pythonista. New round.

Install stash. restart app.
Update stash, restart app.
Install requests, restart app.
Install darkskylib:

[~/Documents]$ pip install darkskylib
<class 'ModuleNotFoundError'>: No module named 'chardet'
[~/Documents]$ pip install chardet
<class 'ModuleNotFoundError'>: No module named 'chardet'

JonB

Is there a reason you need the latest requests, vs the one built into pythonista?

You need to check site-packages, site-packages-3, and site-packages-2, and remove (via file browser menu in pythonista, under Modules) requests, chardet, and idna folders, if present. Then force quit pythonista.

Then import requests should work.

Then, pip should be working again.

If you need new requests version, you could try
pip install idna
pip install chardet
pip install requests

I have not tried this with the current stash version... A safer approach might be to use pip download, then manually unzip/untar and copy the folders manually.

JonB

I finally got around to trying this on the latest stash. @bennr01, it seems the issue is that pip is not detecting doendencies using the whl.

Because of this, it is necesary to manually instal each dependency:

rm -f site-packages*/requests
rm -f site-packages*/chardet
rm -f site-packages*/idna

pip install requests
pip install chardet
pip install idna

Do this all before running any other pip commands after starting the app.
The last three must be done within a single sesson.
Then force quit pythonista. import requests, and all should be happy.

@bennr01 here is the whl install, showing it couldnt find dependencies.

[~/Documents]$ pip install chardet
Querying PyPI ... 
Downloading package ...
Opening: https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl

Save as: /private/var/mobile/Containers/Data/Application/06FD02C2-F3B4-4512-8629-018C9F64EA15/tmp/chardet-3.0.4-py2.py3-none-any.whl (133356 bytes)
    133356  [100.00%]
Installing wheel: chardet-3.0.4-py2.py3-none-any.whl...
Extracting wheel..
Extraction finished, running handlers...
Running handler 'WHEEL information checker'...
Wheel generated by: bdist_wheel (0.29.0)
Running handler 'dependency handler'...
Running handler 'top_level.txt installer'...
Copying /private/var/mobile/Containers/Data/Application/06FD02C2-F3B4-4512-8629-018C9F64EA15/tmp/wheel_tmp/chardet-3.0.4-py2.py3-none-any.whl/chardet -> /private/var/mobile/Containers/Shared/AppGroup/C534C622-2FDA-41F7-AE91-E3AAFE5FFC6B/Pythonista3/Documents/site-packages-3
Running handler 'console_scripts installer'...
Cleaning up...
Package installed: chardet
[~/Documents]$ pip install idna
Querying PyPI ... 
Downloading package ...
Opening: https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl

Save as: /private/var/mobile/Containers/Data/Application/06FD02C2-F3B4-4512-8629-018C9F64EA15/tmp/idna-2.8-py2.py3-none-any.whl (58594 bytes)
     58594  [100.00%]
Installing wheel: idna-2.8-py2.py3-none-any.whl...
Extracting wheel..
Extraction finished, running handlers...
Running handler 'WHEEL information checker'...
Wheel generated by: bdist_wheel (0.32.2)
Running handler 'dependency handler'...
Warning: could not find 'metadata.json', can not detect dependencies!
Running handler 'top_level.txt installer'...
Copying /private/var/mobile/Containers/Data/Application/06FD02C2-F3B4-4512-8629-018C9F64EA15/tmp/wheel_tmp/idna-2.8-py2.py3-none-any.whl/idna -> /private/var/mobile/Containers/Shared/AppGroup/C534C622-2FDA-41F7-AE91-E3AAFE5FFC6B/Pythonista3/Documents/site-packages-3
Running handler 'console_scripts installer'...
No entry_points.txt found, skipping.
Cleaning up...
Package installed: idna
[~/Documents]$ ls site-packages-3

site-packages-3/:
Readme.md chardet idna midiutil requests

JonB

I will also say, that running pip install plotly with latest stash caused pip to install all of the plotly folders into site-packages-3, instead of the plotly folder, which totally borked pythonista, and it crashes immediately upon open, even with safemode and pythonista3://. could possibly be my fault for hitting ctrl-z during the process.

so... don't do that

bennr01

@JonB

I will also say, that running pip install plotly with latest stash caused pip to install all of the plotly folders into site-packages-3, instead of the plotly folder, which totally borked pythonista, and it crashes immediately upon open, even with safemode and pythonista3://. could possibly be my fault for hitting ctrl-z during the process.

so... don't do that

There should be a fix in the dev branch of the main repo, or did the fix not work?

@JonB said:

I finally got around to trying this on the latest stash. @bennr01, it seems the issue is that pip is not detecting doendencies using the whl.

Thanks for the info, i will take a look at it.
This line looks interesting: Warning: could not find 'metadata.json', can not detect dependencies!.
Apparently the .whl-format is a bit more flexible than i tought...

headsphere

Not sure if this is related or if I should start another thread but the error I’m getting when installing plotly is related to installing the attrs dependency:

Querying PyPI ... 
Downloading package ...
Opening: https://files.pythonhosted.org/packages/23/96/d828354fa2dbdf216eaa7b7de0db692f12c234f7ef888cc14980ef40d1d2/attrs-19.1.0-py2.py3-none-any.whl

Save as: /private/var/mobile/Containers/Data/Application/AA8947EF-3A9D-4275-BC4A-9708A8765CD1/tmp//attrs-19.1.0-py2.py3-none-any.whl (35784 bytes)
     35784  [100.00%]
Installing wheel: attrs-19.1.0-py2.py3-none-any.whl...
<class 'UnicodeDecodeError'>: 'ascii' codec can't decode byte 0xe2 in position 5690: ordinal not in range(128)
[~/Documents]$ version

Here is the output I’m getting from version:

StaSh v0.7.2
Pythonista 3.2 (320000)
iOS 12.2 (64-bit iPad7,4)
Python 3.6.1
root: ~/Documents/site-packages/stash
core.py: 2019-05-04 17:47:40
SELFUPDATE_TARGET: master
BIN_PATH:
  ~/Documents/bin
  ~/Documents/stash_extensions/bin
  ~/Documents/site-packages/stash/bin

Any clever ideas?

bennr01

@headsphere You could try disabling wheels for the installation using pip --verbose install --no-binary :all: attrs.