Sync to Dropbox

markhamilton1

Finished a new sync program called Synchronator to sync from Pythonista to Dropbox and visa versa. It uses the new Dropbox API (the old one is deprecated). It can be found in Github at https://github.com/markhamilton1/Synchronator. I have now been using this to sync between iPad-Dropbox-iPhone. Take a look at the project wiki at https://github.com/markhamilton1/Synchronator/wiki.

Any feedback would be appreciated. This should work in either Python 2 or 3.

agalli

I tried it, and I get this stack trace:

****************************************
*     Dropbox File Syncronization      *
****************************************

Loading Local State

Updating From Dropbox

Saving Local State

Checking For New Or Updated Local Files
	Uploading:  DropboxSetup.py -- Not Found Remotely
Traceback (most recent call last):
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/Synchronator/Synchronator.py", line 300, in <module>
    check_local(dbx, state)
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/Synchronator/Synchronator.py", line 207, in check_local
    state.check_state(dbx, path)
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/Synchronator/Synchronator.py", line 66, in check_state
    self.upload(dbx, path, '-- Not Found Remotely')
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/Synchronator/Synchronator.py", line 171, in upload
    mute=True)
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/site-packages/dropbox/base.py", line 1252, in files_upload
    f,
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 249, in request
    timeout=timeout)
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 341, in request_json_string_with_retry
    timeout=timeout)
  File "/private/var/mobile/Containers/Shared/AppGroup/256C51EF-C1A2-4394-964A-C26B18D4F04B/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 385, in request_json_string
    type(request_binary))
TypeError: expected request_binary as binary type, got <class 'str'>

markhamilton1

Sorry for the delay. I have been looking for a solution but haven't found it yet.

Someone pointed out that they think this is because something in my code is not Python 3 compatible. I have been looking at what it could be, but I realized there is a simple work-around in the interim, just put #!/usr/bin/python2 as the first line in Synchronator.py. Because Pythonista is capable of running both it should work and Synchronator will sync all files. This is why I love Pythonista!

I have updated the code on GitHub with this change.

I still want to understand what is not working, but at the moment it seems that it is failing in the dropbox package. Still more than likely something I am doing wrong but haven't found it yet.

JonB

https://github.com/dropbox/dropbox-sdk-python/issues/76

this might be your problem? See link in the link..

ccc

import dropbox
print(dropbox.__version__)  # '6.4.0'

JonB

change
with open(path, 'r') as local_fr:
to
with open(path, 'rb') as local_fr:

in two places. files_upload takes bytes, not str, which you get by reading in binary mode, or else encoding.

markhamilton1

Thanks Jon, I have made the change and uploaded it to the GitHub repo.

agalli

Works perfectly now, thanks

nobregajoao

Hi Mark, everyone.

I am new to pythonista and came across this, thank you Mark for developing and sharing it!

I followed your guide / wiki and have installed the latest Dropbox package (8.0.0) via slash.

Both your scripts setup in the app and when I run it it gives a stack error of ValueError:

Thanks in advance for any help!

Joao

```
Dropbox File Syncronization      *
```

Loading Local State

Cannot Find State File -- Creating New Local State

Updating From Dropbox
Traceback (most recent call last):
File "/private/var/mobile/Containers/Shared/AppGroup/7C871155-8C31-4773-B218-2D35DDD5C4CC/Pythonista3/Documents/SyncDropbox/Synchronator.py", line 297, in <module>
check_remote(dbx, state)
File "/private/var/mobile/Containers/Shared/AppGroup/7C871155-8C31-4773-B218-2D35DDD5C4CC/Pythonista3/Documents/SyncDropbox/Synchronator.py", line 218, in check_remote
state.execute_delta(dbx)
File "/private/var/mobile/Containers/Shared/AppGroup/7C871155-8C31-4773-B218-2D35DDD5C4CC/Pythonista3/Documents/SyncDropbox/Synchronator.py", line 104, in execute_delta
results = dbx.files_list_folder('', True)
File "/private/var/mobile/Containers/Shared/AppGroup/7C871155-8C31-4773-B218-2D35DDD5C4CC/Pythonista3/Documents/site-packages/dropbox/base.py", line 715, in files_list_folder
None,
File "/private/var/mobile/Containers/Shared/AppGroup/7C871155-8C31-4773-B218-2D35DDD5C4CC/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 234, in request
timeout=timeout)
File "/private/var/mobile/Containers/Shared/AppGroup/7C871155-8C31-4773-B218-2D35DDD5C4CC/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 326, in request_json_string_with_retry
timeout=timeout)
File "/private/var/mobile/Containers/Shared/AppGroup/7C871155-8C31-4773-B218-2D35DDD5C4CC/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 411, in request_json_string
timeout=timeout)
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/requests/sessions.py", line 505, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/requests/sessions.py", line 462, in request
resp = self.send(prep, **send_kwargs)
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/requests/sessions.py", line 574, in send
r = adapter.send(request, **kwargs)
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/requests/adapters.py", line 371, in send
timeout=timeout
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/urllib3/connectionpool.py", line 566, in urlopen
timeout_obj = self._get_timeout(timeout)

File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/urllib3/connectionpool.py", line 309, in _get_timeout
return Timeout.from_float(timeout)
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/urllib3/util/timeout.py", line 155, in from_float
return Timeout(read=timeout, connect=timeout)
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/urllib3/util/timeout.py", line 98, in init
self._connect = self._validate_timeout(connect, 'connect')
File "/var/containers/Bundle/Application/FC4FD503-11FB-4FF7-AFBC-6134FAAE69C8/Pythonista3.app/Frameworks/Py2Kit.framework/pylib/site-packages/urllib3/util/timeout.py", line 128, in _validate_timeout
"int or float." % (name, value))
ValueError: Timeout value connect was Timeout(connect=30, read=30, total=None), but it must be an int or float.

markhamilton1

Thanks for bringing this to my attention!

So everything I can find online seems to indicate that this was a bug that was exposed in the Requests library when Dropbox updated their library to 8.0. I hate to ever point fingers since I have caused my own share of bugs.

If you run 'pip install requests' you should get version 2.18.1 or better. They fixed the issue in their 2.17.1 release from what I can find.

After installing this later version of Requests you will need to quit Pythonista and restart it to have it load the newest versions of the libraries.

After that Synchronator should run just fine.

Mark
Keep Calm and Code Python

Phuket2

@markhamilton1, just want to say thanks. I just used Synchronator to do a backup. Worked very well. On my first run it took a long time to run because of many files, i was worried about it timing out. Maybe using console.set_idle_timer_disabled True/False at the start and end of the sync might be a worthwhile thing.
But again thanks, it did work perfectly

nobregajoao

@markhamilton1
Works perfectly now!

Thank you! This opens a new whole new world

Cheers

amdescombes

Hi Mark,

Synchronator seems great, unfortunately I keep getting this error when I run it on Pythonista 3 using Python 3.5.
I used StaSh pip install to install the latest dropbox and requests to no avail, here is the stack trace, it seems to be a timeout problem :(

Updating From Dropbox

```
Dropbox File Syncronization      *
```

Loading Local State

Cannot Find State File -- Creating New Local State

Updating From Dropbox
Traceback (most recent call last):
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/urllib3/util/timeout.py", line 125, in _validate_timeout
float(value)
TypeError: float() argument must be a string or a number, not 'Timeout'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/private/var/mobile/Containers/Shared/AppGroup/895491B4-6F48-4CA2-AF6A-1A663749BD60/Pythonista3/Documents/bin/Utilities/Synchronator/Synchronator.py", line 425, in <module>
check_remote(dbx, state)
File "/private/var/mobile/Containers/Shared/AppGroup/895491B4-6F48-4CA2-AF6A-1A663749BD60/Pythonista3/Documents/bin/Utilities/Synchronator/Synchronator.py", line 333, in check_remote
state.execute_delta(dbx)
File "/private/var/mobile/Containers/Shared/AppGroup/895491B4-6F48-4CA2-AF6A-1A663749BD60/Pythonista3/Documents/bin/Utilities/Synchronator/Synchronator.py", line 174, in execute_delta
results = dbx.files_list_folder('', True)
File "/private/var/mobile/Containers/Shared/AppGroup/895491B4-6F48-4CA2-AF6A-1A663749BD60/Pythonista3/Documents/site-packages/dropbox/base.py", line 1336, in files_list_folder
None,
File "/private/var/mobile/Containers/Shared/AppGroup/895491B4-6F48-4CA2-AF6A-1A663749BD60/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 234, in request
timeout=timeout)
File "/private/var/mobile/Containers/Shared/AppGroup/895491B4-6F48-4CA2-AF6A-1A663749BD60/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 325, in request_json_string_with_retry
timeout=timeout)
File "/private/var/mobile/Containers/Shared/AppGroup/895491B4-6F48-4CA2-AF6A-1A663749BD60/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 409, in request_json_string
timeout=timeout,
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/requests/sessions.py", line 512, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/requests/sessions.py", line 469, in request
resp = self.send(prep, **send_kwargs)
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/requests/sessions.py", line 577, in send
r = adapter.send(request, **kwargs)
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/requests/adapters.py", line 377, in send
timeout=timeout
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/urllib3/connectionpool.py", line 547, in urlopen
timeout_obj = self._get_timeout(timeout)
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/urllib3/connectionpool.py", line 303, in _get_timeout
return Timeout.from_float(timeout)
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/urllib3/util/timeout.py", line 155, in from_float
return Timeout(read=timeout, connect=timeout)
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/urllib3/util/timeout.py", line 98, in init
self._connect = self._validate_timeout(connect, 'connect')
File "/var/containers/Bundle/Application/44E531D9-C860-4F8B-9A15-16BE6F3BE756/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/site-packages/urllib3/util/timeout.py", line 128, in _validate_timeout
"int or float." % (name, value))
ValueError: Timeout value connect was Timeout(connect=30, read=30, total=None), but it must be an int or float.

Thank in advance
Cheers
Andre

Dangthrimble

I've just installed Pythonista, the dropbox and requests packages and Synchronator. When I run DropboxSetup.py I get the following:

/private/var/mobile/Containers/Shared/AppGroup/EAEDE395-145A-4A8B-A282-E6D1409E3D51/Pythonista3/Documents/site-packages/requests/init.py:80: RequestsDependencyWarning: urllib3 (1.4) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Traceback (most recent call last):
File "/private/var/mobile/Containers/Shared/AppGroup/EAEDE395-145A-4A8B-A282-E6D1409E3D51/Pythonista3/Documents/Dropbox/DropboxSetup.py", line 54, in <module>
import dropbox
File "/private/var/mobile/Containers/Shared/AppGroup/EAEDE395-145A-4A8B-A282-E6D1409E3D51/Pythonista3/Documents/site-packages/dropbox/init.py", line 3, in <module>
fronds in my UJ UK m .dropbox import version, Dropbox, DropboxTeam, create_session # noqa: F401
File "/private/var/mobile/Containers/Shared/AppGroup/EAEDE395-145A-4A8B-A282-E6D1409E3D51/Pythonista3/Documents/site-packages/dropbox/dropbox.py", line 17, in <module>
import requests
File "/private/var/mobile/Containers/Shared/AppGroup/EAEDE395-145A-4A8B-A282-E6D1409E3D51/Pythonista3/Documents/site-packages/requests/init.py", line 90, in <module>
from urllib3.exceptions import DependencyWarning
ImportError: cannot import name 'DependencyWarning'

JonB

make sure you are using the latest stash.

selfupdate

Install requests

pip install requests

For me, I had to update chardet and idna. Update failed, but install then fixed it.

pip update chardet
pip install chardet
pip update idna
pip install idna
pip update dropbox
pip install dropbox

in console:

M=[m for m in sys.modules if 'requests' in m]
for m in M:
   del sys.modules[m]
import requests

Dangthrimble

@JonB can you explain the purpose of step 4? And with all the packages that have been installed, why am I only interested in requests? Thanks

JonB

you could just force quit pythonista. The issue is that requests is imported by stash, so to force the new version to import, you have to delete everything related from sys.modules.

Dangthrimble

@JonB why don't I have to do the same for the other packages that have been installed? Do you mean StaSh doesn't use the other packages?

Dangthrimble

@JonB I tried the first line at the console and got:

>>> M=[m for m in sys.modules if 'requests' in m]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'sys' is not defined

JonB

import sys

Again, the only reason I did this was to check that the import was successful, without a full quit -- it took a few tries, to figure out that I needed to update chardet, then idna.

Since you know what you need to update already, just double tap the home button and swipe to kill pythonista. When you come back, import requests and if it works, you are good to go