# coding: utf-8
## Download a GameFaqs.com FAQ in printable text format
## eg http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226
## v0.1

import os, sys, re, random, appex, console, clipboard, html2text, requests, dialogs


RE_URL = re.compile(ur'^http(s)?://(www\.)?gamefaqs\.com/.*/faqs/[0-9]{3,8}$', re.IGNORECASE)


def main():
    if appex.is_running_extension():
        url = appex.get_url()
    else:
        url = clipboard.get().strip()
        if not RE_URL.match(url):
            try:
                url = console.input_alert("Enter gamefaqs URL", "", "https://www.gamefaqs.com/")
            except KeyboardInterrupt:
                sys.exit(0)
    
    newurl = "{0}?print=1".format(url)
    if RE_URL.match(url):
        h = html2text.HTML2Text()
        r = requests.get(
                         url=newurl, 
                         headers={"User-agent": "Mozilla/5.0{0:06}".format(random.randrange(999999))}
                         )
        html_content = r.text.decode('utf-8')
        rendered_content = html2text.html2text(html_content)
        filename = url.partition("gamefaqs.com/")[-1].partition("/")[-1].partition("/faqs")[0]+".txt"
        filepath = os.path.join(os.path.expanduser("~/Documents"), filename)
        
        with open(filepath, "w") as fo:
            fo.write(rendered_content)
        
        console.hud_alert("Success! Saved to '~/Documents/{0}'".format(filename), "success")
        dialogs.share_url("file:///"+filepath)

if __name__ == '__main__':
    main()

I'll use this FAQ as the example document: http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226.

Appending ?print=1 as a parameter, ie, http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226**?print=1**, simplifies the document for printing such that in a browser, the content appears to be plaintext (of course it's not, it's HTML).

How can I save the document with Pythonista as actual plaintext?

I often switch between Pythonista Beta and 2.0 (or 1.5 as it were), and because my iPad is only 16Gb, I can't download both concurrently (though I'm pretty sure this isn't an option anyhow??))

There has to be a better way to:

1. Make backups (as a zip for eg), and more importantly
2. Restore said backups!

I used @omz PythonistaBackup to make a 300Mb zip file, Backup.zip. How do I:

1. get this into Pythonista (beta)
2. extract it
3. (Preferably) avoid manually moving and copying the files from ~/Documents/Backup (or wherever the extracted zip files are)

.?

@omz Yep, see code above, it works!

# coding: utf-8
## Download a GameFaqs.com FAQ in printable text format
## eg http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226
## v0.1

import os, sys, re, random, appex, console, clipboard, html2text, requests, dialogs


RE_URL = re.compile(ur'^http(s)?://(www\.)?gamefaqs\.com/.*/faqs/[0-9]{3,8}$', re.IGNORECASE)


def main():
    if appex.is_running_extension():
        url = appex.get_url()
    else:
        url = clipboard.get().strip()
        if not RE_URL.match(url):
            try:
                url = console.input_alert("Enter gamefaqs URL", "", "https://www.gamefaqs.com/")
            except KeyboardInterrupt:
                sys.exit(0)
    
    newurl = "{0}?print=1".format(url)
    if RE_URL.match(url):
        h = html2text.HTML2Text()
        r = requests.get(
                         url=newurl, 
                         headers={"User-agent": "Mozilla/5.0{0:06}".format(random.randrange(999999))}
                         )
        html_content = r.text.decode('utf-8')
        rendered_content = html2text.html2text(html_content)
        filename = url.partition("gamefaqs.com/")[-1].partition("/")[-1].partition("/faqs")[0]+".txt"
        filepath = os.path.join(os.path.expanduser("~/Documents"), filename)
        
        with open(filepath, "w") as fo:
            fo.write(rendered_content)
        
        console.hud_alert("Success! Saved to '~/Documents/{0}'".format(filename), "success")
        dialogs.share_url("file:///"+filepath)

if __name__ == '__main__':
    main()

I'll use this FAQ as the example document: http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226.

Appending ?print=1 as a parameter, ie, http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226**?print=1**, simplifies the document for printing such that in a browser, the content appears to be plaintext (of course it's not, it's HTML).

How can I save the document with Pythonista as actual plaintext?

@ccc said:

grab the name [ ... ] and navigate there in stash.

Means type the following into stash:

cd /var/mobile/Containers/Bundle/Application/C370CA19-1410-4DA3-975F-CBA563809E66/Pythonista.app/Frameworks/PythonistaKit.framework/pylib/site-packages/
ls

😋
I had no idea about this! Thx!

@JonB said:

the built in modules come preloaded in the app library, not in docs. You can view these folders from the Standard Library in the file browser, or grab the name as you already did, then navigate there in stash. There is no direct way to get there from Documents, because of the way apple sandboxing works. Actually, we probsbly should open a pull request in stash to provide an environment variable to the library folder.

as for ripemd160.... are you doing an from xxx import *?

Yep :)

If so, you can often expect issues with namespace clashes. Also, if you aborted an import,

Can you clarify this? Would

try:
    from urllib import add_opener # python3
except ImportError:
    from urllib2 import add_opener # python2

trigger the error?

that usually results in failure to import later, unless you restart pythonista. In the rare event that this is real, do
import pdb; pdb.pm() to figure out what is trying to be called.

@JonB said:

the built in modules come preloaded in the app library, not in docs. You can view these folders from the Standard Library in the file browser, or grab the name as you already did, then navigate there in stash.

Forgive me if this is a basic question, but how do I go about this? Ie how do I use stash to navigate to the Std Library directory?

as for ripemd160.... are you doing an from xxx import *? If so, you can often expect issues with namespace clashes. Also, if you aborted an import, that usually results in failure to import later, unless you restart pythonista. In the rare event that this is real, do
import pdb; pdb.pm() to figure out what is trying to be called.

Quick bug:

• closing Pythonista or locking the iPad while watching a video (mp4) using Quick View does not stop the video (and audio) from playing
• Importing ripemd160 module raises a NoneType not Callable when Pythonista modules (eg photos, dialogs, clipboard) have already been loaded, however ere is no such issue if the module is run first (Link to come)

@ccc said:

First off, a massive congratulations to you @omz for the scene rewrite and especially the new docs for that module. The intro will rapidly get any wannabe game developer hooked!! It will break a few things but your moving sk functionality into scene is brilliant and performance seems impressive.

Bugs

• Almost all third party modules are not current with PyPI.
• Many third party modules are present but not listed in the documentation: Crypto ctypes dateutil ecdsa flask html2text html5lib httplib2 itsdangerous jedi jinja2 mechanize midiutil mpmath oauth2 parsedatetime pycparser pyflakes pygments pyparsing PyPDF2 pytz reportlab simpy six sqlalchemy sympy thrift werkzeug wsgiref xhtml2pdf yaml

Out of interest, how do you find these modules?
For example:

import oauth2, os
odir = os.path.dirname(oauth2.__file__)
# odir = /var/mobile/Containers/Bundle/Application/C370CA19-1410-4DA3-975F-CBA563809E66/Pythonista.app/Frameworks/PythonistaKit.framework/pylib/site-packages/oauth2
# so oauth2 dir is in site-packages/oauth2

If I use Stash to look for the ~/site-packages/oauth2 directory, it's not there. What gives?

Edit: In Stash:
cd ~/Documents/site-packages
ls -a1
cd oauth2 <- ERROR

(Excuse what may be a very basic question!)

I forgot to report back; the script does work, but there are a few issues with simplekml (mainly because or recursion limit and it doesn't like the unwieldy default directory length, I'm told, though these are iOS limitations, not Pythonista😉)

Thanks for the input, much appreciated

I'm trying to edit and create Google Earth KML (keyhole markup language, based on XML); on Windows with iPython I usually use simplekml, however I'm having a tough time trying to install this, as the tar.gz doesn't want to open when using downloadista.pypi("simplekml", "1.2.8").

Any suggestions for using Pythonista with KML files would be greatly appreciated