# coding: utf-8
## Download a GameFaqs.com FAQ in printable text format
## eg http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226
## v0.1
import os, sys, re, random, appex, console, clipboard, html2text, requests, dialogs
RE_URL = re.compile(ur'^http(s)?://(www\.)?gamefaqs\.com/.*/faqs/[0-9]{3,8}$', re.IGNORECASE)
def main():
if appex.is_running_extension():
url = appex.get_url()
else:
url = clipboard.get().strip()
if not RE_URL.match(url):
try:
url = console.input_alert("Enter gamefaqs URL", "", "https://www.gamefaqs.com/")
except KeyboardInterrupt:
sys.exit(0)
newurl = "{0}?print=1".format(url)
if RE_URL.match(url):
h = html2text.HTML2Text()
r = requests.get(
url=newurl,
headers={"User-agent": "Mozilla/5.0{0:06}".format(random.randrange(999999))}
)
html_content = r.text.decode('utf-8')
rendered_content = html2text.html2text(html_content)
filename = url.partition("gamefaqs.com/")[-1].partition("/")[-1].partition("/faqs")[0]+".txt"
filepath = os.path.join(os.path.expanduser("~/Documents"), filename)
with open(filepath, "w") as fo:
fo.write(rendered_content)
console.hud_alert("Success! Saved to '~/Documents/{0}'".format(filename), "success")
dialogs.share_url("file:///"+filepath)
if __name__ == '__main__':
main()
Welcome!
This is the community forum for my apps Pythonista and Editorial.
For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.

Wizardofozzie
@Wizardofozzie
wish I was a little bit taller, I wish I was a baller, I wish I had a girl who looked good, I would call her...
I wish I had a rabbit in a hat with a bat, and a '64 Impala
Best posts made by Wizardofozzie
-
RE: Download "plain text" (HTML) document and save content as text
-
Download "plain text" (HTML) document and save content as text
I'll use this FAQ as the example document: http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226.
Appending
?print=1
as a parameter, ie, http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226**?print=1**, simplifies the document for printing such that in a browser, the content appears to be plaintext (of course it's not, it's HTML).How can I save the document with Pythonista as actual plaintext?
Latest posts made by Wizardofozzie
-
There must be a better way to make backups _and_ *restore* them
I often switch between Pythonista Beta and 2.0 (or 1.5 as it were), and because my iPad is only 16Gb, I can't download both concurrently (though I'm pretty sure this isn't an option anyhow??))
There has to be a better way to:
- Make backups (as a zip for eg), and more importantly
- Restore said backups!
I used @omz PythonistaBackup to make a 300Mb zip file,
Backup.zip
. How do I:- get this into Pythonista (beta)
- extract it
- (Preferably) avoid manually moving and copying the files from
~/Documents/Backup
(or wherever the extracted zip files are)
.?
-
RE: Download "plain text" (HTML) document and save content as text
@omz Yep, see code above, it works!
-
RE: Download "plain text" (HTML) document and save content as text
# coding: utf-8 ## Download a GameFaqs.com FAQ in printable text format ## eg http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226 ## v0.1 import os, sys, re, random, appex, console, clipboard, html2text, requests, dialogs RE_URL = re.compile(ur'^http(s)?://(www\.)?gamefaqs\.com/.*/faqs/[0-9]{3,8}$', re.IGNORECASE) def main(): if appex.is_running_extension(): url = appex.get_url() else: url = clipboard.get().strip() if not RE_URL.match(url): try: url = console.input_alert("Enter gamefaqs URL", "", "https://www.gamefaqs.com/") except KeyboardInterrupt: sys.exit(0) newurl = "{0}?print=1".format(url) if RE_URL.match(url): h = html2text.HTML2Text() r = requests.get( url=newurl, headers={"User-agent": "Mozilla/5.0{0:06}".format(random.randrange(999999))} ) html_content = r.text.decode('utf-8') rendered_content = html2text.html2text(html_content) filename = url.partition("gamefaqs.com/")[-1].partition("/")[-1].partition("/faqs")[0]+".txt" filepath = os.path.join(os.path.expanduser("~/Documents"), filename) with open(filepath, "w") as fo: fo.write(rendered_content) console.hud_alert("Success! Saved to '~/Documents/{0}'".format(filename), "success") dialogs.share_url("file:///"+filepath) if __name__ == '__main__': main()
-
Download "plain text" (HTML) document and save content as text
I'll use this FAQ as the example document: http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226.
Appending
?print=1
as a parameter, ie, http://www.gamefaqs.com/ps3/959558-fallout-new-vegas/faqs/61226**?print=1**, simplifies the document for printing such that in a browser, the content appears to be plaintext (of course it's not, it's HTML).How can I save the document with Pythonista as actual plaintext?
-
RE: Bug list for beta release 160037
@ccc said:
grab the name [ ... ] and navigate there in stash.
Means type the following into stash:
cd /var/mobile/Containers/Bundle/Application/C370CA19-1410-4DA3-975F-CBA563809E66/Pythonista.app/Frameworks/PythonistaKit.framework/pylib/site-packages/ ls
😋
I had no idea about this! Thx! -
RE: Bug list for beta release 160037
@JonB said:
the built in modules come preloaded in the app library, not in docs. You can view these folders from the Standard Library in the file browser, or grab the name as you already did, then navigate there in stash. There is no direct way to get there from Documents, because of the way apple sandboxing works. Actually, we probsbly should open a pull request in stash to provide an environment variable to the library folder.
as for ripemd160.... are you doing an
from xxx import *
?Yep :)
If so, you can often expect issues with namespace clashes. Also, if you aborted an import,
Can you clarify this? Would
try: from urllib import add_opener # python3 except ImportError: from urllib2 import add_opener # python2
trigger the error?
that usually results in failure to import later, unless you restart pythonista. In the rare event that this is real, do
import pdb; pdb.pm()
to figure out what is trying to be called.@JonB said:
the built in modules come preloaded in the app library, not in docs. You can view these folders from the Standard Library in the file browser, or grab the name as you already did, then navigate there in stash.
Forgive me if this is a basic question, but how do I go about this? Ie how do I use stash to navigate to the Std Library directory?
as for ripemd160.... are you doing an
from xxx import *
? If so, you can often expect issues with namespace clashes. Also, if you aborted an import, that usually results in failure to import later, unless you restart pythonista. In the rare event that this is real, do
import pdb; pdb.pm()
to figure out what is trying to be called. -
RE: Bug list for beta release 160037
Quick bug:
- closing Pythonista or locking the iPad while watching a video (mp4) using
Quick View
does not stop the video (and audio) from playing - Importing ripemd160 module raises a
NoneType not Callable
when Pythonista modules (egphotos
,dialogs
,clipboard
) have already been loaded, however ere is no such issue if the module is run first (Link to come)
- closing Pythonista or locking the iPad while watching a video (mp4) using
-
RE: Bug list for beta release 160037
@ccc said:
First off, a massive congratulations to you @omz for the
scene
rewrite and especially the new docs for that module. The intro will rapidly get any wannabe game developer hooked!! It will break a few things but your movingsk
functionality intoscene
is brilliant and performance seems impressive.Bugs
- Almost all third party modules are not current with PyPI.
- Many third party modules are present but not listed in the documentation: Crypto ctypes dateutil ecdsa flask html2text html5lib httplib2 itsdangerous jedi jinja2 mechanize midiutil mpmath oauth2 parsedatetime pycparser pyflakes pygments pyparsing PyPDF2 pytz reportlab simpy six sqlalchemy sympy thrift werkzeug
wsgirefxhtml2pdf yaml
Out of interest, how do you find these modules?
For example:import oauth2, os odir = os.path.dirname(oauth2.__file__) # odir = /var/mobile/Containers/Bundle/Application/C370CA19-1410-4DA3-975F-CBA563809E66/Pythonista.app/Frameworks/PythonistaKit.framework/pylib/site-packages/oauth2 # so oauth2 dir is in site-packages/oauth2
If I use Stash to look for the
~/site-packages/oauth2
directory, it's not there. What gives?Edit: In Stash:
cd ~/Documents/site-packages
ls -a1
cd oauth2
<- ERROR(Excuse what may be a very basic question!)
-
RE: Google Earth KML: best module? (simplekml installation issues)
I forgot to report back; the script does work, but there are a few issues with simplekml (mainly because or recursion limit and it doesn't like the unwieldy default directory length, I'm told, though these are iOS limitations, not Pythonista😉)
Thanks for the input, much appreciated
-
Google Earth KML: best module? (simplekml installation issues)
I'm trying to edit and create Google Earth KML (keyhole markup language, based on XML); on Windows with iPython I usually use simplekml, however I'm having a tough time trying to install this, as the tar.gz doesn't want to open when using
downloadista.pypi("simplekml", "1.2.8")
.Any suggestions for using Pythonista with KML files would be greatly appreciated