Get Safari page source code in share extension.
Is there a way to access the HTML source code displayed in safari in the Pythonista share extension?
I know that I can get the url via the
appexmodule and then use
requests.get()to load the source code, but that won't work properly when the website requires a user to be logged in to display certain content.
I tried to work around that by loading the page in a offscreen
WKWebView, waiting for the delegate callback and then using the
WKWebView has one major advantage: it saves cookies. This means that I only need to log in once and I don't need to display the webView for all following requests.
js_eval_completionHandlergets never called
@lukaskollmer Why not just use
ui.WebView? It should save cookies too.
@omz I didn't know that
@omz is there a way to have
ui.WebViewload a URL without presenting the webView?
Webview also lets you use a few delegate methods which can be useful.
(edit: pointed to correct message)
the webview never needs to be presented to load.
@JonB I can't get it to load the page until I called
webView = ui.WebView() webView.delegate = WebViewDelegate() webView.load_url(url) webView.present()
If I remove
present(), the delegate won't get called
This works for me:
Possibly the delegate only gets called after it is presented? I seem to recall something like that. If so, you should be able to present it once, then close it, or present in a panel.
See for example:
This shows the delegate getting called without being presented.
When I tried to grab outerHTML directly at the end of the script, it has no data, because the page had not yet loaded. In the delegate webview_did_finish_load, it works fine.
Also, you could have a for loop that checks for document.readyState, to allow a more script-y sequence of commands. though you have to be a little careful because the readyState may go through complete two times initially.
ui.WebViewshare cookies between the main app and the extension?
does not look like it.
you can get cookies this way ( it would be possible to save this to a file which could be in a shared spot)
from objc_util import * storage=ObjCClass('NSHTTPCookieStorage').sharedHTTPCookieStorage() print(storage.cookies())
I have seen some stackoverflow code using NSKeyedArchiver to turn this into NSData
though i have not yet had luck in using NSKeyedUnarchiver to go from data back to an cookie storage object (just have not tried hard yet)
You can also get the binary cookie file from
From Apple Developer website .
Accessing a Webpage
To add webpage access and manipulation to your app extension, perform the following steps:
In the NSExtensionActivationRule dictionary in your app extension’s Info.plist file, give the NSExtensionActivationSupportsWebPageWithMaxCount key a nonzero value. (To learn more about the activation rule dictionary, see Declaring Supported Data Types for a Share or Action Extension.)
I was able to get the source code of the current Safari web page using the following code:
req = appex.get_web_page_info()
From there I was able to parse through the HTML using BeautifulSoup.