Welcome!
This is the community forum for my apps Pythonista and Editorial.
For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.
Get Safari page source code in share extension.
-
Requests allows you enter userid and password credentials via multiple authentication schemes http://docs.python-requests.org/en/master/user/advanced/#custom-authentication
-
@ccc I'm pretty sure that's still not what he's asking for. He's asking about how to use
NSExtensionContent
to reliably view the exact content that is loaded into Safari. As I understand it, this would allow:- Viewing content that is only available through authentication. I think he's talking about form-based authentication, not HTTP auth, which I think is possible through
mechanize
but notrequests
. - Viewing DOM nodes that have been manipulated by JavaScript through user interaction. Visiting this page anew would not include these modifications, because they resulted from explicit user actions
I'm sure there are other advantages to seeing the exact HTML that Safari is seeing.
- Viewing content that is only available through authentication. I think he's talking about form-based authentication, not HTTP auth, which I think is possible through
-
Of course form based authorization can be done with requests! You just have to formulate the request correctly, which sometimes might be tricky to figure out. WebView is another way.
I have not tried the NSExtensionContext, but reading the docs you need
NSExtensionJavaScriptPreprocessingFile in the info.plist, and maybe one other related to weburls. Here is what is currently in to plistNSExtension = { NSExtensionAttributes = { NSExtensionActivationRule = "SUBQUERY(extensionItems, $extensionItem, SUBQUERY($extensionItem.attachments, $attachment, (ANY $attachment.registeredTypeIdentifiers UTI-CONFORMS-TO \"public.url\" OR ANY $attachment.registeredTypeIdentifiers UTI-CONFORMS-TO \"public.image\" OR ANY $attachment.registeredTypeIdentifiers UTI-CONFORMS-TO \"public.audio\" OR ANY $attachment.registeredTypeIdentifiers UTI-CONFORMS-TO \"public.text\")).@count > 0).@count > 0"; }; NSExtensionPointIdentifier = "com.apple.ui-services"; NSExtensionPrincipalClass = PAESlidingContainerViewController; };
So, this won't work unless @omz adds this permission. I suspect that is a step apple would not be comfortable with.
-
@JonB I've tried using the JS preprocessing when I originally built the app extension (that was still on iOS 8), but for some reason, I couldn't get it to work reliably. I'm not sure why exactly, but in some cases, having this key in the Info.plist resulted in getting no data at all in the extension. Given that there are app extensions that work with this pretty reliably (1Password...), it's probably either something I did wrong, or something that was fixed in the meantime. I might give it another shot sometime.
-
I tried to work around that by loading the page in a offscreen
WKWebView
, waiting for the delegate callback and then using the-evaluateJavascript:completionHandler
method to fetch the html.WKWebView has one major advantage: it saves cookies. This means that I only need to log in once and I don't need to display the webView for all following requests.
Problem: The
-evaluateJavaScript:completionHandler
method is block based and I can't get it to work (@omz @JonB do you have any idea what I'm doing wrong?)@on_main_thread def js_eval_completionHandler(_cmd, _obj_ptr, err_ptr): print('Did call completionHandler') def webView_didFinishNavigation_(_self, _cmd, _webView, _navigation): webView = ObjcInstance(_webView) snippet_content_load_block = ObjCBlock(js_eval_completionHandler, restype=None, argtypes=[c_void_p, c_void_p, c_void_p]) webView.evaluateJavaScript_completionHandler_('document.documentElement.outerHTML.toString()', snippet_content_load_block)
The
js_eval_completionHandler
gets never called -
@lukaskollmer Why not just use
ui.WebView
? It should save cookies too. -
@omz I didn't know that
ui.WebView
can evaluate JavaScript as well -
@omz is there a way to have
ui.WebView
load a URL without presenting the webView? -
Webview also lets you use a few delegate methods which can be useful.
If you start mucking about with javascript/webviews, I suggest implementing a window.onerror, some console.log functions, and a corresponding delegate method to print to the console. This makes javascript work MUCH less mysterious. See for example here, the lines through the end of the delegate are my standard starting place for touching any javascript.
(edit: pointed to correct message) -
w.load_url
the webview never needs to be presented to load.
-
@JonB I can't get it to load the page until I called
present()
webView = ui.WebView() webView.delegate = WebViewDelegate() webView.load_url(url) webView.present()
If I remove
present()
, the delegate won't get called -
This works for me:
w=ui.WebView()
w.load_url('http://www.google.com')
src=w.eval_js('document.documentElement.outerHTML')Possibly the delegate only gets called after it is presented? I seem to recall something like that. If so, you should be able to present it once, then close it, or present in a panel.
-
Actually this works fine for me. Perhaps your delegate or url has some issues. You also need to wait until,the page loads, either via javascript or using the delegate.
See for example:
https://gist.github.com/jsbain/80cdc7dd82da23cbe16c9befef91d707This shows the delegate getting called without being presented.
When I tried to grab outerHTML directly at the end of the script, it has no data, because the page had not yet loaded. In the delegate webview_did_finish_load, it works fine.Also, you could have a for loop that checks for document.readyState, to allow a more script-y sequence of commands. though you have to be a little careful because the readyState may go through complete two times initially.
-
@omz does
ui.WebView
share cookies between the main app and the extension? -
does not look like it.
you can get cookies this way ( it would be possible to save this to a file which could be in a shared spot)from objc_util import * storage=ObjCClass('NSHTTPCookieStorage').sharedHTTPCookieStorage() print(storage.cookies())
I have seen some stackoverflow code using NSKeyedArchiver to turn this into NSData
cookieData= ObjCClass('NSKeyedArchiver').archivedDataWithRootObject_(storage.cookies())
though i have not yet had luck in using NSKeyedUnarchiver to go from data back to an cookie storage object (just have not tried hard yet)
You can also get the binary cookie file from
cookiefile=re.findall(r'/private[^,]*',str(ObjCInstance(storage._cookieStorage()).description()))[0] cookiedata=open(cookiefile).read()
-
From Apple Developer website .
Accessing a Webpage
In Share extensions (on both platforms) and Action extensions (iOS only), you can give users access to web content by asking Safari to run a JavaScript file and return the results to the extension. You can also use the JavaScript file to access a webpage before your extension runs (on both platforms), or to access or modify the webpage after your extension completes its task (iOS only). For example, a Share extension can help users share content from a webpage, or an Action extension in iOS might display a translation of the user’s current webpage.
To add webpage access and manipulation to your app extension, perform the following steps:
Create a JavaScript file that includes a global object named ExtensionPreprocessingJS. Assign a new instance of your custom JavaScript class to this object.
In the NSExtensionActivationRule dictionary in your app extension’s Info.plist file, give the NSExtensionActivationSupportsWebPageWithMaxCount key a nonzero value. (To learn more about the activation rule dictionary, see Declaring Supported Data Types for a Share or Action Extension.)
When your app extension starts, use the NSItemProvider class to get the results returned by the execution of the JavaScript file.
In an iOS app extension, pass values to the JavaScript file if you want Safari to modify the webpage when your extension completes its task. (You use the NSItemProvider class in this step, too.)
To tell Safari that your app extension includes a JavaScript file, add the NSExtensionJavaScriptPreprocessingFile key to the NSExtensionAttributes dictionary. The value of the key should be the file that you want Safari to load before your extension starts. For example:<key>NSExtensionAttributes</key> <dict> <key>NSExtensionJavaScriptPreprocessingFile</key> <string>MyJavaScriptFile</string> <!-- Do not include the ".js" filename extension --> </dict>
-
I was able to get the source code of the current Safari web page using the following code:
req = appex.get_web_page_info()
print(req['html'])From there I was able to parse through the HTML using BeautifulSoup.