Reverse engineering challenge to cvp

[mikael]

Is there a way to add help docs for custom components, so that they would be included when you select Help... from the editor pop-up?

[cvp]

@mikael Really, do you think I have time to play? 😂 Yes, I have it, ok, ok, sorry.

Seriously, I suppose @omz could answer better than me.

But, I suppose it could be possible, but so, I don't know how.
It should be possible to intercept (monkey patching?) the help in the menu.
I remember that I've already searched, without success, to add a user option in this menu, but some years ago, I was too young for that 😀

If @omz or @jonb (the masters) do not answer, I agree to think about it...

[cvp]

@mikael

Really, do you think I have time to play?

It was humor...

[JonB]

Are we talking about the pythonista docs that lives in a standalone tab, or the pop up version that shows up from the context menu?

I think I was able to add items to the context menu years ago, but the problem was that each editor tab has its own that gets overwritten everytime you change tabs , so there wasn't a way to get changes to stick. But there is probably a callback in the controller of SUEditor or something to show the local help overlay that could perhaps be swizzled.

I don't think we have write access to the main help menu files -- otherwise we could write our own HTML. Iirc the help menu is just a webview .

[JonB]

Ok, a few minutes with obj_browser in my objc_hacks repo:
There seem to be three classes involved with quick help:
PA3QuickHelpViewController is the main one, with a navigation view and a content view controller which are used to display content, etc.

When you tap help, an PA3QuickHelpViewController instance is created. Let’s call that qhvc for short.

Then, qhvc.initWithSearchTerm(searchTerm) is called, with an NSString argument containing the selected text. This object has a searchTerm attribute which presumably gets set there.

A moment later, someone calls

qhvc.setSearchResults(search_results)

Where search_results is an NSArray of NSDictionary, that look like this:

<b'__NSArrayI': (
        {
        path = "py3/ios/ui.html";
        rank = 3;
        title = "ui.TableViewCell";
        type = class;
    },
        {
        path = "py3/ios/ui.html";
        rank = 3;
        title = "ui.TableViewCell.accessory_type";
        type = attribute;
    },
...

So, setSearchResults is the one to swizzle — presumably we could hijack the results, adding in our own search results.

When you click a link, the search result dict is returned, and then the b'PA2QuickHelpContentViewController gets created, and initWithURL is called with an NSURL the form

<b'NSURL': zip:///private/var/containers/Bundle/Application/710C2D0F-E012-441A-9075-28DA41A05518/Pythonista3.app/Documentation.zip/py3/Ios/ui.html#TableViewCell>

Presumably the code just pretends the .zip file url to the search results path, and appends the title to the bookmark. So, we’d need to also swizzle initWithURL to either point to the old url, or else some other file of ours, depending on what is in the path.

[cvp]

@JonB said:

Are we talking about the pythonista docs that lives in a standalone tab, or the pop up version that shows up from the context menu?

I think @mikael was talking about the popup menu (UIMenuController)

[cvp]

@JonB said:

So, setSearchResults is the one to swizzle — presumably we could hijack the results, adding in our own search results.

When you click a link, the search result dict is returned, and then the b'PA2QuickHelpContentViewController gets created, and initWithURL is called with an NSURL the form

<b'NSURL': zip:///private/var/containers/Bundle/Application/710C2D0F-E012-441A-9075-28DA41A05518/Pythonista3.app/Documentation.zip/py3/Ios/ui.html#TableViewCell>
Presumably the code just pretends the .zip file url to the search results path, and appends the title to the bookmark. So, we’d need to also swizzle initWithURL to either point to the old url, or else some other file of ours, depending on what is in the path.

Whaaaaaa...

[JonB]

Okay... successfully swizzled initWithURL.
As a proof of concept, this hijacks the help target, and sends it to google.

This is just a proof of concept — next would be to swizzle the search results to insert our own.

import swizzle
from objc_util import *
import urllib.parse

def initWithURL_(_self,_sel, _url):
		'''called with an nsurl. lets try hijacking the url, to show google'''
		url=ObjCInstance(_url)
		#print('url is',url.absoluteURL())
		parsedurl=urllib.parse.urlparse(urllib.parse.unquote(str(url.absoluteString())))
		self=ObjCInstance(_self)
		rtnval= self.originalinitWithURL_(nsurl('http://google.com/search?q={}'.format(parsedurl.fragment)))
		return rtnval.ptr


cls=ObjCClass('PA2QuickHelpContentViewController')
swizzle.swizzle(cls,'initWithURL:',initWithURL_)

[cvp]

@JonB Marvelous, as usual

[cvp]

@JonB said:

next would be to swizzle the search results to insert our own.

We will do it....

cls2 = ObjCClass('PA2QuickHelpViewController')
def setSearchResults_(_self,_sel,_search_results):
	self=ObjCInstance(_self)	# PA2QuickHelpViewController
	search_results = ObjCInstance(_search_results)
	print('search:',self.searchTerm(),'results=',search_results)
	self.originalsetSearchResults_(search_results)

swizzle.swizzle(cls2,'setSearchResults:',setSearchResults_) 

[cvp]

@mikael First step. Not needed to say that it is thanks to @JonB

# https://forum.omz-software.com/topic/6244/reverse-engineering-challenge-to-cvp
import swizzle
from objc_util import *
import urllib.parse

def initWithURL_(_self,_sel, _url):
        '''called with an nsurl. lets try hijacking the url, to show google'''
        url = ObjCInstance(_url)
        #print(url)
        if 'http' in str(url):
          i = str(url).find('http')
          t = str(url)[i:]
          url = nsurl(t)
          #print(url)
        self=ObjCInstance(_self) # PA2QuickHelpContentViewController
        rtnval = self.originalinitWithURL_(url) 
        return rtnval.ptr


cls=ObjCClass('PA2QuickHelpContentViewController')
swizzle.swizzle(cls,'initWithURL:',initWithURL_)

cls2 = ObjCClass('PA2QuickHelpViewController')

def setSearchResults_(_self,_sel,_search_results):
	self=ObjCInstance(_self)	# PA2QuickHelpViewController
	search_results = ObjCInstance(_search_results)
	#print(search_results)
	new_search_results = []
	for elem in search_results:
		new_search_results.append(ns(elem))

	new_search_results.append(ns({'path':"https://github.com/mikaelho/pythonista-gestures", 'rank':10, 'title':"gestures", 'type':'mod'}))
	
	#print('search:',self.searchTerm(),'results=',new_search_results)
	self.originalsetSearchResults_(new_search_results)

swizzle.swizzle(cls2,'setSearchResults:',setSearchResults_)

[mikael]

@cvp, @JonB, wow! I was away for just a little while. And I thought this would probably be something that could not be done.

If I followed what you are doing:

1. We can add to search results, presumably swizzling in pythonista-startup.
2. We can hijack the help display to show the relevant help.

I guess we can get the term searched for directly from the editor selection, and then do the custom search as part of the setSearchResults_ swizzle?

Still need to find some search engine solution, and a way for our custom modules to register the help documentation to be indexed and opened up when a relevant hit is selected. Do we know of some efficient solution that iOS would provide as a built-in?

[cvp]

@mikael Rome was not built in a day... It was a first step.

[mikael]

@cvp, don’t get me wrong, I am truly impressed with the secret sauce distilled in such a short time.

But now that it looks like it could be actually possible, I started thinking what would be needed to make this generally useful.

[cvp]

@mikael said:

I guess we can get the term searched for directly from the editor selection,

def setSearchResults_(_self,_sel,_search_results):
	self=ObjCInstance(_self)	# PA2QuickHelpViewController
	print('search term = ',self.searchTerm()) 

[cvp]

@mikael said:

But now that it looks like it could be actually possible, I started thinking what would be needed to make this generally useful.

I had understood. Sorry but I always try to put some humor 😀

[cvp]

Step 2, assuming you have your own doc in a file like Pythonista local help, a zip containing a tree of html files.
This script only to test speed of a user search on the entire Pythonista doc.
Search is not really perfect, and does not yet identify if the found description is for a module, a class, a function etc...as the front icon allows it.
But it is quick, even with my quick and dirty Python code, as usual.

Try with help on nsurl for instance. If you try on str, it is slower because this word exists in all files.

# https://forum.omz-software.com/topic/6244/reverse-engineering-challenge-to-cvp
import swizzle
from objc_util import *
import urllib.parse

def initWithURL_(_self,_sel, _url):
        '''called with an nsurl. lets try hijacking the url, to show google'''
        url = ObjCInstance(_url)
        #print(url)
        if 'https' in str(url):
          i = str(url).find('http')
          t = str(url)[i:]
          url = nsurl(t)
          #print(url)
        elif 'myzip://' in str(url):
          i = str(url).find('myzip://')
          t = str(url)[i+2:]
          url = nsurl(t)
          #print(url)
        self=ObjCInstance(_self) # PA2QuickHelpContentViewController
        rtnval = self.originalinitWithURL_(url) 
        return rtnval.ptr


cls=ObjCClass('PA2QuickHelpContentViewController')
swizzle.swizzle(cls,'initWithURL:',initWithURL_)

cls2 = ObjCClass('PA2QuickHelpViewController')

def setSearchResults_(_self,_sel,_search_results):
	self=ObjCInstance(_self)	# PA2QuickHelpViewController
	search_term = str(self.searchTerm()).lower()
	search_results = ObjCInstance(_search_results)
	#print(search_results)
	new_search_results = []
	for elem in search_results:
		new_search_results.append(ns(elem))
		
	# Assume you have your own doc as zipped tree of html files
	doc_zip = '/private/var/containers/Bundle/Application/34BAEE1A-BC33-4D6F-A0C1-B733E4991F31/Pythonista3.app/Documentation.zip'
	import zipfile
	with zipfile.ZipFile(doc_zip, 'r') as zipObj:
		# Get list of files names in zip
		listOfiles = zipObj.namelist()
		for elem in listOfiles:
			if elem.startswith('py3') and elem.endswith('.html'):
				content = zipObj.read(elem).decode('UTF-8').lower()
				lines = content.split('\n')
				for line in lines:
					if line.find(search_term) >= 0:
						my_path = 'myzip://' + doc_zip + '/' + elem
						new_search_results.append(ns({'path':my_path, 'rank':10, 'title':search_term, 'type':'mod'}))
						#print(my_path,line)
						break

	# Assume you have your own doc on the web
	new_search_results.append(ns({'path':"https://github.com/mikaelho/pythonista-gestures", 'rank':10, 'title':"gestures", 'type':'mod'}))
	
	#print('search:',self.searchTerm(),'results=',new_search_results)
	self.originalsetSearchResults_(new_search_results)

swizzle.swizzle(cls2,'setSearchResults:',setSearchResults_)

[mikael]

@cvp, thanks! Wanted to try this out, but

/private/var/containers/Bundle/Application/34BAEE1A-BC33-4D6F-A0C1-B733E4991F31/Pythonista3.app/Documentation.zip

... is not found on my phone. Probably the cryptic code part of the path is different. Could you please remind me how to find the right path, as it is different from the Document files?

[cvp]

@mikael strange, in this topic you got it, don't you?

[JonB]

I wonder if pydoc.ModuleScanner.run would do the trick, rather than requiring a way of registering docs. Just use built in docstrings.
Or a modified version that searches only non-built in modules. ModuleScanner seems to search all modules for docstring matching the keyword. Then could generate pydoc htmldoc on the fly, maybe.