Reverse engineering challenge to cvp

mikael

@cvp, @JonB, wow! I was away for just a little while. And I thought this would probably be something that could not be done.

If I followed what you are doing:

We can add to search results, presumably swizzling in pythonista-startup.
We can hijack the help display to show the relevant help.

I guess we can get the term searched for directly from the editor selection, and then do the custom search as part of the setSearchResults_ swizzle?

Still need to find some search engine solution, and a way for our custom modules to register the help documentation to be indexed and opened up when a relevant hit is selected. Do we know of some efficient solution that iOS would provide as a built-in?

cvp

@mikael Rome was not built in a day... It was a first step.

mikael

@cvp, don’t get me wrong, I am truly impressed with the secret sauce distilled in such a short time.

But now that it looks like it could be actually possible, I started thinking what would be needed to make this generally useful.

cvp

@mikael said:

I guess we can get the term searched for directly from the editor selection,

def setSearchResults_(_self,_sel,_search_results):
	self=ObjCInstance(_self)	# PA2QuickHelpViewController
	print('search term = ',self.searchTerm())

cvp

@mikael said:

But now that it looks like it could be actually possible, I started thinking what would be needed to make this generally useful.

I had understood. Sorry but I always try to put some humor 😀

cvp

Step 2, assuming you have your own doc in a file like Pythonista local help, a zip containing a tree of html files.
This script only to test speed of a user search on the entire Pythonista doc.
Search is not really perfect, and does not yet identify if the found description is for a module, a class, a function etc...as the front icon allows it.
But it is quick, even with my quick and dirty Python code, as usual.

Try with help on nsurl for instance. If you try on str, it is slower because this word exists in all files.

# https://forum.omz-software.com/topic/6244/reverse-engineering-challenge-to-cvp
import swizzle
from objc_util import *
import urllib.parse

def initWithURL_(_self,_sel, _url):
        '''called with an nsurl. lets try hijacking the url, to show google'''
        url = ObjCInstance(_url)
        #print(url)
        if 'https' in str(url):
          i = str(url).find('http')
          t = str(url)[i:]
          url = nsurl(t)
          #print(url)
        elif 'myzip://' in str(url):
          i = str(url).find('myzip://')
          t = str(url)[i+2:]
          url = nsurl(t)
          #print(url)
        self=ObjCInstance(_self) # PA2QuickHelpContentViewController
        rtnval = self.originalinitWithURL_(url) 
        return rtnval.ptr


cls=ObjCClass('PA2QuickHelpContentViewController')
swizzle.swizzle(cls,'initWithURL:',initWithURL_)

cls2 = ObjCClass('PA2QuickHelpViewController')

def setSearchResults_(_self,_sel,_search_results):
	self=ObjCInstance(_self)	# PA2QuickHelpViewController
	search_term = str(self.searchTerm()).lower()
	search_results = ObjCInstance(_search_results)
	#print(search_results)
	new_search_results = []
	for elem in search_results:
		new_search_results.append(ns(elem))
		
	# Assume you have your own doc as zipped tree of html files
	doc_zip = '/private/var/containers/Bundle/Application/34BAEE1A-BC33-4D6F-A0C1-B733E4991F31/Pythonista3.app/Documentation.zip'
	import zipfile
	with zipfile.ZipFile(doc_zip, 'r') as zipObj:
		# Get list of files names in zip
		listOfiles = zipObj.namelist()
		for elem in listOfiles:
			if elem.startswith('py3') and elem.endswith('.html'):
				content = zipObj.read(elem).decode('UTF-8').lower()
				lines = content.split('\n')
				for line in lines:
					if line.find(search_term) >= 0:
						my_path = 'myzip://' + doc_zip + '/' + elem
						new_search_results.append(ns({'path':my_path, 'rank':10, 'title':search_term, 'type':'mod'}))
						#print(my_path,line)
						break

	# Assume you have your own doc on the web
	new_search_results.append(ns({'path':"https://github.com/mikaelho/pythonista-gestures", 'rank':10, 'title':"gestures", 'type':'mod'}))
	
	#print('search:',self.searchTerm(),'results=',new_search_results)
	self.originalsetSearchResults_(new_search_results)

swizzle.swizzle(cls2,'setSearchResults:',setSearchResults_)

mikael

@cvp, thanks! Wanted to try this out, but

/private/var/containers/Bundle/Application/34BAEE1A-BC33-4D6F-A0C1-B733E4991F31/Pythonista3.app/Documentation.zip

... is not found on my phone. Probably the cryptic code part of the path is different. Could you please remind me how to find the right path, as it is different from the Document files?

cvp

@mikael strange, in this topic you got it, don't you?

JonB

I wonder if pydoc.ModuleScanner.run would do the trick, rather than requiring a way of registering docs. Just use built in docstrings.
Or a modified version that searches only non-built in modules. ModuleScanner seems to search all modules for docstring matching the keyword. Then could generate pydoc htmldoc on the fly, maybe.

JonB

@mikael uncomment the first print url line, then select a normal help item.

omz

I'm truly impressed by what you've been able to accomplish in such a short amount of time! It might even make sense to provide a built-in hook for this kind of thing, not quite sure yet about the implications.

cvp

@mikael Did you find the right path?

cvp

@omz It could be sufficient that you allow the concaténation of a user (zipped) doc file to the standard one.

omz

@cvp I don't know, I would guess that augmenting the search results with online sources could potentially be more useful, not sure though.

cvp

@omz I agree but a big advantage of your (marvelous, did I already say it 😀?) application is that the entire doc is local and thus available off-line.

omz

@cvp That's certainly true (and thanks for the kind words!), and I can definitely see both use cases, but running a script hook would allow you to do that as well, I guess.

JonB

@omz Are you using your own pregenerated keyword index in your search?
Or are you actually indexing on the fly?

Also, not sure if you have ever seen jedi-vim-- since jedi already (sort of) knows what module a highlighted term belongs to, one option might be to use something akin to jedi-vim for showing the pydoc/docstring associated with an object. There are some other jedi-vim features which would be cool , (showing function prototypes wlin the autocomplete) though I'm not sure what that would really look like in iOS without getting really cluttered.

It would totally be useful to just search docstrings in user modules that are not already included in the built in docs, and just show that, even if not fancy HTML formatted -- hooking user generated .zips would be cool, but only useful for docs specifically written to be indexed by pythonista. Going with generic pydoc would allow any module to hook into the quickhelp. My quick tests with pydoc.ModuleSearcher seem like it is pretty quick ( and also provides callback and quit capability, so ought to be pretty performance)

omz

@JonB Yes, I'm using a pregenerated keyword index, though I guess using jedi to extract docstrings could be an interesting option as well, might be more accessible than a script hook.

cvp

@mikael said:

Could you please remind me how to find the right path, as it is different from the Document files?

open doc tab (via ? button), then swipe to right and run this script

# open doc tab (via ? button), then swipe to right and run this script
import console
import editor
from   objc_util import *
import ui

@on_main_thread
def searchDocPath():
	win = ObjCClass('UIApplication').sharedApplication().keyWindow()
	main_view = win.rootViewController().view() 
	ret = '' 
	def analyze(v,indent):
		ret = None
		for sv in v.subviews():
			#print(indent,sv._get_objc_classname())
			if 'UIWebBrowserView' in str(sv._get_objc_classname()):
				print(sv.webView().mainFrameURL())
			ret = analyze(sv,indent+'  ')
			if ret:
				return ret
	ret = analyze(main_view,'')
	return ret

if __name__ == "__main__":
	searchDocPath()

cvp

@mikael I didn't know where to post this, congratulations 😀
Ha, Finland