Recognize text from picture

chrillek

Hi,
I'm aware that this thread is about a year old. But maybe someone can nevertheless alighten me. I'm trying to do a similar thing in JavaScript for Automation (JXA), and I see this line in your example:

for result in req.results():
            print(result.text())

translated to JavaScript, that's

results.forEach(r => {
      console.log(r.text);
 })

and that works like a charm. I'm just wondering why, since according to Apple's documentation, the results object doesn't even have a text property, only string (cf. https://developer.apple.com/documentation/vision/vnrecognizedtext?language=objc)

I was first wondering if text is perhaps a nice Python thing, but since the same works in JavaScript, I'm sure that I'm missing something obvious in Apple's documentation. Does anyone know what (and where I should be looking)?

Thanks a lot in advance
Christian

JonB

Does string not work?

Often there are undocumented or decrecates features available in objc objects. Often we just poke around using autocomplete (which ultimately uses some of the introspection objc features of the objc runtime (which let you get a list of methods or instance vars, etc)

chrillek

Does string not work?

It does, but only in a very convoluted way, like so:

results.forEach(r => {
      console.log(r.topCandidates(1).js[0].string.js)
})

The js in the middle is required to convert the ObjC array returned by topCandidates to a JavaScript array (and again to convert the NSString returned by string to a JS string). But using string directly at r does not work.

we just poke around using autocomplete

I gues that happens in XCode (the poking around)?

JonB

No, the exploration happens in pythonista, in the console. Once you have an object, dir(variable) lists the methods and such, or frankly just typing a letter and autocomplete suggestions does it's thing.

If you're not using a bridging library like
https://github.com/TooTallNate/NodObjC
I'd suggest that you do, since it might take care of a lot of the annoying bits like converting every type to js equivalents, and let's you access some of the dynamic introspection stuff that makes objc pretty neat.

Under the hood, there are objc runtime functions that let you get lists of method names. For instance,see
https://github.com/jsbain/objc_hacks/blob/master/print_objc.py
For how you can do it in python. Or, look at the NodObjC code for class.js and core.js -- it looks like it does something similar, using the objc copy_methodsList, etc, and adds those as J's callable functions to the prototype. Then your favorite J's debugger ought to show you what is there...

JonB

In this case, looking at the headers for VNTextObservation shows the text attribute.

https://github.com/xybp888/iOS-Header/blob/master/13.0/Frameworks/Vision.framework/VNRecognizedTextObservation.h

chrillek

Thanks a lot for that. Apple's documentation doesn't mention any of these properties :-(

Arsh_gabbi

This post is deleted!

ItsKruml

How do I take the recognized stuff and make a mathematical problem to solve out of it?

Attirail

Hello,
thanks for your subject. I made an app 4 yeard ago with Pythonista and i found nothing to recognize from camera the 15 number of a bingo card like this one.
My app still working with writing manually the number.

i will try now to update it with text recognition and see if it can work with your code.

alt text
On this example what i need as a result is : (18,37,55,62,89,9,13,36,48,77,5,26,58,66,80)
I think i can have some issues with the 'noise' around the number i need (text outside the tab or the repetition in small character). Is it possible to make it first see the tab and only look inside each row and column and not outside ?

Hope we can do something robust :)

JonB

@Attirail that will be a little tricky because of the small and large numbers in each box. Detecting the outer box probably won't be too bad, then if you have known grid dimensions I'd think you could then find the grid without much trouble. But then you'd have to essentially only grab a specific region of each grid.