Recognize text from picture
This script recognizes text from a camera or photo library picture. Sharing it since iOS 13 has made it this easy, and Apple Shortcuts do not have support for it (yet, I bet).
Adjust languages on the first row.
language_preference = ['fi','en','se'] import photos, ui, dialogs import io from objc_util import * load_framework('Vision') VNRecognizeTextRequest = ObjCClass('VNRecognizeTextRequest') VNImageRequestHandler = ObjCClass('VNImageRequestHandler') def pil2ui(pil_image): buffer = io.BytesIO() pil_image.save(buffer, format='PNG') return ui.Image.from_data(buffer.getvalue()) selection = dialogs.alert('Get pic', button1='Camera', button2='Photos') ui_image = None if selection == 1: pil_image = photos.capture_image() if pil_image is not None: ui_image = pil2ui(pil_image) elif selection == 2: ui_image = photos.pick_asset().get_ui_image() if ui_image is not None: print('Recognizing...\n') req = VNRecognizeTextRequest.alloc().init().autorelease() req.setRecognitionLanguages_(language_preference) handler = VNImageRequestHandler.alloc().initWithData_options_(ui_image.to_png(), None).autorelease() success = handler.performRequests_error_([req], None) if success: for result in req.results(): print(result.text()) else: print('Problem recognizing anything') ```
revision = VNRecognizeTextRequest.currentRevision() supported = VNRecognizeTextRequest.supportedRecognitionLanguagesForTextRecognitionLevel_revision_error_(0, revision, None)
@mikael I had also seen that but it supports French, thus ...buggy?
BTW, I'm impressed from accuracy ( for Latin based texts ).
@mikael @cvp have you tried setting
customWordsattrib of the request? Or, turn off
usesLanguageCorrection? (Since the language is en-US you DON'T want language correction when trying to detect other languages!)
I gather they are looking for words you'd find in an English dictionary. So perhaps façade, or tête-à-tête might recognize, while other examples wouldn't?
I've tried with unknown language codes like xx and yy in setRecognitionLanguages_ and the result is the same. It seems that characters are recognized in any languages.
My last test on a French text was entirely correct
Asie-Pacifique La mission économique belge en Chine cible de cyberattaques massives
Tried with False: à still recognized as a
Edit : even with
Is there any code examples of how to recognize text with ios 12.4.3 for the i pad mini 2 that would be cool to add it to my sodoku app game
def pil2ui(pil_image): buffer = io.BytesIO() pil_image.save(buffer, format='PNG') return ui.Image.from_data(buffer.getvalue())
is memory leaking buffer which has been proven to crash Pythonista when multiple images are processed. A better approach is to use a context manager to force the .close().
def pil2ui(pil_image): with io.BytesIO() as buffer: pil_image.save(buffer, format='PNG') return ui.Image.from_data(buffer.getvalue())
Regardless of language restrictions, I have found the simple and reliable ability to pick text from paper to be useful for me almost weekly - URLs, email addresses, reservation codes, laptop serial numbers, etc.
With the use, I noted that the original script had some issues:
- Difficult to find and open when quickly needed.
- Slow to get from the picked photo to recognized text.
- Results are a pain to copy from the Console as it likes to jump around just as you’ve selected the text to copy.
Point #1 was fixed with a simple Apple Shortcuts shortcut to make the script easy to run.
Point #3 was resolved by presenting the recognized text in a TableView, with tap to copy.
Point #2 took a bit more doing.
photosmodule wants to return PIL images, and that results in two very slow conversions - first the module converts the UIImage to PIL, and then I converted that back to a PNG image for recognition. I found some @cvp code in this thread and replaced
objc_utilpickers, which return PNG data almost directly.
And hey presto! Not just faster recognition, but instantaneous - and with much better quality than with the only contender app I could find (Prizmo Go).
Updated script here.
@mikael Thanks for your great 🎁for New Year
Question for a specialist of this forum.
In the last post of @mikael , I see my user as @cvp but as a black text and not clickable blue, although I've received a notification "Mikael mentioned you...".
How is that possible?
Happy last day of the decade to everyone who shares my calendar!
I finessed the script a bit with the ability to select, copy or share multiple items, and nicer icons.
@cvp, noted and wondered about the lack of the link for your handle, no idea why.
Does this work with the new iPad OS ????
@sodoku, do you mean if the latest versions have included robust support for non-English characters? Not to my knowledge.
So good news I got an iPhone 11 and I’m testing this on it, for sudoku, do the pictures taken save anywhere when used, just curious if I have to delete them because after I use it the pictures don’t show up in my pictures app
Also what’s the updated code posted by Mikael on GitHub used for is it same as this one posted here or not because it’s so much longer and bigger then this code posted on the forum, is it a better version then this one on the forum
It supports taking a picture normally and then selecting it from the photo library when you use the tool, or just snapping a quick ”disposable” in-tool image which is not saved.
This post is deleted!last edited by
So there are a few edits in this thread I don’t know how to piece together to have the best edited version of this???