recognize Number from Picture

[DavinE]

@cvp,

I know why it doesn't work for me, he analyzes the image differently ... Unfortunately, the output is different for whatever reason!

So it would work, but unfortunately not an option for me 😊

But thanks for your help!

[JonB]

I'm having trouble following the conversation -- is the problem still that you are getting the NoneType error?

Try this in your delegate handler
infodict=ObjCInstance(info)
print(infodict)
print (infodict.allKeys())

(At least, I think that's how one gets the keys from a NSDictionary.. you might need a for loop)

There are a few possible key values that the info object can return. @cvp, maybe you know a better way if printing out all options ...

[cvp]

@JonB said:

maybe you know a better way if printing out all options ...

Unfortunately, no

[cvp]

@JonB said:

is the problem still that you are getting the NoneType error?

For me the problem has disappeared when I used original photo. in this case you can set the allowsEditing flag to False

[DavinE]

@JonB said:

I'm having trouble following the conversation -- is the problem still that you are getting the NoneType error?

Try this in your delegate handler
infodict=ObjCInstance(info)
print(infodict)
print (infodict.allKeys())

(At least, I think that's how one gets the keys from a NSDictionary.. you might need a for loop)

There are a few possible key values that the info object can return. @cvp, maybe you know a better way if printing out all options ...

my Problem is the Following one.

i get this output with that settings:
img = ObjCInstance(info)['UIImagePickerControllerEditedImage']
picker.allowsEditing = True

['Versand: LKW-Anlieferung', 'Objekt: 47491709, Fornoff', 'Bestellt von Herrn xxx xxx', 'Position', 'Bezeichnung', 'Menge', 'Preis', '1', '1179629', 'Hager eHZ-Anschlusskassette', 'KU73S20', '1 - St', '74,20', '5-pol. mit Spannungsabgriff', 'Produktinformationen im Internet', '2', '3500670', 'Hager SLS-Schalter 35A', 'HTS335E', '1 - St', '74,50', '3polig E für Sammelschiene', 'Produktinformationen m Internet', '3', '3500156', 'Hager Ausschalter 3polig sperrbar SH363S', '1 - St', '32,44', '63A 2,5TE sperr- u. plombierbar', 'Produktinformationen m Internet', '4', '3532239', 'Hager Schütz 63A 4S 230V', 'ESC463S', '1 St', '55,10', 'brummfrei', '•Produktinformationen im Internet', '5', '1177739', 'Hager Plombierkappe für Schütze 3M ESC003', '1 - St', '0,84', '1 St wurde in Rückstand genommen.', 'Produktinformationen m Internet', 'Auftragsverfolgung:', 'Gesamtwert:', 'EUR', 'Mehrwertsteuer:', '19,0', '%', 'Endbetrag:', 'EUR', 'Es gelten die mit Ihnen vereinbarten Zahlungsbedingungen.', 'Liefertermine sind unverbindlich. Durch die Weltweite COVID-19 Situation kann es seitens der Hersteller zu unvorherseht', 'Lieferschwierigkeiten kommen. Metallzuschläge und Versandkosten sind nicht skontofähig. Diesem Auftrag liegen unsere', 'neugefassten Allgemeinen Lieferbedingungen (Stand Mai 2018) zugrunde, die Sie unter www.moster.de abrufen können.', 'Wir sind hier: Ludwigshafen am Rhein', 'Neustadt an der Weinstraße - Offenbach - Karlsruhe - Weinheim - Kaiserslauter', 'Moster Elektrogroßhandelsges. mbH', 'Niederlassung:', 'Ernst-Boehe-Straße 10', 'Hauptstraße 62', 'Handelsregister Amtsgericht', 'Geschäftsführer', 'Rudolf Peter Moster', '67059 Ludwigshafen', 'Ludwigshafen', 'Telefon (06 21) 5 90 04 - 0', '63924 Kleinheubach', 'HRB-Nr. 2149', 'Götz Aumüller', 'E-Mail info@moster.de', 'Telefon: 09371 4098-0', 'USt-IdNr. DE197966312', 'E-Mail', 'www.moster.de', 'kleinheubach@moster.de']

orderConfirmation -- [['47491709'], ['1179629', '1', '74,20'], ['3500670', '1', '74,50'], ['3500156', '1', '32,44'], ['3532239', '1', '55,10'], ['1177739', '1', '0,84']]

that is correct and i can work with it.

when i use the following settings:
img = ObjCInstance(info)['UIImagePickerControllerOriginalImage']
picker.allowsEditing = False

get i this:

['Rudolf Peter Moster', 'Geschäftsführer', 'Jelinwny 2109', '%', 'VI TE', "OL'99", '0,84', 'an3', 'an3', '06 L', "O7'7 L", 'Preis', '09 7L', 'IS L', 'IS L', 'ZI899626130 INPHISN', 'IS L', '†S I', 'IS I', 'Endbetrag:', 'nupuebs}wyJejs|BeJs)epuBH', '6717 IN-AHH', 'Ludwigshafen', "L ZOZ'60 t0 :wnjea", 'Mehrwertsteue', 'pemqweses', 'Menge', '0-8605 LLEE', 'E-Mail', 'OZSE/NY', 'HTS335E', 'SE9tOS]', 'SE9CHS Jequueds Bijodg de}leyossny JeBeH', 'joule}u| w! usuonewojurynpold', 'wu6qesbunuueds Hw "jod-g', 'Hosseyssn|yosuy-ZHeJeBeH', '¡DEISNON - ujeya we uajeysbimpn ulely puis 11M 13 neIsIasjey - WeyujaM - aynsuey - usequeHo - e sensujam Jap ue', 'puis ou! queje!7 youna yo! pulque un D¡eM}eM e!p COVID-19 Situation kann es seitens der yesIeyJoAun nz Je||e sIeH', 'eJesun uebe!l beuyny wesela (ersandkosten sind nicht skontofähig. pun edelyosnzijejen uswwoy us ex bue!MupsiejeIT', 'in Rückstand genommen', 'E00OSE WE ezInYas any oddexue|qwold106eH', '}ewe}u| w! Produktinformationen', 'kleinheubach@moster.d', 'ypegneyuie|x +7689', '79 eyensidneH', ':BunsseuepeIN', 'Telefon:', 'euelyos|ewes uny a biods', 'Bezeichnung', 'Bestellt uOA leuq3 youin uwue', '‹önnen. uenqe unter www.moster.de Sie olp epunJ6nz (8 LOZ ¡ew puels) neugefassten Allgemeinen Lieferbedingungen', '¡eule}ui wi usuonewoyurynpold', '}eulejuj wi usuoneuojurynpold', '63A 2,5TE sperr- u. plombierbar', 'euse}ui w! usuonewojurynpold', 'V98 JO}leyS-S7S JeBeH', 'A0EZ St V$9 ZInyOS JeBEH', '@ wunig', 'ubunbuipeqsbunjuezueuequieuenueuyi HIW alp ue106 sg', 'unMIS', 'Moser Elektrogroßhandelsges Hqu', '0 - 90 04 , ( L7 90) uojejel', 'E-Mail info@moster.de', 'Ernst-E ehe-Straße 10', "ep'Je}so u M", 'uejeys6impn 7 690 49', 'Helqo 47491709, Fornoff', 'Versand: LKW-Anlieferunc', '3500670', '3500156', '3532239', '6EZZLI L', 'nbjouansbeyny', '1179629', 'y', 'V', '+', 'uon!sod', 'C']

orderConfirmation -- []

i think the language is the Problem...

[cvp]

@DavinE said:

that is correct and i can work with it.

Thus you don't have anymore the error on img.ptr

[DavinE]

@cvp said:

@DavinE said:

that is correct and i can work with it.

Thus you don't have anymore the error on img.ptr

with your changes, no:

img = ObjCInstance(info)['UIImagePickerControllerOriginalImage']
picker.allowsEditing = False

only that, that the recognized Text makes no sense...

[cvp]

@DavinE did you edit the picked image? In this case, you don’t see the original image but it is still hidden in the edited image.

[DavinE]

@cvp,
It's for both Tests the same Image, and the image is Original nothing in it is edited.

[cvp]

@DavinE could you post in Imgur the image that does not give correct info with my code

[DavinE]

@cvp said:

@DavinE could you post in Imgur the image that does not give correct info with my code

Sure, here it is:

https://imgur.com/a/rbOtAg0

[cvp]

@DavinE that is the immediate result of my code after having picked your image, without passing via an "use" screen

1179629
3500670
3500156
3532239
1177739

With

    img = ObjCInstance(info)['UIImagePickerControllerOriginalImage'] 
.
.
.
def get_photo_action(picker_type,sender):
.
.
.
        picker.allowsEditing = False

Is that not what you want?

[DavinE]

@cvp,

i add more code into it because i want more Data from the Pic.

This is mine recognize Function:

def recognize(self, image_data):
            req = self.VNRecognizeTextRequest.alloc().init().autorelease()
            handler = self.VNImageRequestHandler.alloc().initWithData_options_(
                image_data, None
            ).autorelease()
            success = handler.performRequests_error_([req], None)
            if success:
                recognized_text = [str(result.text()) for result in req.results()]
                # print(recognized_text)
                price = None
                orderConfirmation = []
                for txt in recognized_text:
                    if 'Objekt' in txt:
                        txt = txt.replace(',', '')
                        # Einlesung der Ersten Seite der Auftragsbestätiigung
                        if len(str([int(temp)for temp in txt.split() if temp.isdigit()][0])) is 8:
                            if orderConfirmation is None:
                                orderConfirmation = [int(temp)for temp in txt.split() if temp.isdigit()][0]
                            else:
                                if orderConfirmation != [int(temp)for temp in txt.split() if temp.isdigit()][0]:
                                    if console.alert(
                                        'Objektnummer Fehler ?',
                                        f'Die Angabe welcher in dem Objekt hinterlegt ist unterscheiden sich:\nAktive Nr.: {orderConfirmation}\nNeue Nr.: {[int(temp)for temp in txt.split() if temp.isdigit()][0]}',
                                        'weiter', 'abbrechen',
                                        hide_cancel_button=True
                                    ) is 2:
                                        sys.exit(0)
                        else:
                            if orderConfirmation is None:
                                console.alert('Objektnummer Fehler!!', 'An der Einzulesenden Auftragsbestätigung wurde keine Projektnummer gefunden!!', hide_cancel_button=False)
                            else:
                                console.alert('Objektnummer ??', 'An der Einzulesenden Auftragsbestätigung wurde keine Projektnummer gefunden ??\nGehört dem gelichen kunden an ?', 'weiter', hide_cancel_button=False)
                                        
                    if len(txt) == 7 and txt.isnumeric():
                        orderNumber = txt
                    if 'St' in txt and not len(txt) >= 8:
                        price = True
                        piece = '+%s' % (re.findall('([0-9]+)', txt)[0])
                    if price:
                        if ',' in txt and not len(txt) >= 7 or '.' in txt and not len(txt) >= 7:
                            txt = txt.replace(',', '.')
                            orderConfirmationPrice = float(txt)
                                
                            orderConfirmationPrice = round((orderConfirmationPrice + ((orderConfirmationPrice * 25) / 100)), 2)
            
                            price = False
                            
                            # designation__short = SQL
                            # QRCode = SQL

                            designation__short = 'some text'
                            QRCode = '0001'

                            temp_orderNumber = []
                            for temp_requestOrderConfirmation in orderConfirmation:
                                temp_orderNumber.append(temp_requestOrderConfirmation[1])

                            if not orderNumber in temp_orderNumber:
                                orderConfirmation.append([designation__short, orderNumber, piece, txt, orderConfirmationPrice, QRCode, ''])
                            else:
                                console.alert('Lieferantennummer existiert', f'Die Gescannte Lieferantennummer existiert bereits in der Liste:\n{orderNumber}', 'überspringen', hide_cancel_button=True)
                            del orderNumber, piece
                        
                print(f'orderConfirmation -- {orderConfirmation}')
                print(len(orderConfirmation))
            else:
                console.hud_alert('Failed to recognize anything')

and here it only works with the Edited mode...

[cvp]

@DavinE I guess that you recognize function has bugs because with mine, and

    img = ObjCInstance(info)['UIImagePickerControllerOriginalImage']
        picker.allowsEditing = False

I get all infos from your image with

def recognize(image_data):
        req = VNRecognizeTextRequest.alloc().init().autorelease()
        handler = VNImageRequestHandler.alloc().initWithData_options_(
            image_data, None
        ).autorelease()
        success = handler.performRequests_error_([req], None)
        if success:
            recognized_text = [
                str(result.text())
                for result
                in req.results()
            ]
            for txt in recognized_text:
               if 1==1:#len(txt) == 7 and txt.isnumeric():
                   print(txt)
        else:
            dialogs.hud_alert('Failed to recognize anything')

With OriginalImage, Your code hangs if I do not skip these lines

                                    print('here')
                                    continue
                                    if console.alert('Objektnummer Fehler ?',f'Die Angabe welcher in dem Objekt hinterlegt ist unterscheiden sich:\nAktive Nr.: {orderConfirmation}\nNeue Nr.: {[int(temp)for temp in txt.split() if temp.isdigit()][0]}',hide_cancel_button=True) is 2:
                                        sys.exit(0)

[DavinE]

@cvp said:

@DavinE I guess that you recognize function has bugs because with mine, and

    img = ObjCInstance(info)['UIImagePickerControllerOriginalImage']
        picker.allowsEditing = False

I get all infos from your image with

def recognize(image_data):
        req = VNRecognizeTextRequest.alloc().init().autorelease()
        handler = VNImageRequestHandler.alloc().initWithData_options_(
            image_data, None
        ).autorelease()
        success = handler.performRequests_error_([req], None)
        if success:
            recognized_text = [
                str(result.text())
                for result
                in req.results()
            ]
            for txt in recognized_text:
               if 1==1:#len(txt) == 7 and txt.isnumeric():
                   print(txt)
        else:
            dialogs.hud_alert('Failed to recognize anything')

With OriginalImage, Your code hangs if I do not skip these lines

                                    print('here')
                                    continue
                                    if console.alert('Objektnummer Fehler ?',f'Die Angabe welcher in dem Objekt hinterlegt ist unterscheiden sich:\nAktive Nr.: {orderConfirmation}\nNeue Nr.: {[int(temp)for temp in txt.split() if temp.isdigit()][0]}',hide_cancel_button=True) is 2:
                                        sys.exit(0)

That was my mistake.....

you need to add: @ui.in_background
like that:

    @ui.in_background
    def recognize(self, image_data):

then it works and you see i get nothing as a result in my array only with the Edited Mode

[cvp]

@DavinE I have the same result with both edited or original BUT if you change the mode, please restart Pythonista before each new run, because you change a delegate of a class you created.

orderConfirmation -- [['some text', '1179629', '+1', '74.20', 92.75, '0001', ''], ['some text', '3500670', '+1', '74.50', 93.12, '0001', ''], ['some text', '3500156', '+1', '32.44', 40.55, '0001', ''], ['some text', '3532239', '+1', '55.10', 68.88, '0001', ''], ['some text', '1177739', '+1', '0.84', 1.05, '0001', '']]
5

[DavinE]

@cvp said:

@DavinE I have the same result with both edited or original BUT if you change the mode, please restart Pythonista before each new run, because you change a delegate of a class you created.

orderConfirmation -- [['some text', '1179629', '+1', '74.20', 92.75, '0001', ''], ['some text', '3500670', '+1', '74.50', 93.12, '0001', ''], ['some text', '3500156', '+1', '32.44', 40.55, '0001', ''], ['some text', '3532239', '+1', '55.10', 68.88, '0001', ''], ['some text', '1177739', '+1', '0.84', 1.05, '0001', '']]
5

It doesn't work for me ...
This is all of my code, maybe you will find a bug:

# https://raw.githubusercontent.com/mikaelho/pythonista-misc/master/pic2text.py
import ui, dialogs, re, console
from ctypes import c_void_p
from objc_util import ObjCClass, UIColor, ObjCInstance, on_main_thread, c, create_objc_class, NSBundle, load_framework


class main():
    def __init__(self):
        def imagePickerController_didFinishPickingMediaWithInfo_(self, cmd, picker, info):
            pick = ObjCInstance(picker)
            pick.setDelegate_(None)
            ObjCInstance(self).release()
            pick.dismissViewControllerAnimated_completion_(True, None)
            
            img = ObjCInstance(info)['UIImagePickerControllerOriginalImage']
            png_data = ObjCInstance(UIImagePNGRepresentation(img.ptr))
            recognize(png_data)

        load_framework('Vision')
        self.VNRecognizeTextRequest = ObjCClass('VNRecognizeTextRequest')
        self.VNImageRequestHandler = ObjCClass('VNImageRequestHandler')

        (picker_photos, picker_camera) = (0, 1)

        UIImagePNGRepresentation = c.UIImagePNGRepresentation
        UIImagePNGRepresentation.argtypes = [c_void_p]
        UIImagePNGRepresentation.restype = c_void_p

        recognize = self.recognize

        self.root = ui.View()
        self.root.background_color = 'white'

        self.SUIViewController = ObjCClass('SUIViewController')

        self.MyPickerDelegate = create_objc_class('MyPickerDelegate', methods=[imagePickerController_didFinishPickingMediaWithInfo_], protocols=['UIImagePickerControllerDelegate'])

        b1 = ui.ButtonItem()
        b1.tint_color='black'
        b1.image = ui.Image('iob:ios7_photos_32')
        b1.action = partial(self.get_photo_action,0)
        b2 = ui.ButtonItem()
        b2.tint_color='black'
        b2.image = ui.Image('iob:camera_32')
        b2.action = partial(self.get_photo_action,1)
        b3 = ui.ButtonItem()
        b3.tint_color='black'
        b3.image = ui.Image('iob:ios7_folder_outline_32')
        b3.action = self.get_file_action

        self.root.right_button_items = (b1,b2,b3)

        self.root.present()

    @on_main_thread
    def get_photo_action(self, picker_type,sender):
            picker = ObjCClass('UIImagePickerController').alloc().init()
            
            delegate = self.MyPickerDelegate.alloc().init()
            picker.setDelegate_(delegate)
            
            picker.allowsEditing = False
            picker.sourceType = picker_type

            vc = self.SUIViewController.viewControllerForView_(
                self.root.objc_instance)
            vc.presentModalViewController_animated_(picker, True)

    @ui.in_background
    def recognize(self, image_data):
            req = self.VNRecognizeTextRequest.alloc().init().autorelease()
            handler = self.VNImageRequestHandler.alloc().initWithData_options_(
                image_data, None
            ).autorelease()
            success = handler.performRequests_error_([req], None)
            if success:
                recognized_text = [str(result.text()) for result in req.results()]
                # print(recognized_text)
                price = None
                orderConfirmation = []
                for txt in recognized_text:
                    if 'Objekt' in txt:
                        txt = txt.replace(',', '')
                        # Einlesung der Ersten Seite der Auftragsbestätiigung
                        if len(str([int(temp)for temp in txt.split() if temp.isdigit()][0])) is 8:
                            if orderConfirmation is None:
                                orderConfirmation = [int(temp)for temp in txt.split() if temp.isdigit()][0]
                            else:
                                if orderConfirmation != [int(temp)for temp in txt.split() if temp.isdigit()][0]:
                                    if console.alert(
                                        'Objektnummer Fehler ?',
                                        f'Die Angabe welcher in dem Objekt hinterlegt ist unterscheiden sich:\nAktive Nr.: {orderConfirmation}\nNeue Nr.: {[int(temp)for temp in txt.split() if temp.isdigit()][0]}',
                                        'weiter', 'abbrechen',
                                        hide_cancel_button=True
                                    ) is 2:
                                        sys.exit(0)
                        else:
                            if orderConfirmation is None:
                                console.alert('Objektnummer Fehler!!', 'An der Einzulesenden Auftragsbestätigung wurde keine Projektnummer gefunden!!', hide_cancel_button=False)
                            else:
                                console.alert('Objektnummer ??', 'An der Einzulesenden Auftragsbestätigung wurde keine Projektnummer gefunden ??\nGehört dem gelichen kunden an ?', 'weiter', hide_cancel_button=False)
                                        
                    if len(txt) == 7 and txt.isnumeric():
                        orderNumber = txt
                    if 'St' in txt and not len(txt) >= 8:
                        price = True
                        piece = '+%s' % (re.findall('([0-9]+)', txt)[0])
                    if price:
                        if ',' in txt and not len(txt) >= 7 or '.' in txt and not len(txt) >= 7:
                            txt = txt.replace(',', '.')
                            orderConfirmationPrice = float(txt)
                                
                            orderConfirmationPrice = round((orderConfirmationPrice + ((orderConfirmationPrice * 25) / 100)), 2)
            
                            price = False
                            
                            # designation__short = SQL
                            # QRCode = SQL

                            designation__short = 'some text'
                            QRCode = '0001'

                            temp_orderNumber = []
                            for temp_requestOrderConfirmation in orderConfirmation:
                                temp_orderNumber.append(temp_requestOrderConfirmation[1])

                            if not orderNumber in temp_orderNumber:
                                orderConfirmation.append([designation__short, orderNumber, piece, txt, orderConfirmationPrice, QRCode, ''])
                            else:
                                console.alert('Lieferantennummer existiert', f'Die Gescannte Lieferantennummer existiert bereits in der Liste:\n{orderNumber}', 'überspringen', hide_cancel_button=True)
                            del orderNumber, piece
                        
                print(f'orderConfirmation -- {orderConfirmation}')
                print(len(orderConfirmation))
            else:
                console.hud_alert('Failed to recognize anything')

    def get_file_action(self, sender):
        f = console.pick_document()
        with open(f, mode='rb') as fil:
            img_data = fil.read()
            self.recognize(img_data)


if __name__ == '__main__':
    main() 

[cvp]

@DavinE if thé wanted answer is not

orderConfirmation -- [['some text', '1179629', '+1', '74.20', 92.75, '0001', ''], ['some text', '3500670', '+1', '74.50', 93.12, '0001', ''], ['some text', '3500156', '+1', '32.44', 40.55, '0001', ''], ['some text', '3532239', '+1', '55.10', 68.88, '0001', ''], ['some text', '1177739', '+1', '0.84', 1.05, '0001', '']]
5

Could you give me what you hope as answer

[DavinE]

@cvp said:

@DavinE if thé wanted answer is not

orderConfirmation -- [['some text', '1179629', '+1', '74.20', 92.75, '0001', ''], ['some text', '3500670', '+1', '74.50', 93.12, '0001', ''], ['some text', '3500156', '+1', '32.44', 40.55, '0001', ''], ['some text', '3532239', '+1', '55.10', 68.88, '0001', ''], ['some text', '1177739', '+1', '0.84', 1.05, '0001', '']]
5

Could you give me what you hope as answer

that is correct.
But i don‘t get it....

My Output is empty...

[cvp]

@DavinE did you remove Pythonista from memory and restart it?