I write code that scrapes and displays web pages, fairly standard stuff. Some of my scripts crash one minute then can run the next i.e. no obvious cause. I am thinking Pythonista may be too light weight for what I do so a MacBook may be better. Any thoughts on this? Could I find the same happening on the MacBook?
@hewjet, what are you using for the scraping on Pythonista?
You should use a list comprehension instead of a generator expression on line 17.
@ccc, can’t see what you are replying to.
@mikael perhaps some kind of humor because the topic request is not very detailed?
@cvp, ah, HUMOR! I had it once with caviar.
Check for the pythonista exception file (sorry, I forget the name) -- if the crash was due to an exception, you'll get a cryptic objc traceback that you post here.
The other main reason for crashes is running out of memory. IOS is unsympathetic to poor memory use and will kill your process without warning or explanation.
Looking at your code would help. Manually deleting large objects, like requests responses is a good idea, then call gc.collect(), can sometimes help.
Using a lot of image assets can cause crashes, you have to wrap portions of your code with
with objc_util.autoreleasepool(): # memory hogging code here
Are you using
requests? Are you using threads/asyncio, etc? Given you are doing web scraping, I suspect much of your problem has to do with either what you get back from the websites, or timing and blocking code, etc.
You really ought to post some skeleton code, folks can look for obvious problems.
_objc_exception.txt in Pythonista root
Mikael, JonB, thanks for your replies.
As you correctly determined, my post relates to general stability of Pythonista. Is it just intermittently buggy?
The crash happens after the scrape. I scrape in a separate script not prone to crashing so the scraped data (which is smaller than 1MB) is fixed when reading back and this is an intermittent failure! I read the resulting text file, compute from it then display stock charts. During stock chart accessing it crashes. Totally random. Good one pass, crashes out the next. The point of the crash varies too. The crash is completely out i.e. not a syntax error. Returns back to iOS. Must restart Pythonista.
The memory requirement to display a page of Bigchart cannot be much.
I shall check my error log.
If you force quit pythonista, run the script once, does it ever crash on the first run?
Or only after you run a second time?
There are several ways to crash when scripts run against due to how the globals clearing works.
Very rarely does pythonista "just crash". There is almost always a reason, and when it crashes it is usually repeatable.if you go through the same series of steps.
At which point you can start stepping through to find the exact line causing the crash.
Are you using matplotlib to draw your plots? Matplotlib can cause issues sometimes.
Are you using objc_util anywhere? Any sort of callbacks require that you retain references to valid objects, even across global resets, and this is often a source of problems like "my script always crashes the second time".
Are you using threading, or asyncio?. This is another hard to get right with pythonistas environment.
Are you displaying the plots in an ImageView? If so you need to use the nsreleasepool to manage memory. Images never get cleared unless you call the objc methods, and you will get crashes after a few hundred to thousand images created since starting pythonista.
Are you using webview? That has its own set of considerations.
As others have said, it is hard to debug without seeing code.. post your code, or a sanitized version.
Thanks again JonB,
It crashes first time, anytime, totally random.
There are no globals, no matplotlib, no objc_util, no ImageView, no asyncio, no threading, no webview. I use webbrowser.
I can understand that memory may need to be released but it can crash before any webbrowser call.
Just now Pythonista crashed out to iOS after I stopped execution and while I was thumbing through the results on the console i.e. not a script problem. This seems to answer my question that Pythonista is inherently unstable.
I am also encountering slow execution on my <1MB data txt file so MacBook could be a good investment and I have to take pot luck again in choosing a python interpreter. iCloud seeks also seem slow but that's unavoidable.
But on researching I find the old MacBook range does not offer much faster execution and can be slower at times. Macbook M1 is faster but not by anywhere near as much as Apple is claiming and some apps need reinterpretation by Rosetta which slows it enormously.
All this makes me wonder if re-installing pythonista would help and would it find my old files automatically
What device and OS are you using? I've never heard of anyone crashing in the console, except in a few specific known bugs (for instance, using python 2.7 + matplotlib, at one time, autocomplete crashes with certain objects, but it does so every time.)
Check your site-packages/pythonista_startup, to see if you have installed anything that could be causing issues. Also, did you check the for the _objc_exception file? There is also the @dgelessus startup script which has additional exception handling.
Sometimes, a faulty pythonista install can result in crashes. You might consider backing up your documents folder, then reinstalling Pythonista. Note removing pythonista will result in loss of all files, unless you have both pythonista 2.0 and pythonista 3.0 installed.
I should also ask..are you using pythonista 3, latest app store version? Are you using the python 3.6 interpreter?
@hewjet, the one area where I have seen instability is the console, if I print a lot to it, and fast. If you are dumping a lot of text there, you mighr consider if you really need it, or alternatively putting the text to a file and/or a TextView instead.