Welcome!
This is the community forum for my apps Pythonista and Editorial.
For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.
How does the iOS backup work?
-
Re slowness, you might be able to place those many files in folders prefixed by period, in which case I think pythonista doesn't display them (not sure if it still has to crawl them though). You can still get to them programatically, etc.
If you don't want to back up those files, but do want to backup everything else... I wonder if you can stick files in the non-appgroup folder. I don't know whether you can set iCloud to backup one but not the other.
-
https://developer.apple.com/library/archive/qa/qa1719/_index.html
If you don't want to backup certain files, it appears you can call an obi function to exclude a file from backup. I haven't tried it.
-
@JonB Good to know but my problem is that I want the file be part of the backup but I would have imagined that the backup did work by modified blocks only. I already have written such code 40 years ago on mainframe to only save on disk modified parts of big files.
Perhaps, I could
- backup the entire file
- then set it as 'not to be in the backup'
- at each modification, save this in a journal file, which will be in the backup
- when this journal file becomes too big, save the entire file in the backup
- delete the journal file
... and so on
Édit: or perhaps:
-
set the file permanently 'not to be in the backup'
-
copy the file in iCloud Drive/Pythonista3
-
at each modification, save this in a journal file, which will be in the backup
-
when this journal file becomes too big, copy the entire file in iCloud
-
delete the journal file
Perhaps easier way to check date/time/size of the copied version
-
@cvp, one more option could be to use Python standard dbm instead of sql. They can be set to automatically split the storage into several files.
-
@mikael Thanks for the info, but anyway, if I split, for instance, 2GB in 100 files of 20MB, each insert of update of a record could oblige the backup of a lot of bytes instead of a little bit. I'll check that.
-
@mikael I don't know dbm but I've read " A dbm database can only store strings, both as keys and values".
If it is still true, my keys are string but my values are images... -
Could you keep files on disk, and just store file name? That would keep your DB to small text, and images themselves probably change rarely
-
@JonB yes but in this cases, the number of files will stay 30.000, isn't it?
Folders names are authors names
Files names are books names
Files are covers png -
@JonB you are right, the images never change. The only modification in the dB or files tree is to add an author/book/cover. I'm not sure that this info could help to find the right way but so you have all infos
-
@cvp, yeah. I guess you could encode/decode the images, but would have to experiment what kind of control you have on the number of files.
-
If you place the 10000 folders underneath a folder called .DB or something, I wonder if that still causes slowness. I'm thinking I have git repos with hundreds of commits resulting in thousands of files stored in . git, and don't necessarily have massive slowdown in creating or moving files, but I'm not sure.
Could maybe have heirarchys to keep single folders small -- for instance git deals with tons of commits/blobs/trees, each which is its own folder based on the hash. But top level folders are first 2 letters of the hash, thus making it fast to lookup a particular hash -- top folder has < 256 subfolders, etc.
-
@JonB You are right (as usual 🙄), all my 10.000 folders are under a folder that I prefixed with a dot, and creating or moving a file has retrieved its normal speed.
Thus, thanks a lot. I'll keep my 30.000 files without a dB...
I'll have to modify some scripts to access these sub-folders or files because the upper folder is not visible.Although, I still think that iOS backup should not be so designed, by file...But, ok, it is not vital.
A last time (for today), thanks for your time. -
-
Unforeseen consequence: uploads via SFTP to my backup server ADrive.com does not work anymore from sub_folder of hidden folder 😢 (error 2 "no such file").
Exactly same process to my iMac (where standard SFTP server runs) works successfully.
Edit: ADrive server seems to have (temporary, I hope) problems. The same error occurs actually even for an upload from a visible folder.
Just when I test my modification...😢and😅 -
Folder rename (drop/add the leading dot) before and after sftp operations might be a workaround.
-
@ccc Yes sir but I still hope it is a temporary problem. Wait and see
-
Problem disappeared today, it was thus a server problem...occurred at a bad moment 😅
-
@spammer Thanks for the info, I didn't know but as I said, I've used the solution adviced by @JonB
-
@cvp, unfortunately that post was a spam (copy of an earlier post in this thread).
Would be nice to maybe have an area where posting would be limited to ”members only”, i.e. having gone through some kind of a challenge that could be only responded to from within Pythonista.
And of course another area where people would not have to be similarly verified and could ask ”should I buy Pythonista for xyz” questions.
-
@mikael Thanks, shame on me