Welcome!
This is the community forum for my apps Pythonista and Editorial.
For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.
StaSh updated for 1.6 beta with ObjC functions
-
@Phuket2 Keep in mind that "hidden" files are much less hidden in shell environments than on a desktop system like Windows or OSX. In the shell there are no hidden files as such, it's just that
ls
by default doesn't show files starting with a dot, which are by convention "hidden". Desktop systems like Windows or OSX have a different concept of "hidden", namely an extra file flag that has nothing to do with Unix dot files. To see that kind of hidden files on a desktop system, you often need to change some hard to find setting. Which is probably a good thing, considering how many users (want to) know nothing about how their OS works internally and would be worried about all those desktop.ini files that they never created. -
ls .*
will show you hidden files and directories on the commandline. -
@dgelessus and @ccc, I really do get your points. But we are in 2015 now. I know in some way it's very easy to have hidden files. I just think the days of hiding things should be over. But I also think every file known to man should have a Json style header which every OS adheres to. I know we need another committee to make that happen. But pure text files etc... Are just an idea in the past based on transmission speeds , storage space etc. is a old idea.
Every single file could have a header (binary or text) with a set of attributes. So whatever the committee come up with , they will be there regardless. Then of course you should be able to extend it for app specific use. I am not really smart, but I could see how this could help revolutionise mobile/desktop computing. -
Think of the period like a leading _ in python, which is a way of signalling something is intended to be private. like in python, if you dare you can still use or modify it, but by default it won't be shown by some dir type commands. But it is very easy to see (i believe in stash we have the alias la defined which lists all files, basically ls -la), and nothing prevents you from modifying anything in the folder, though you are likely to completely break your repo unless you know what you are doing. Like python, these are often poorly documented or likely to change... you can use the files, but dont't depend on them.
i don't quite get what you mean about a json file for every file in the system. what would go in such a header? for instance, for git, a file in /refs/heads/branches is the name of the branch, and the contents is a 20 character sha1 pointing to a commit in the object store, which makes branch creation basicslly free. what would the header say for such a file, and how would that help you muck about in the .git folder? How many bytes are you going to spend documenting the format of a 20 byte file? While HD space is cheap, device space is not....
-
@JonB , maybe I am out of step. But I don't think I am. Unix, and all modern operating systems are living in the past (well, just what they know). No new ideas are coming as far as file wide Standardisation.
You talk about a 20 byte header. The header from what ever crazy ISO committee could be implied by the OS. Then extendend , for example if the header decided is a JSON header. Not saying it should be JSON, just one idea to illustrate a point. Then the OS could imply the header, you could then extended it. In compatibility mode, the OS could strip off the header, and pass the text file, PIL or whatever format im
I know it seems out there, but it's not really. -
@Phuket2 You mean something like SGML DTDs or XML schemas, but for any kind of file? That might be possible, but such a StandardBinarySchema should not go at the beginning of every file. If anything, the file header should contain some kind of unique reference that points to a StandardBinarySchema. That would mean much less duplication, but you'd still have a lot of extra data when you have e. g. many small files.
Then there's also the problem of how such a StandardBinarySchema would look. Should it be a BNF variant for binary data? (That would make certain file structures difficult to describe.) Should it be regex? (Now you have two problems.) Should it be a Turing-complete language? (Now Windows has another file type to "protect" the user from.) How would you handle nested structures (a PDF file containing a PNG image)? Where would the OS get schemas that it doesn't have locally? How would updates or different versions of a schema be managed without conflicts and breaking old applications?
I don't think the current situation is so bad that you'd need this kind of "file content description". Many formats already have some kind of relatively unique header, for example UTF-16 text files start with the byte order mark
"\xFE\xFF"
(which may be reversed if the file is stored in this crazy little-endian byte order), ZIP archives start with"PK\x03\x04"
, MS-DOS/Windows exe files start with"MZ"
. If you want to know the format of a file, use the awesomefile
utility that is available on almost any Unix-like system. If you want to know how a certain file format is structured, ask your favorite search engine. -
@dgelessus , thanks for the complete reply. Yes, depending on the block size a one byte file will still consume the size of the formatted block size as far as I know it.
But I don't think a OS/device wide std file header (whatever format) would be prohibitive, size wise. But let's say a JSON style header. Also variable length, so after the standard stuff the developer could insert his/hers own nodes. Maybe there are some issues such as byte order etc, but this would be handled by the OS.
Look, I understand this would not be easy, but these committees are setup to handle the difficult questions as well as our futures.
Look maybe I am dreaming. But the concept at least sounds easy and clear enough to me.I personally think if you ask any developer, if you could attach a JSON style header to any file you create, eg PNG, txt, TIFF, CSV etc... And not have that header interfere with another program that expects a file type of a certain kind, I think it would be a resounding HELL YES! Yes, also security issues. Would also be easy to rewrite the header. But again, this would be for the big brains in the ISO committee to work out.
I still think there was a big improvement with the release of xml. But it solved only a few problems.
Sorry, I know this is not the place to discuss these issues. But my frustration takes over sometimes :) I have always been the can do type of person. My idea is that if we can put people on the moon, then there is not much we can't achieve here only lonely planet Earth.
-
I am still not sure you have explained what you think should go IN the header?
Keep in mind, as well, that as a developer you would need to be responsible for creating such a header (how is the os supposed to know what the file you created is for?). For instance, maybe you want to dump out a log file from your virtual cell class .... NOT SO FAST ! first you need to spend time documenting the format of the log file. What if you change your mind and realize you need to add a time field.... not so fast, better update that json first. If such a header is somehow mandatory, it means you cannot touch a file unless you modify the header. If it is not somehow mandatory, or if "unstructured data" is an option, I think you will find most developers opt for that one.
If you are not talking about some sort of full schema definition, but just some kind of "comment" left by the defeloper, many file systems already provide such a mechanism. NTFS offers an "Alternative Data Stream", which a developer can append whatever info he/she wants. The file system stores a reference to the alternate stream. Funny thing about it, though, is that Windows does not make it easy to find or access, partially because the intent is to store info intended to be inaccessible (for instance, on a windows pc it knows that you have downloaded a file via the internet... that info is stored in an ADS record for each file). Of course the problem is that this exists only on windows, and so emailing a file, zipping it up, copying it to a FAT32 thumbdrive, etc loses that record.
Also... this discussion started with frustration about hidden files. I guess I am still unclear how a json header would help protect the user from files that the developer has suggested they neednt mess with.
-
@JonB, your right. I didn't describe what should go in the header. That would be left to a committee to work out.
All I am trying to say is while all files need their varing formats internally, I would think to have a enclosing format around the file that could be read in a consistent way by any application on any platform without knowing the internals of the file would be a great thing. Yes, I might be as simple as a web address, authors name, creation location (gps data). Yeah, I can see why someone would say, why would you want gps data in a text file. It's the old saying you build it they will come. Meaning people will invent ways to use this extra information in new and exciting ways.
But again, I am not thinking so much about the data that would be stored (I am confident that a committee could come up with some smart relevant information).
In my simplistic idea, this type of idea could help harmonise/update OS file deficiencies/differences.
@JonB, look I know I am really simplifying it and there are would be Technical difficulties to solve. But it has to getting than it is today. Regardless if it is good or bad. File systems have to evolve. Also I know they have. Everyone's off in there own world doing there own thing. That's my impression anyway.
I am guessing the poor lonely text file's upgrade has only been double byte/Unicode. I remember .rtf files, PICT files etc. This is good, but file specific, I am looking for a global upgrade.
I really wish I could express myself better on this topic in writing. I am sure I would have no problems articulating my views verbally.
So, I think I end my message with, We can wait for operating systems to get with the program and supply a wealth of information to files globally, or a new format describing a heading for the want of a better word that wraps around all files.
Lol, thinking if I should post or not.... Yeah I will. I hate to appear stupid....but it's clear in my head. Maybe I should make a video instead. But debating verbally is not my weakest point :) -
@JonB, not exactly what we were talking about, but same idea would apply.
JPeg lockdown: Restriction options sought by committee
http://www.bbc.co.uk/news/technology-34538705
With a well thought out universal file header, all sorts of benefits could be gained and implemented reasonably easily. But now a the JPEG committee will spend a lot of time trying to solve a problem for only one file format. I think the issue they are trying to take here and other issues should be addressed across the whole spectrum of files.
I know is an old discussion we had, but seen this article today and instantly reminded me of our discussion here