Did anybody check that ".." can't be fooled to do the same thing on HFS+? In particular, how does the character sequence "dot" "zero-width-utf8" and "dot" work? Or "zerowidth" "dot" "zerowidth"? Does it work like ".."? Because if it does, your fix is incomplete, and people can populate things in random places above the git tree.

Finally, did you check that "tolower" works on a ucs_char_t? It's not supposed to, afaik.

Quite frankly, HFS+ is probably the worst filesystem ever. Christ what shit it is. NTFS used to have similar issues with canonicalizing utf8 (ie using non-canonical representations of slashes etc). I think they at least fixed them. The OS X problems seem to be fundamental.

+Linus Torvalds mpm checked '.<u200c>.' (or one of its cousins) and it doesn't do anything useful. '..' appears to be hardcoded in a different layer of the darwin kernel. So that's something.

For a great rant about HFS+ by +John Siracusa​ check out http://5by5.tv/hypercritical/56

+Linus Torvalds Yeah, Peff explored the double-dot and single dot cases while trying to come up with the initial version of the patch, soon after we learned about the glitch.

By the way we belatedly started a closed mailing list on git security in the mid-way while dealing with this one. Do you mind to be in the circle, or are you too busy to bother (or you trust us well enough running around without you)?

+Philip Durbin I didn't listen to all of it, but while +John Siracusa isn't a fan of HFS+, he's not even ranting about the true insanities of that filesystem.

Sure, it's old. Sure, it does a horrible job of actually protecting your data. But those are more "it's not a great filesystem" issues. They aren't "that's incredible crap designed by morons that have a hard time figuring out how to feed themselves".

The true horrors of HFS+ are not in how it's not a great filesystem, but in how it's actively designed to be a bad filesystem by people who thought they had good ideas. 

The case insensitivity is just a horribly bad idea, and Applie could have pushed fixing it. They didn't. Instead, they doubled down on a bad idea, and actively extended it - very very badly - to unicode. And it's not even UTF-8, it's UCS2 I think.

Ok, so NTFS did some of the same. But apple really took it to the next level with HFS+.

There's some excuse for case insensitivity in a legacy model ("We didn't know better"). But people who think unicode equivalency comparisons are a good idea in a filesystem shouldn't  be allowed to play in that space. Give them some paste, and let them sit in a corner eating it. They'll be happy, and they won't be messing up your system.

And then picking NFD normalization - and making it visible, and actively converting correct unicode into that absolutely horrible format, that's just inexcusable. Even the people who think normalization is a good thing admit that NFD is a bad format, and certainly not for data exchange.  It's not even "paste-eater" quality thinking. It's actually actively corrupting user data. By design. Christ.

And Apple let these monkeys work on their filesystem? Seriously?

There are lots of good reasons to not move to ZFS (cough-Oracle-cough), but they could have pushed people to case-sensitive HFS+, which would have then made it much easier to (in the long run) migrate to anything else saner. But no. There is a case sensitive option, but Apple actively hides it and doesn't support it.

The stupidity, it burns.

So you had all these people who made really bad decisions and actively coded for them. And I find that kind of "we actively implement shit" much more distasteful than just the "ok, we don't implement a lot of clever things" that John complained about.

Rant over.

+Junio C Hamano I'm ok with being added to a git security list. That said, I suspect it's probably saner to just know that you can contact me directly if there's something that is actively relevant to any of my old design or code, and you have a commit where I did something stupid and want to rant at me.

+Linus Torvalds My complaints about HFS+ extend a bit beyond the lack of clever (i.e., modern) features. I wrote about it at Ars Technica a few years ago:

http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/#hfs-problems

As for case sensitivity, it is supported in OS X (though using it has tended to expose bugs in many apps, including Apple's own). I believe iOS has always used case-sensitive HFS+ exclusively, so there's one small concession to modernity. I'm not privy to the details of modern file system projects within Apple, but I'm sure there are many sad stories to tell about to the ongoing reign of HFS+…

+John Siracusa And I checked and they still convert to NFD under case sensitive HFS+. And yes, older Carbon apps tends to be a pain in particular. I wonder why the "HFS+ Private Data" hack was chosen when Rhapsody existed even back in 1997. Unfortunately, I think Classic Mac OS depended on case-insensitive comparisons on non-ASCII chars.

You can pick a conservative character set for RedSea.  I don't care.  For interoperability, least common denominator makes life pleasant.  I lost enthusiasm for file attributes and metadata once I had to interoperate with FAT32.  Even though I want to remove FAT32 and ISO9660, I think headaches can be avoided not putting metadata on files.  I have a couple new attributes.  Let's go with minimal common denominator characters --- screw Europe, LOL.

I don't think I screwed-up ".."  I could have, LOL.  Linux doesn't have to search parent directories for file-not-found, but I do.  Yeah, my root directory files are would be read but not written.  When you say "/Home/Notes.TXT" if it is not found, it reads Notes.TXT from root, but everything looks just as though it came from /Home/Notes, so it is not a write vbulnerabiliy.

Linus, you can get your name in the Bible and be famous.

BTW, one other stupidity is that Linux's HFS+ implementation refuses to mount journaled volumes when Apple designed it to be backward compatible.

+John Siracusa +Linus Torvalds This reminds me of the lack of linux support for exfat. I would rather argue that people that have the ability to make something more compatible but choose not to are far worse than making something broken. Speaking of the intricacies of a file system that has honestly held its own for the last ten years is stupid anyway. This is like complaining about slackware.

+Derek Dickerson exfat support for linux exists it a FUSE implementation. Exfat cannot be mainlined due to patent issues.

I want to remove my Exfat and ISO9660 when I command Microsoft and Linux on authority from God to put RedSea in theirs.

RedSea File system documentation.  Think of a 1040EZ tax form. http://www.templeos.org/Wb/Doc/RedSea.html

+Derek Dickerson So like every other non-Windows operating system ever, then.

+Aidan Taniane OSX supports exfat

+Derek Dickerson Well 2 out of hundreds aint too bad, I suppose.

I'm trying to reduce down to just one filesystem.  I have FAT32, ISO9660 and RedSea.  I want just RedSea.  I'm trying to make God's temple perfect and unblemished and limited to 100,000 lines of code.  I have a vision that is really beautiful.  Linux and Windows and VMware must support RedSea.  I'm like Moses and because I said so. Watch my demands https://www.youtube.com/watch?v=EalSFnq9GsY

+Derek Dickerson Apparently you don't know there's a native Linux kernel driver for it. https://github.com/dorimanx/exfat-nofuse 

I always suspected Apple's HFS sucks, but after reading Linus' comments here I stand in awe of how bad things actually are. Good luck trying to explain this to the fanboys, though.

Can someone tell me why HFS+ sucks? I use OS X on a daily basis and never had issues with anything hard drive wise.

Listen to the podcast, +Ryan Hamel 

+Ryan Hamel Did you read the comments here? Especially those by +Linus Torvalds 

Yeah, I was not in favor of case insensative RedSea.  When I said I wouldn't mind limiting to FAT32 characterset, then what do you do for other filesystems.  God wants it perfect, that means we use all 256 less the obvious ones.

+Lucas Dohring that's what I was going to add. Dunno what +Derek Dickerson​ is on about. 

All I can think of: http://xkcd.com/927/

Anyone who doesn't like HFS should move to Xsan.

Excuse me, but I haven't really seen any comment here that would describe WHAT'S wrong with HFS+. All I've read is "It's bad, it's case-insensitive, not modern, made by morons, it's bad". Any real-life examples that would make me understand why HFS is bad? Like, something I can run into in real life that would ruin my day?

+Илья Панфёров, you could have read above that they (and Linus in particular) don't want to reveal the problem in order to avoid its using. If they say so, they know what they're talking about. Read https://labs.spotify.com/2013/06/18/creative-usernames/ (or http://habrahabr.ru/post/183878/ in Russian), for instance, to understand the problem. The one with HFS+ and the one described in the article are very alike.

+Anton Yablokov I'm not talking about one problem in particular, but rather problemS that make HFS+ "the worst" like they say. One issue does not make a thing inheritably worst. 

+Илья Панфёров Read more, understand more:
To quote +Linus Torvalds  "The case insensitivity is just a horribly bad idea, and Apple could have pushed fixing it. They didn't. Instead, they doubled down on a bad idea, and actively extended it - very very badly - to unicode." ... "There's some excuse for case insensitivity in a legacy model ("We didn't know better"). But people who think unicode equivalency comparisons are a good idea in a filesystem shouldn't  be allowed to play in that space."

I've had issues with directories disappearing because of the case insensitivity. I copied an eclipse project over onto a Mac and I had named it with the first letter lowercase(I believe) and then wanted to change it so I renamed it by simply changing the first letter to be uppercase and also changed some stuff in the source code on the other box and recopied it over onto the Mac. If my memory serves me correctly they both were there but I could only get at the first one because HFS+ is case preserving but not case sensitive. It took a bit to realize what was going on and why my changes to the code didn't make it over onto the Mac. Prior to that I had no experience with Mac OS so I wasn't aware of this "wonderful" feature.

With a limited effort, couldn't the Linux community bring patent-free UDF to the level where it could replace exFAT? Most operating systems already support UDF to various degrees.

Probably because of some original AZERTY screw-up, many French people (wrongly) think uppercase characters should not have accents. Microsoft Word has a special toggle so let each user pick his favorite side. Now I wonder which side HFS picked...?

Next great idea for Apple and others: the language independent filesystem. It translates filenames on the fly!

WOW! That's like the biggest consumer issue ever. Should I use Linux?

So can someone show an example of when it is a good idea to have two or more files spelled exactly the same but with different character capitalizations?

So far we've seen that this can maybe break on Eclipse. Shocking.

+Linus Torvalds's argument that HFS+ lacks case sensitivity by default is a bad design choice holds water in the weird Unicode normalization cases, but otherwise fails to appreciate the fact that at some point the file system must represent something useful to users. In other words, without considering UX, this conversation has been about bashing HFS+ for not defaulting to a "feature" it already has. But with UX in the picture, I cannot imagine seeing case sensitivity help anyone ever with already overloaded hierarchical metaphor of file systems use computers more effectively. This begs the question then that if you need case sensitive files/directories, you are obviously not human, so why do I care if it is in my filesystem? I know that is the POSIX way, but perhaps there are better ways to represent the data.

Cheers

+Joe Gorse Is there any way to +100500 you?

The problem with case-insensitive filesystems can be summarized in very few words: a great idea - until it breaks in a myriad of different ways. It's very similar to T9 or voice recognition. They work... most of the time. Because it's so costly or time-consuming when they fail, many people avoid them.

The reason people want case-sensitive filesystems has absolutely nothing to do with the desire to have filenames which differ only by case, no one ever does that in practice. It's just because Reliability Matters.

+Дмитрий Лебедев If only. =)

+Marc Herbert Interesting argument, though it fails to pass the test which would make case sensitivity worthy of existing as a feature.

In terms of reliability, one could say that the problem has been introduced by those who seem to think filesystems should support arbitrarily represented data in filenames. shrug Personally, I'll use the filesystems which were written for use by both human beings and computers rather than strictly computers alone.

But as far as quantifying comparative reliability of filesystems, that seems like a punting contest if I ever saw it. In general though, I'd say reliability has gone down on all fronts in computing. No use in pinning the blame on HFS+ for that. There are much bigger fish to fry really...

Cheers

+Joe Gorse OK, now I understand the crucial technical fact you happen to miss: whereas case-insensitivity is indeed a feature (user-friendly for some but too complex and thus unreliable), case-sensitivity is not a feature at all, it's the complete lack of any filename feature. Not a single line of code has ever been "introduced" to implement case-sensitivity. In fact there's even a good argument that POSIX filenames are too extreme in "their lack of features": http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html

+Linus Torvalds
the one big reason i still use linux, android, and never use apple products,

+Marc Herbert I see your point. Though the simple POSIX implementation may be less amenable to the new "Unicode" world we increasingly find ourselves these days.

It still mystifies me as to why people get so heated up about things like this (see above thread). Tl;dr: someone tried to address a limitation in the computing metaphor and created a bug. Flame on

I suppose Linus et al. may be so focused on the micro implementations that the bigger picture gets lost.

The larger challenges in computing at stake here are fundamental. At the core of the issue here is data. POSIX was and is a very powerful means of thinking about data, albeit nuanced in implementation for both the computer and the user. At the center of the POSIX doctrine, everything is represented as a file (even "streams of bytes") within a descriptive hierarchy. Thus relatively simple implementations could be achieved for arbitrary data flow from data sources and sinks. Interfaces and drivers needed to implement the system calls for reading and writing data. Thus, POSIX is a conceptually straightforward and powerful metaphor for computing in general.

And so, what can we do within the spirit of POSIX in order to strengthen the UX (human side) of the equation? That is the goal.

Cheers

+Linus Torvalds You are my idol !
Open Source - a great idea !!!

+Linus Torvalds Of course!!! this is scared!