| http://www.linuxfromscratch.org/blfs/view/svn/introduction/locale-issues.html |
| |
| "The POSIX standard mandates that the filename encoding is the encoding implied by the current LC_CTYPE locale category." |
| |
| ------- |
| |
| http://mail.nl.linux.org/linux-utf8/2001-02/msg00103.html |
| |
| From: Markus Kuhn |
| |
| Tom Tromey wrote on 2001-02-05 00:36 UTC: |
| > Kai> IMAO, a *real* filesystem should use some encoding of ISO 10646 - |
| > Kai> UTF-8, UTF-16, or UTF-32 are all viable options. The same should |
| > Kai> be true for the kernel filename interfaces. |
| > |
| > I like this, but what should I do right now? |
| |
| The POSIX kernel file system interface is engraved into stone and |
| extremely unlikely to change. File names are arbitrary binary strings, |
| with only the '/' and '\0' bytes having any special semantics. You can |
| use arbitrary coded character sets on it as long as they do not |
| introduce '/' and '\0' bytes spuriously. Writers and readers have to |
| somehow agree on what encoding to use and the only really practical way |
| is to use the same encoding on all systems that share files. Eventually, |
| everyone will be using UTF-8 for file names on POSIX systems. Right now, |
| I would recommend users to use only ASCII for filenames, as this is |
| already UTF-8 and therefore simplifies migration. Using the ISO 8859, |
| JIS, etc. filenames should soon be considered deprecated practice. |
| |
| > I work on libgcj, the runtime component of gcj, the Java front end to |
| > GCC. In libgcj of course we use UCS-2 everywhere, since that is what |
| > Java does. Currently, for Unixy systems, we assume that all file |
| > names are UTF-8. |
| |
| The best solution is to assume that the file names are in the |
| locale-specific multi-byte encoding. Simply use mbrtowc and wcrtomb to |
| convert between Unicode and the locale-dependent multi-byte encoding |
| used in file names and text files if the ISO C 99 symbol |
| __STDC_ISO_10646__ is defined (which guarantees that wchar_t = UCS). On |
| Linux, this has been the case since glibc 2.2. |
| |
| > (Actually, we do something notably worse, which is |
| > assume that file names are Java-style UTF-8, with the weird encoding |
| > for \u0000.) |
| |
| \u0000 = NUL was never a character allowed in filenames under POSIX. |
| Raise an exception if someone tries to use it in a filename. Problem |
| solved. |
| |
| I never understood, why Java found it necessary to introduce two |
| distinct ASCII NUL characters. |
| |
| ------ |
| |
| Interesting idea. Use iconv to create shift-jis or other mbcs test cases. |
