FS#5487 - filenames problem with non-UTF8 locales in GTK2 apps

Attached to Project: Arch Linux
Opened by Roman Kyrylych (Romashka) - Saturday, 30 September 2006, 21:55 GMT
Last edited by Roman Kyrylych (Romashka) - Wednesday, 08 November 2006, 23:26 GMT
Task Type Bug Report
Category System
Status Closed
Assigned To Jan de Groot (JGC)
Architecture not specified
Severity Medium
Priority Normal
Reported Version 0.7.2 Gimmick
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

See original message here: http://bbs.archlinux.org/viewtopic.php?p=194235#194235

The problem: when using non-UTF8 locale (I use uk_UA.KOI8-U) when user saves files from GTK2 applications with non-Latin characters in filename then filename will become a mess of ugly chars, because no charset translation are made by GLib.
The solution: create /etc/profile.d/gtk+.sh with this code inside:
G_BROKEN_FILENAMES=1
export G_BROKEN_FILENAMES
G_FILENAME_ENCODING=@locale
export G_FILENAME_ENCODING
This task depends upon

Closed by  Jan de Groot (JGC)
Friday, 29 December 2006, 10:55 GMT
Reason for closing:  Fixed
Additional comments about closing:  fixed in glib2 2.12.6-3.
About G_BROKEN_FILENAMES and G_FILENAME_ENCODING: G_BROKEN_FILENAMES=1 is the old variable name that always works, the other one is the new way which has to be set per locale. We'll use the old method instead, the same method used by Fedora and Redhat.
Comment by Roman Kyrylych (Romashka) - Monday, 02 October 2006, 13:41 GMT
The code above should be generated to locale.sh from rc.sysinit for all non-UTF-8 locales (the same way as for LANG).
Comment by Roman Kyrylych (Romashka) - Sunday, 15 October 2006, 09:52 GMT
Here is the modified section of rc.sysinit:
# If locale is *.utf set console to Unicode mode
if [ "`echo $LOCALE | /bin/grep -i utf`" ]; then
stat_busy "Setting Consoles to UTF-8"
/usr/bin/kbd_mode -u
/usr/bin/dumpkeys | /bin/loadkeys --unicode
# the $CONSOLE check helps us avoid this when running scripts from cron
echo 'if [ "$CONSOLE" = "" -a "$TERM" = "linux" -a isatty ]; then echo -ne "\e%G"; fi' >>/etc/profile.d/locale.sh
stat_done
else
echo 'if [ "$CONSOLE" = "" -a "$TERM" = "linux" -a isatty ]; then echo -n "G_BROKEN_FILENAMES=1\nexport G_BROKEN_FILENAMES\nG_FILENAME_ENCODING=@locale\nexport G_FILENAME_ENCODING"; fi' >>/etc/profile.d/locale.sh
fi

Yes, I know that this line is very long, you can replace it with heredoc syntax.

I also attached fixed rc.sysinit (cannot make patch because I'm on windoze machine now).
Comment by Roman Kyrylych (Romashka) - Saturday, 02 December 2006, 16:16 GMT
Anyone can confirm this bug?
Comment by Jan de Groot (JGC) - Saturday, 02 December 2006, 16:35 GMT
What happens when you install 2 locales on your system?

en_US.UTF-8
ru_UA.KOI8-U

The first doesn't need it, the 2nd does need it. Both are choosable, no matter where you put the boot language to...
Comment by Roman Kyrylych (Romashka) - Saturday, 02 December 2006, 16:47 GMT
I don't understand this. :-/

When I have LOCALE="uk_UA.KOI8-U" in rc.conf then "broken filenames" feature will be activated on boot.
When I have LOCALE="en_US.UTF-8" these additional variables will not be written to locale.sh.

How do you switch locales at run-time?
And what to do with current way of writing LANG setting to /etc/profile.d/locale.sh then? When you switch locale at run-time it will be reset on relogin.
Comment by Jan de Groot (JGC) - Saturday, 02 December 2006, 17:12 GMT
When I have en_US.UTF-8 in /etc/rc.conf, the hack won't get activated on boot, but when the uk_UA.KOI8-U is in my locale.gen, people can login from GDM using that language, resulting in broken filenames. The "at runtime" change of locales is just an environment setting and can have effects on your whole working environment if you set it before starting your working environment. This is what GDM does for example.
Comment by Roman Kyrylych (Romashka) - Saturday, 02 December 2006, 17:51 GMT
My patch may cause some problems when defaut locale is non-UTF-8 but user logins with GDM and some UTF-8 locale. I don't know how "broken filenames" feature behaves with UTF-8 locales.

This can be fixed by changing my patch to generate this:
if [ "`echo $LANG | /bin/grep UTF-8`" ]; then
G_BROKEN_FILENAMES=1
export G_BROKEN_FILENAMES
G_FILENAME_ENCODING=@locale
export G_FILENAME_ENCODING
fi



About switching locales after boot:
when switching UTF-8 <=> non-UTF-8 users will have problems in console, because line
if [ "$CONSOLE" = "" -a "$TERM" = "linux" -a isatty ]; then echo -ne "\e(K"; fi
or
if [ "$CONSOLE" = "" -a "$TERM" = "linux" -a isatty ]; then echo -ne "\e%G"; fi
is added to /etc/profile.d/locale.sh only on boot.

This can be fixed by making rc.sysinit generate this code:
if [ "`echo $LANG | /bin/grep UTF-8`" ]; then
if [ "$CONSOLE" = "" -a "$TERM" = "linux" -a isatty ]; then echo -ne "\e%G"; fi
else
if [ "$CONSOLE" = "" -a "$TERM" = "linux" -a isatty ]; then echo -ne "\e(K"; fi
fi

Should I file separate bug report for this?
Comment by Roman Kyrylych (Romashka) - Saturday, 02 December 2006, 18:01 GMT
ouch, grep UTF-8 should be replaced with grep -i utf
Comment by Roman Kyrylych (Romashka) - Friday, 08 December 2006, 13:03 GMT
Here is the patch that
1) fixes a problem with filenames in GTK2 with non-UTF-8 locale (not tested yet),
2) attempts to fix a ptoblem with changing locale at runtime which will harm text console, but I have the feeling that this will not work, I have to test it too.

I think about changing the way LANG and LOCALE are set in iniscripts. I need to investigate all issues more.
Comment by Jan de Groot (JGC) - Friday, 08 December 2006, 13:31 GMT
I wonder why that file should be generated on boot, the condition to create it could be included in the file.
Comment by Roman Kyrylych (Romashka) - Friday, 08 December 2006, 13:39 GMT
Yes, I agree. But above in rc.sysinit there are command to generate LANG to the same file.
I think that it will be better to not generate LANG to that file, and include these lines that my patch have directly in /etc/profile.d/locale.sh. Then it will work as needed.
If LANG will be just exported in rc.sysinit, will that work for all user logins? (I cannot test it now)
Comment by Roman Kyrylych (Romashka) - Saturday, 16 December 2006, 20:19 GMT
Ah, LANG cannot be exported from rc.sysinit. It should be set in /etc/profile to be working.
It is possible to set only LANG in /etc/profile and eliminate LOCALE from rc.conf, but this is kinda ugly. :-/
Comment by Jan de Groot (JGC) - Thursday, 28 December 2006, 11:31 GMT
ok, from redhat, csh version in profile.d:

## This caused GLib2 applications to convert filenames from
## locale encoding to UTF-8. If the locale encoding is already
## UTF-8 then it makes no difference.

setenv G_BROKEN_FILENAMES 1

I'll add it to the next release of glib2.

Comment by Roman Kyrylych (Romashka) - Thursday, 28 December 2006, 11:36 GMT
you mean

G_BROKEN_FILENAMES=1
export G_BROKEN_FILENAMES
G_FILENAME_ENCODING=@locale
export G_FILENAME_ENCODING

or

setenv G_BROKEN_FILENAMES 1

?

The first is recommended by Glib manual.
Comment by Roman Kyrylych (Romashka) - Thursday, 28 December 2006, 11:40 GMT
I mean not setenv/exzport stuff of course, but that setting G_FILENAME_ENCODING seems to be needed too.

Loading...