Arch Linux

Looking at this I will export LANGUAGE and LC_ALL, but no more reading /etc/profile, which is get from systemd environment. systemd will read the /etc/locale.conf file and manage kernel command line override properly.

LC_ALL is explicitly disallowed in locale.conf(5), so I don't think exporting it is a good idea.

I read: «Note that LC_ALL may not be configured in this file. For details about the meaning and semantics of these settings, refer to locale(7).»

I understand that there is no guarantee (may not) but not it's forbidden (must not).

The previous sentence says "The following locale settings may be set using /etc/locale.conf". I think this paragraph lists what is allowed and disallowed in the file.

LC_ALL is commonly a last resort override variable. localectl explicitly doesn't list or support LC_ALL.

```
$ export notavar
$ declare -p notavar reallynotavar
declare -x notavar
bash: declare: reallynotavar: not found
```

I did not realize export had that effect on unset variables, but I think it should be harmless as bash still seems to consider it unset.

Anyway, regarding LC_ALL, it should be fine to export it. The reason it should not be set in locale.conf(5) is not because it shouldn't be used, merely that it should only be used in explicit circumstances. Exporting it *in those circumstances that it is already set* seems to be the right choice I guess.

With filesystem 2017.12-2 in tty I get C as LANG and no LANGUAGE. Both are set in /etc/locale.conf .
If I copy /etc/locale.conf to ~/.config/locale.conf and relogin both are set correctly.

# /etc/locale.conf loads and overrides by kernel command line is done by systemd

echo "$LANG" in locale.sh spits out "C" before the LANG=${LANG:-C} line is executed, so the assumption that systemd does this may not be quite correct.

I spoke too soon. On a new TTY login, echo "$LANG" spits out ''. I suspect the root cause of this is that systemd explicitly unsets the locale variables when starting getty@.service, so there is no locale setting in the login session.

You are right.

From getty@.service :
# Unset locale for the console getty since the console has problems
# displaying some internationalized messages.
UnsetEnvironment=LANG LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION

https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/filesystem&id=42bcd7b15b45f32569aab2c88cffd4c5d7818be1 as a "fix" for this task has broken locales for anyone logging in on the tty via getty due to it unsetting all of the envions.

A workaround appears to be copying /etc/locale.conf to XDG_CONFIG_HOME/locale.conf

Nevermind, the workaround involving copying the file to XDG_CONFIG_HOME doesn't work either.

So things remain broken.

I think we should be coherent with systemd-localed regarding LC_ALL; and as far I tested systemd ignore LC_ALL in locale.conf (and on kernel line overrides).
So, I will stop exporting it in next release.

About locales not set for getty services, this is something done by systemd upstream on purpose and we, until now, override it by mistake.
If there is no more issue and UnsetEnvironment in the getty@.service can be removed, I suggest you to contact upstream to make this fixed in all distribution.
If this can be overridden only in your case, you can remove the UnsetEnvironment= by a partial overrides in /etc/systemd/system/getty@.service.d/

According to https://github.com/systemd/systemd/commit/1640944a847249d3f5f0fb0d5a5f820a82efaed0 the problem is with getty itself using the locale variables not the shells in it and these variables should be set in a post-login script.

Breaking a previously working post-login locale for all getty users because you disagree with a systemd *policy* (not a bug), by claiming "we overrode it by mistake" (we didn't, systemd expects us to set it ourselves) seems rather broken.

If upstream decides that there is nothing to fix, and reaffirms their belief that it should be set in a post-login shell... are we just going to say "no, systemd should do it", and leave *locale* broken as a matter of Arch policy?

This change pushed into [core] will generate a lot of strain on all support venues since it breaks the locales for every getty user, which on arch will be a large number (according to my own, made up, statistics).

Upstream leaves this up to the distributors to solve and I highly doubt they will change their stance.

a)remove the UnsetEnvironment line in getty@.service in the systemd packages and let users deal with the funny question-marks when using the linux console (minor issue since they will get them anyway, just for different characters)

b)work the locale.conf changes back into locale.sh in a fashion that lets users still override them from the kernel commandline (is this ever useful on anything that is not a "live CD"?)

Unhelpful contribution, but I've long since stopped waiting for this issue to be fixed properly and now NoExtract=etc/profile.d/locale.sh while using my own approach.

Certainly systemd is wrong to unset the environments because it's frankly none of their business as to whether or not the console "has issues" or not, but through misguided "good intentions" they do so anyway to shield users from any potential issues.

Likewise it is wrong for Arch to break functionality like this and do nothing about it. The least they could have done is either revert the changes and then start thinking of a new mechanism, or, change the mechanism to test an alternative (i.e. this is what [testing] is for).

PS: *Never* export LC_ALL (why was this even considered?). Users can do that themselves. They may load the gun and shoot their own feet, but Arch (or any distro) shouldn't be complicit in doing the latter on their behalf.

Not sure why only getty is discussed; I use slim as my login manager and suddenly have LANG=C everywhere. :)

It's been 4+ days with locales totally broken here, seblu, it's time to stop this insanity.

@nl6720: Thanks for pointing this.

@Eli: The systemd policy was well summed by Lennart in the bug report pointed by nl6720: «... the system locale should be read and set at one place, and not at multiple, and that one place should be PID 1.» This is precisely what I attempted to achieve. I presumed (wrongly) that our locale.sh was loading /etc/locale.conf because it was from a pre systemd era and has never be cleaned since, that's why I improperly used the «by mistake» stanza.

Despite this ambitious target, they decided to workaround a getty issue with CIJK languages by relying on distro shell scripts: «Hence, for now don't pass any locale settings to getty/login, and thus rely on the classic post-login script fragment to set and fix the locale.». I didn't know that and that change the story.

@Alexander: Your b) proposal make sense but I only reverted the loading of /etc/locale.conf in 2018.1. I have no time and interest in developing something to parse the kernel command line. Not to mention, that systemd environment (and so locales) can be overrode by others mechanism.

@Earnestly: Unhelpful answer: Go teach your opinions to Intel engineers, they deserve your help in that moment and your story about guns.

@Evangelos: Do you know what's happen? Is slim which reset the whole environment or this is specific to locales?

@Doug: Why did you wait 4 days to claim that your locales was totally broken? Only the regression on getty was reported until today.
What do you mean by totally broken, is there something else than getty (and slim) broken by letting PID1 define the locales for the system?
A request for a quick fix would do the job as well that «stop this insanity», so there is a 2018.1 in testing with a quick fix. Hope you enjoy.

Display managers / login methods are very diverse in environment setup for spawned sessions: lightdm sets/preserves a few specific variables (preserves locale), slim manually rebuilds an environment from scratch (effectively unsets locale), systemd's configuration for getty appears to preserve everything (but specifically unsets locale), gdm was too much for to read over right now, and I'm not sure what happens over ssh but IMO it's just as important as any other login method. In summary, there is inconsistent behavior with regard to environment, including locale, in user sessions.

Unless all login methods can be relied upon to preserve the system locale for user sessions, it is likely that locale.sh will need to read the system locale for the foreseeable future, as inelegant as it may be.

Let's see, you have multiple reports of the changes not working as intended, you are exporting LC_ALL (a serious WTF), and you let this go for 4 days. Yes, I will call that insanity even if I hadn't chimed in myself. Does one more voice saying "me too" make a difference? You thought it was only an issue with getty, so it only affects everyone who logs into a terminal. That could easily be a large chunk or even a majority of Arch users, so that's not a valid argument.

Like Earnestly, I wasn't bothering waiting for a fix for my own machines, was just frustrated that people were continuing to experience this with no response in sight.

@Jeb: Thanks for looking into what login managers might do; your explanation for SLiM sounds spot on.

@Sébastien: The changes in filesystem 2018.1-1 seem to be the most appropriate course of action; cheers for the quick fix. Please note that LC_NAME is exported twice and LC_PAPER got lost after the changes. I'd suggest generating the list programmatically to avoid such issues; something like running the following command and then copy-pasting the output to locale.sh: https://paste.xinu.at/wvZ/

I'm happy to see it fixed, but it proves what I was complaining about last time. The package is being largely untested and changes pushed without any input from other team members, just like Andreas Radke is still waiting for lp group to be provided by filesystem after it was accidentally removed. Don't be surprised people aren't as patient as they were at the beginning.

In defense of @seblu, exporting LC_ALL is mostly a noop because if XDG_nonsense_vars are not set, LC_ALL is not set either. Then, LANG is set to C and locale(1) outputs "C" with empty LC_ALL. It is also not true that this change created lots of breakage. I only noticed it because my mutt thread trees started showing up as "aaa" when accessed over ssh, but I blamed it on xterm (which was update together with filesystem).

I have a few questions though:
(1) In locale.sh we use [[ -n "$VAR" ]], is it important that the variable is set but empty, and does it make a difference to test for unset $VAR with [[ -v VAR ]]? I don't know all possible issues (security-related as well) associated with locales because myself I mostly use C/POSIX locale.
(2) Sourcing _anything_ from files in $HOME in a system-wide script seems wrong. Do we have to do it?
(3) AFAIU, XDG_variables are for GUIs, no? Why these vars are being tested at all? What sets XDG_CONFIG_HOME other than a display manager. For example, on my machine, $XDG_CONFIG_HOME is not set and I'm using xfce4 started via startx(1).

(0) The only complaint against LC_ALL I heard was "no-ops are silly".

(0.5) Claiming that it is "not true that this change created lots of breakage" because you mostly use C/POSIX locale is far more laughable than the previous accusations about the change itself. Please do not come to the bugtracker merely to talk about how your personal lack of breakage means there isn't a problem.
It is one thing for us to debate what the proper fix is and (maybe) disagree about whose fault it is... it is another thing entirely for you to stick your head in the sand and claim that there is no problem for all other Arch users.

(1) [[ -v VAR ]] is a bashism, and locale.sh is and must be POSIX-compatible.

(2) Yes, it is not a "system-wide script" it is a "system-wide rc file loaded by a user-level interactive shell". It makes an incredible lot of sense to do so.

(3) You have completely misunderstood it, then. XDG standards have nothing to do with GUIs at all. XDG is used for allowing users to relocate the default ~/.config location for a wide variety of user configuration files, across GUI, CLI, TUI, and daemonized software. It is never set by display managers, it is in fact set by ~/.pam_environment or ~/.bashrc or whatever, and if not explicitly set by the user it is defined that applications should fall back on ~/.config.

Thanks :)

(0) Pls search this thread for comments by Doug. I thought that LC_ALL is something evil...

(0.5) I'm just saying that the amount of comments is not commensurate with the seriousness of this issue. Why would you mask locale.sh in pacman.conf if all you need to do is "cp /etc/locale.conf $HOME/.config"??

(1) OK, yes, you are right. Since Arch uses bash as a system shell in all scripts like mkinitcpio and pacman-key, I thought it is appropriate in locale.sh. Still, you didn't answer my main question :) I'll do some research.

(2) If a file being sourced in ~/.config is broken, this will become obvious at every login. Otherwise, this breakage must be enabled from .bash{rc,_profile}

(3) My question was which applications :) And if it is set by user shell init files, how can this possibly work??? bash(1) manpage says that /etc/profile is loaded *before* stuff in ~ (by default of course).

> Comment by Leonid Isaev (lisaev) - Friday, 05 January 2018, 17:56 GMT
> (2) Sourcing _anything_ from files in $HOME in a system-wide script seems wrong. Do we have to do it?

For what it's worth, this is the locale.sh (locale2.sh) script I used to replace the one from filesystem: https://ptpb.pw/cy_i/sh

(0.5) Oh my, you're right. I guess we should just leave bugs broken, because the user can always fix the system by modifying their $HOME. /s

(1) LC_*="" will do strange and wonderful things to prevent inheriting $LANG, but that is really the user's fault. There is no nice POSIX way to test for a set, empty variable (well, I guess we could use ${var+x} but I did specify "nice" :D ), so we might as well fix broken variables I guess? But we no longer bother with that, which is fair because if the user sets LC_*="" then they should probably expect breakage.
Breaking dash or busybox ash for no good reason with syntax errors is a no-no, though. And [[ -v VAR ]] doesn't have any real benefit over exporting it just in case.

(2) If you enable it from ~/.config/locale.conf, then you have enabled it.

(3) It may be unlikely anyone will use `bash -i` in an existing interactive shell, but pam runs before bash and will load this. You could do something like:
```
# XDG compliance, explicitly set fallback values Just In Case™
XDG_CACHE_HOME DEFAULT=@{HOME}/.cache
XDG_CONFIG_HOME DEFAULT=@{HOME}/.config
XDG_DATA_HOME DEFAULT=@{HOME}/.local/share
```

Earnstly, why do you have to do that? What if an entry in ~/.config/locale.conf is like LANG<something evil> (no IFS separator)? <something evil> doesn't need to be malicious, it can be erroneously generated by some software. Again, there must be a hierarchy: system-wide presets -> user-defined presets. I don't want a file in my .config to be magically sourced just because it is there -- it makes tracking issues more complex than it needs to be.

Eli:
(0.5) Yes, my point exactly :) When systemd breaks things like that (see IP sandbox for logind) ppl are happy to change their systems in non-trivial and obscure ways, but when filesystem package introduces a trivial breakage, everyone complains as hell?

(1) Ok, I understand :)

(2) I *brutally* disagree. .bashrc and .bash_profile are documented files, but where is ~/.config/locale.conf documented? Hell, you don't even suggest to parse it, just blindly source? Fantastic!

(3) I can do arbitrarily stupid things, but why do I want to set XDG var in a special .pam_env file? Just to conform to FDO?

*We* are filesystem's upstream.

~/.config/locale.conf is documented where all Arch specific documentation belongs: https://wiki.archlinux.org/index.php/Locale#Overriding_system_locale_per_user_session

As for XDG vars, the reason (when you are not blatantly showing an instructional example) is that one would want to *not* conform to anything, and set their own. If the variable doesn't exist, locale.sh conforms to XDG standards and falls back on ~/.config, so I'm really not sure what the issue is.

I'm not complaining about XDG vars because I don't care about them. I am complaining about blind sourcing of files in user $HOME in a way that *is not controlled by user*. How can I stop this as a user, other than asking my admin to complain here? I mean, we have /etc/skel, why not put this in /etc/skel/.bashrc?

And BTW, no other file in the /etc/profile hierarchy cares about files in $HOME or $XDG*.

@Jeb: yes, I agree.

@Evangelos: Damn, I made this mistake when I tried to optimize the output on 2 lines. I used your script to push a 2018.1-2. Thanks.

@Doug: This change was pushed 31/12 in the evening and roll-backed 4th in the night as soon I get there is a real impact. Before, it was unclear to me if I have the time to find a proper fix to keep the direction or if we have to rollback in emergency. So, yes get feedback from known people like you help. Putting aside that we are in testing, in the middle of the new year eve, I understood your frustration and I'm sorry for that.

I have few quesions:
- What are the technical grounds of your opinion on LC_ALL was a «serious WTF»?
- Your machines are all in testing?
- I used to think that our users are mosting login via a graphical login manger or ssh. Except iso image, which was not hit by that, what usage can be to use getty?

@Bartłomiej: I can share with you about filesystem that I was too optimistic on the change, chose a bad window and didn't took the required time to test. So, there is room for improvements. It's good that you choose cups as an example. I didn't got answer to my question from the maintainer on a-d-p last September. When a bug was open ( FS#56818 ) in late December, I asked and waited for opinions about solutions from several maintainers, to not make people unhappy and don't get complain from you. But, you are still and expecting that the fix come from filesystem where it could have been done by the maintainer month ago, which know cups best than me.

"LC_ALL is an environment variable that overrides all of these. It is typically used in scripts that run particular programs. For example, configure scripts generated by GNU autoconf use LC_ALL to make sure that the configuration tests don’t operate in locale dependent ways.

Some systems, unfortunately, set LC_ALL in /etc/profile or in similar initialization files. As a user, you therefore have to unset this variable if you want to set LANG and optionally some of the other LC_xxx variables."
https://www.gnu.org/software/gettext/manual/html_node/Locale-Environment-Variables.html

LC_ALL should not be exported by default, it is there for debugging and very few other use-cases.

I assume that the WTF was caused by you shredding locale.sh despite the unfamiliarity with LC_ALL (or how the LC_* vars work in general).

> - I used to think that our users are mosting login via a graphical login manger or ssh. Except iso image, which was not hit by that, what usage can be to use getty?

https://wiki.archlinux.org/index.php/Getty#Automatic_login_to_virtual_console

@seblu: I login via tty and run "exec startx"... I am yet to see an advantage of a DM compared to this scheme. And yes, my machines (about 10) all run [testing]