FS#68757 - [ruby] set GEM_HOME on the environment to avoid inconsistencies
Attached to Project:
Arch Linux
Opened by Felipe Contreras (felipec) - Friday, 27 November 2020, 02:38 GMT
Last edited by Buggy McBugFace (bugbot) - Saturday, 25 November 2023, 20:24 GMT
Opened by Felipe Contreras (felipec) - Friday, 27 November 2020, 02:38 GMT
Last edited by Buggy McBugFace (bugbot) - Saturday, 25 November 2023, 20:24 GMT
|
Details
The instructions in the wiki to allow user installations of
gems mention the --user-install option, but this is ignored
by the bundler program.
First, the instructions say to do this to disable building the documentation: ~/.gemrc: gem: --no-document But this would override the system's /etc/gemrc, which has --user-install, so the user actually has to do: ~/.gemrc: gem: --user-install --no-document Then the instructions proceed to explain --user-install, and that the gems end up in `~/.gems/ruby`. Then the instructions for bundler are mentioned, and the following command is suggested: bundle config path ~/.gem However, that's not the same path gem uses (`~/.gem/ruby`). The actual path can be found running the command `ruby -e 'puts Gem.user_dir'` (e.g. `~/.gem/ruby/2.7.0`). So the instructions are wrong. I and others have contacted the ruby/bundler developers multiple times over the years, and they have refused to fix the issues: * https://github.com/rubygems/rubygems/issues/4027 * https://github.com/rubygems/bundler/issues/710 * https://github.com/rubygems/bundler/issues/2565 * https://github.com/rubygems/bundler/issues/2667 There is a very simple solution to get around all these problems, just distribute a script for profile.d: /etc/profile.d/ruby.sh export GEM_HOME="$(ruby -e 'puts Gem.user_dir')" This way no --user-install is needed: `/etc/gemrc` can be dropped, and all the instructions for `~/.gemrc` and the bundle path config (which is wrong), since bundle uses $GEM_HOME by default. Additionally, we can modify the PATH so the user doesn't have to: export PATH="$PATH:$GEM_HOME/bin" So the instructions about the setup can be removed. So this fixes everything: /etc/profile.d/ruby.sh export GEM_HOME="$(ruby -e 'puts Gem.user_dir')" export PATH="$PATH:$GEM_HOME/bin" |
This task depends upon
Closed by Buggy McBugFace (bugbot)
Saturday, 25 November 2023, 20:24 GMT
Reason for closing: Moved
Additional comments about closing: https://gitlab.archlinux.org/archlinux/p ackaging/packages/ruby/issues/1
Saturday, 25 November 2023, 20:24 GMT
Reason for closing: Moved
Additional comments about closing: https://gitlab.archlinux.org/archlinux/p ackaging/packages/ruby/issues/1
FS#68868In linked issue:
> I will take this ticket as evidence and suggest a hacky solution downstream to my distribution maintainers, since you leave us no other choice.
Here:
> they have refused to fix the issues:
Upstream is actually just annoyed that you spend more time arguing over whether gem should have implemented something bundler later doesn't respect. So they locked your issue. But that does NOT constitute refusing to fix the issue, since they opened https://github.com/rubygems/rubygems/issues/4031 instead to discuss it with fewer personal attacks.
Since it seems logical that upstream is in fact going to fix the issue, and you know it because you even participated in the new issue, I propose you document your workaround on the wiki while we wait for a builtin method that upstream is actively working on. There's really no reason your user configuration for bundler *needs* to be directly packaged in the rubygems package.
Please stop speaking on behalf of Arch when you say we "will" do whatever you are threatening them with.
And it would not be fixed. If instead of censoring discussion you allowed for at least one reply before closing I could explain why.
But do whatever you want. It's only the Arch Linux users that suffer.
https://wiki.archlinux.org/index.php/Talk:Ruby
I would not hold my breath on it getting fixed.
We shouldn't ask **all** our users to manually workaround their issues.
Imagine if every program thought this was a good idea...
The command has been carefully studied by multiple parties (including me) and is obviously the best solution (giving the limitations of RubyGems).
If you have something against Ruby code, I can replicate the exact same functionality using shell.
I do think it's a bit weak to have the instinctual reaction "this program does something I don't like! Let's solve it by forking to external disk utilities during shell initialization".
It's not quite so bad as programs that do `eval "$(fooprog init)" in bashrc, because profile scripts are only run during shell login, not every interactive shell init. But it's still not great, and I think people should think twice before doing it once, rather than getting in the habit of doing it many times.
If it's a known static location then yes, implementing this in pure sh would be preferable as that runs in-process and much faster, at a point in time (e.g. tty login) where you want to optimize for startup responsiveness.
There is also the fact that it's kind of meh to add OS wide environment variables to every process running ever, without a really excellent reason; the Environmental Protection Agency isn't happy with your suggestion. But that isn't quite as big a deal.
The larger issue is that as far as I can tell the only thing this solves is bundle, not gem, and only in the sense that it stops bundle from running sudo stuff for you which would normally get stopped by a password prompt and could be patched out anyway but would be best discussed with upstream as they were open to fixing this!
...
Aside for that, I have implementation comments: there is an append_path function that is supposed to be used for profile scripts to avoid duplicate entries, and given this directory often will not exist it would be nice to add it to PATH only if it does.
And people have thought way more than twice about solutions, for **TEN** years.
I have worked on this issue extensively, and I've sent patches to the rubygems projects to no avail, if you have a better solution let me know. And if you do, why haven't you updated the Ruby wiki page explaining that solution?
We do have a good reason to add the environment variable, as I have already explained in the long details above.
The problem you describe with bundle is not the actual problem. Read again my details above.
And it is not a static location, it depends, I'm attaching a shell script that does basically the same.
Comparing how much each solution takes:
ruby -e 'puts Gem.user_dir' 0.05s user 0.01s system 98% cpu 0.060 total
sh gem_home 0.00s user 0.00s system 94% cpu 0.009 total
You provided four links, 3 of which complain that 'bundle install' runs sudo, and one which complains that bundle doesn't respect --user-install and therefore targets /usr/ instead of $HOME/
The fact that bundle uses /usr/ is of course only a problem because it succeeds, and it only succeeds because it uses sudo, right?
> I've sent patches to the rubygems projects to no avail
From 2013 at least you already created your reputation in the bundler developer community, which definitively involved you claiming you were morally unwilling to submit patches:
Project member:
> If you'd like to add a --user-install option to Bundler that sets the installed path to Gem.user_dir, I would be happy to merge it.
you:
> I would do it if I had any trust in the project. The first time I use Bundler it doesn't work. I report a problem that clearly would affect all Arch Linux users and it gets immediately closed. Why would I even use Bundler? I can just grep the Gemfile and use gem install directly. So much for Bundler's convenience.
me:
> Well gee, I wonder why felipec is so mad that bundler doesn't want to tie itself to the gem implementation and config file, and responds by "losing all trust in the project" and refusing to take up their offer of a patch to add a similar option to bundler, which would be read from e.g. $BUNDLE_USER_INSTALL (settable in /etc/profile.d/bundler.sh) or $HOME/.bundle/config
But as we all know, you did not submit this Arch ticket out of an interest in making ruby or gem or bundler better, but out of an interest in "sticking it to the rubygems team". I guess if you really cared, you'd have submitted the patch for bundler in 2013 and the solution here would have been infinitely easier, and probably present for the better part of a decade to boot.
But of course you insist you've tried so hard and sent patches in trying to be helpful, Mr. I Would Submit Patches If I Had Any Trust In The Project
Wrong.
> From 2013 at least you already created your reputation in the bundler developer community, which definitively involved you claiming you were morally unwilling to submit patches:
I have contributed at least 2700 commits to 28 open source projects, including Ruby MRI itself (which didn't track my contributions).
https://www.openhub.net/accounts/felipec
Stop the ad hominem attacks.
> Well gee, I wonder why felipec is so mad that bundler doesn't want to tie itself to the gem implementation and config file
You don't know what I want.
> refusing to take up their offer of a patch to add a similar option to bundler
That offer does not help me, nor the Arch Linux community. Adding a --user-install option to Bundler does **NOT** fix this issue.
> But as we all know, you did not submit this Arch ticket out of an interest in making ruby or gem or bundler better
Now you are accusing me of bad faith. This is objectively the worst way to reply to a contributor. I am done replying to you.
Anyone who actually wants to fix the actual problem (which Eli isn't even interested in understanding) let me know.
Incorrect, I am not now accusing you of bad faith.
I accused you of bad faith on December 06, 2020 and haven't seen any reason to change my mind.
> I have contributed at least 2700 commits to 28 open source projects
What does this have to do with the bundler project, which you refused to contribute to with the message:
> I would do it if I had any trust in the project. The first time I use Bundler it doesn't work. I report a problem that clearly would affect all Arch Linux users and it gets immediately closed. Why would I even use Bundler? I can just grep the Gemfile and use gem install directly. So much for Bundler's convenience.
If all you can say is "no, you're wrong", but you don't specify what is "right" instead, then all we can possibly judge this ticket based on, is the analysis that analyzed, i.e. mine. And you may find this ticket being closed again.
Protip: talking all about how the other person engages in ad hominem, then engaging in it yourself ("which Eli isn't even interested in understanding" as an excuse to not explain your issue) doesn't really work out so great. But you could always try to NOT be the person that submitted:
FS#68868https://gitlab.gnome.org/GNOME/vte/-/issues/319
and then did the exact same shenanigans here with your attitude toward the rubygems/bundler developers.
This task will be resolved when:
1. gem does user installs by default (not system install)
2. bundler does user installs by default (not system install)
3. both gem and bundler use the same location
As of today only #1 is fulfilled.
The proposed solution:
export GEM_HOME="$(ruby -e 'puts Gem.user_dir')"
In /etc/profile.d/ruby.sh does all of the above.
No other solution has been suggested to fix all of the above.
Originally this task was closed with the rationale "upstream", presumably suggesting upstream will fix this, which is clearly not true. Supposedly the upstream task that is going to fix this is https://github.com/rubygems/rubygems/issues/4031, but that is not true, because it would not necessarily fix all the 3 points above. There is no patch, and no comment since December 2020.
Even if rubygems issue 4031 is fixed, we would still need to set an environment variable like BUNDLE_NO_SUDO, so we still need /etc/profile.d/ruby.sh. And of course the fact that bundler doesn't do sudo doesn't fix *any* of the points above.
Nobody ever suggested that bundler succeeding in installing gems in /usr was the problem.
Now, I have proposed 3 solutions:
1. Set GEM_HOME in /etc/profile.d
2. Apply a patch like the one I sent to rubygems: https://github.com/rubygems/rubygems/pull/4028 (closed with no reason given)
3. Override Gem.default_dir by the functionality rubygems gives packagers: operating_system.rb (see
FS#70961)*Nobody* else has provided another patch, or another solution, not even upstream rubygems developers.
2) I don't see the utility of this, it merely gives another way of specifying the cli option from gemrc.
3) seems plausible. As a matter of curiosity how would you then specify the system install dir (e.g. when packaging in a PKGBUILD I guess?)
> Nobody ever suggested that bundler succeeding in installing gems in /usr was the problem.
I certainly suggested it. If the default is for gem to install to the user dir, and bundler to abort with a permission error unless manually directed to the user dir, then that's a valid and reasonable solution to the problem as far as a distro is concerned, and if people want bundler to work out of the box then that's an upstream bug we don't care about.
Consistency is maintained, and bundler then acts the same way python-pip has since time immemorial until relatively recently when it learned to log a warning and switch to the python user dir upon seeing a permission error in the system dir.
4. A whole branch with patches, tests, documentation, an update to `gem env`, and bug fixes completely ready for upstream to pick up as-is
https://github.com/felipec/rubygems/tree/fc/user-install
Then all Arch Linux packagers have to do is:
def user_install
true
end
Or even better: `Process.uid != 0` to turn this off automatically for the root user.
The whole thing is:
lib/rubygems/commands/environment_command.rb | 5 +++++
lib/rubygems/commands/uninstall_command.rb | 2 +-
lib/rubygems/defaults.rb | 8 ++++++++
lib/rubygems/installer.rb | 20 ++++++++++++--------
lib/rubygems/installer_test_case.rb | 12 ++++--------
lib/rubygems/path_support.rb | 4 +++-
lib/rubygems/test_case.rb | 2 +-
lib/rubygems/uninstaller.rb | 17 ++++++++++++-----
test/rubygems/test_gem.rb | 10 ++++++++++
test/rubygems/test_gem_commands_uninstall_command.rb | 4 ++--
test/rubygems/test_gem_installer.rb | 18 +++++++++---------
test/rubygems/test_gem_uninstaller.rb | 21 ++++++++++++++++++++-
12 files changed, 87 insertions(+), 36 deletions(-)
5. A simple patch that only adds `Gem.user_install` so that distributions can easily pick it up and enable in their operating_system.rb
https://github.com/felipec/rubygems/commit/1dfb46a174e06cb940cc052206eb10ea10bcea1a
6. An even simpler patch that does the same as above without any need for operating_system.rb:
- @home = env["GEM_HOME"] || Gem.default_dir
+ @home = env["GEM_HOME"] || Gem.user_dir
All these solutions work perfectly fine, and all other comments are wrong, because 1) gemrc is ignored by bundler, and 2) to install gems in the system directory all a user has to do is --no-user-install.
Since rubygems maintainers have blocked me I sent a mail with a couple of patches to the ruby-talk mailing list, and a link the the branch in my GitHub repository.
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/441137
Nobody else has proposed any solution, much less a single patch.
And to demonstrate how the rubygems handling of user installs is so bad, here's a list of all the *huge* modifications all distributions do, because they have given up on rubygems doing something sensible:
Debian: 110 lines:
https://salsa.debian.org/ruby-team/rubygems-integration/-/blob/master/lib/rubygems/defaults/operating_system.rb
Fedora: 148 lines:
https://src.fedoraproject.org/rpms/rubygems/raw/rawhide/f/operating_system.rb
Gentoo: 63 lines:
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-ruby/rubygems/files/gentoo-defaults-4.rb
But that's only one of the default files, Gentoo has *three*.
This is the kind quality commits that have led us here:
https://github.com/rubygems/rubygems/commit/6d33ec4a91
That's when they introduced the user installs: if Gem.dir isn't writable, just install to ~/.gem.
I wrote a blog post explaining my solution:
https://felipec.wordpress.com/2022/08/25/fixing-ruby-gems-installation/
My new proposal is even simpler now:
https://github.com/felipec/rubygems/commit/39f6721bce
All Arch Linux would need to do is distribute an `operating_system.rb` file overriding Gem.default_install so it returns Gem.user_dir. All distributions already provide an `operating_system.rb` file, which was created by RubyGems developers precisely for this purpose: to override the defaults.
Another option would be to patch the code to change Gem.default_dir to Gem.user_dir in Gem::PathSupport.initialize which achieves the same purpose. It wouldn't be a clean solution, but certainly not any less clean than the current patch to Gem.install_extension_in_lib, which BTW is the wrong way to do it, the correct way is overriding the method in `operating_system.rb` just like Fedora does.
Right now if users don't configure GEM_HOME `bundle install` will try to install gems in system directories using sudo.
How much longer are you going to keep bothering Arch Linux users in the hopes that upstream fixes this?
There is no need to wait, you are already patching rubygems, just add another patch to fix the problem in the meantime. If and when upstream fixes the problem, then you remove the patch.
If you are interested in fixing this issue I can provide you with a solution that deals with all the problems in a clean way.
PS: the current discussion: https://github.com/rubygems/rubygems/issues/5682