Arch Linux

Yeah, I know this one has been reported a million times.
http://bugs.archlinux.org/task/15351
http://bugs.archlinux.org/task/15335

But this really is a bug. We include warnings for much much less. (Maybe start reporting spurious upgrade warnings as "bugs"?)

It is far to easy to accidentally do this, especially considering the last ISO is from February, back in the readline5 days.

@Aaron: should we embrace a guideline to echo "You've just updated <foobar> that requires massive package rebuild due to soname bump. Make sure you have your packages up to date before reporting bugs." in post_upgrade of packages that cause massive rebuilds?

I know this comment doesn't probably belong here, but I find the linked blog entry utterly funny. The underlying sarcasm, the sterile critics, some totally wrong statements and the impossibility to add comments classify the aforementioned writing as blatant FUD.

Now, to the in-topic: apart from some warnings here and there, how could the dependency problem be solved? I mean, even when doing version checking and using the '>=' operator in the depends array, how can we protect ourselves from ABI-incompatible upstream updates without making package creation a pain?

I'm just brainstorming here, I have no working idea. But. Since there's no upstream standard on software versioning or ABI compatibility, this issue is impossible to predict, so the solution has to be found in dependencies. Particular attention must be made at what I call "the version skipping problem": if the user skips the major update and jumps to the next minor, she still has to be notified in some way. This leads to post_install functions full of old messages and version comparations, which is a problem that already affects many of out packages for different reasons.

A stub of solution could be the addition of a PKGBUILD flag, at least for core packages, that specifies the minimum ABI version needed for important libraries. Then core packages would be checked against it for sanity after every upgrade of such packages. Sure, it would slow down things when such packages are upgraded, but would ensure that a system is still bootable, thus able to be fixed.

Or just modify pacman to impose [core] to be upgraded all at once by default if the user doesn't specify a flag. Especially this proposal is on the fine line that divides sanity checks from user freedom, and I'm not really sure I like it, but it's the simplest to implement.

Other ideas? What do other distros do?

By the way, more here: http://www.reddit.com/r/linux/comments/92et4/the_arch_package_management_debacle/

IMO this is a consequence of the rolling release system. You should always -Syu, no exceptions. I'd even say, it's intended behaviour.

If one should always -Syu, should pacman even present "-Sy single_package" as an options? Or at least print a warning, in the same vein as warning not to build packages as root?

RedShift has it right here -- this behavior is intended. Rolling release means that you update everything at once. If that's not what you're looking for in a distro, then use another one.

"pacman -Sy <pkg>" is an option because pacman does not restrict a users stupidity. In fact, I do use that on occasion but I know why it is wrong in general and what might break.

No onto dealing with this problem...

Option 1: everything (in core?) has two versioned dependencies. e.g 'readline>=6' 'readline<7'. But then even if readline7 is ABI compatible with readline6, we have to do a rebuild. So we are relying on upstream versioning to be sane.

Option 2: readline provides libreadline.so.6 and everything should depend on the libraries it needs and not the package that provides them. This would be less of a hassle if we used -Wl,--as-needed to limit the number of directly linked libraries on our packages.

I dislike option 1 A LOT and option 2 requires rebuilding everything in the repos we decide to implement that scheme in. Despite being a lot of work, I thing option 2 is really the best way to deal with this. I would initially only implement it in [core] and let [extra] gradually catch up.

Option 2 is logical, but specifying .so files in depends array looks very ugly, RPM-like.

I would suggest option 3:

depends=(...) should contain all direct dependencies, even if they are provided by other dependencies anyway - this is good in many aspects, including clearly visible dependency on libjpeg even if package depends on poppler that depends on libjpeg anyway.
.PKGINFO of a package should include the exact versions of the dependencies the package was built with,
e.g. builtwith = foobar-6.1-2 in i-depend-on-foobar package's .PKGINFO,
or alternatively extend 'depend' syntax, e.g. depend = foobar[6.1-2] (note that foobar>=6[6.1-2] is possible as well).
The package that breaks the ABI and triggers rebuild should contain newabi variable with the version of package that caused the latest ABI change,
e.g. newabi=7.0-1 in foobar package's PKGBUILD (and corresponding entry in .PKGINFO, of course).

Then pacman can check for packages that were built with a version of foobar packages < 7.0-1 and hold the upgrade, warn user etc.
And it doesn't really matter if user skips a couple of versions, because it is always clear what packages were built with a version with old ABI.

Of course some changes to makepkg and pacman are required, and not all packages can be moved to the new scheme immediately.

This option can actually be usable for something more than dealing with .so-name bumps (which is why newabi needs to be named in a better way),
for example I can imagine a situation where this option can be used to handle a situation where all packages written in some scripting language need to be updated because of an unexpected very important syntax change in the language itself.

Allan, maybe a third solution to mainly avoid broken systems cause a so mismatch of readline and bash.
We provide in readline PKGBUILD a:
conflicts = (bash<4.0)
(Maybe other packages which make the system broken when not on the same so version, but AFAIK bash is the main package).
So if readline will get accidental updated without updating the releated bash version a conflict waring is given, so
the user knows that something bad will happen.

All other so bump problems could solve on command line, but we must avoid that a login is impossible.

I really don't feel that any modifications are necessary -- As Gerhard suggested, conflicts would be fine. Running -Syu to update is the intended behavior, and anyone straying from it should know what they're doing. Keep things simple, we don't need a fancy dependency system. What we have works, and it works well. User error is unavoidable with any system, and the more you try to prevent it, the more you're going to annoy people who actually know what they're doing. Let those types use a hand-holding distro, and keep Arch simple and clean.

"Let those types use a hand-holding distro, and keep Arch simple and clean."

I don't really consider "Just use -Syu" a simple and clean solution. In fact, Arch's system is not simple, but trivial, and it is certainly not clean. Something like NixOS looks (on the surface) clean and, as a byproduct, simple.

"depends=(...) should contain all direct dependencies, even if they are provided by other dependencies anyway"
This (obviously) slows down the dependency checking in pacman. Some packages have orders of magnitudes more deps than what are specified. However, thinking about this, that would be no worse that checking all needed libraries which is why I though only to do this for [core].

"We provide in readline PKGBUILD a: conflicts = (bash<4.0)"
That is an elegant solution for readline/bash issue but I think it could run into difficulty when pro-actively selecting the needed conflicts for general library updates. Although, doing that for readline/bash may be enough given this is probably one of the worst pairs of packages to break..

I agree, doing it for readline and bash only will be fine

Reposting what I posted on the dev list:

I can't say how irritating this is. It's such a special case it's
ridiculous. bash and readline... ok, but what about people who use zsh
as their primary shell, should we start adding special cases there?
What about people using scsh or fish?

Here's the only sane solution I could possibly think of, but we'd have
to make it a global policy and do it everywhere:

bash: depends=(readline=6.0.XXX)

And rebuild every time. This is the only absolutely complete way to
cover all bases here, without the ability to predict the future
(readline>=6 readline<7 assumes we can predict the future and that 6.1
will not be ABI incompatible).

Suggestion: have pacman do partial dep checking based on what is
specified. So if I say "readline=6.0" pacman will match "6.0.123" and
"6.0.007".