Arch Linux

Please read this before reporting a bug:

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!

FS#43450 - [ca-certificates] Some more concerns / notes on 'system trust', comparison with Fedora/RHEL/Debian

Attached to Project: Arch Linux
Opened by Adam Williamson (AdamWill) - Thursday, 15 January 2015, 03:50 GMT
Last edited by Eli Schwartz (eschwartz) - Sunday, 07 January 2018, 07:14 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Pierre Schmitz (Pierre)
Jan Alexander Steffens (heftig)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No


Hi, Arch-ians!

I work on Fedora QA; I'm not formally a part of the team working on the 'shared system certificates' stuff but I've been doing quite a lot of testing / research / other work on it lately.

As part of that I noted that Arch has broadly adopted Fedora's system (good choice ;>), but I had a few thoughts / notes on the chosen layout for the generated CA trust stores.

It seems a bit unfortunate to choose different primary locations for the extracted trust stores - they're under /etc/pki/ca-trust/extracted on Fedora but /etc/ca-certificates/extracted on Arch, and separated into subdirs by type on Fedora (java / openssl / pem) but not on Arch. Is it possible to consider keeping this consistent? It would be nice if in future we could get more consistency between distros in general on the location of such files.

The other thoughts I had are all about /etc/ssl/certs .

AFAICS, prior to using the Fedora system, you were using the Debian system. AIUI, the Debian system results in /etc/ssl/certs containing individual certificates *in plain PEM format*, with the hash symlinks required to act as an OpenSSL CApath. When I say 'plain PEM format', I mean the form that is identified with "BEGIN CERTIFICATE" and has no purpose information, as opposed to the extended OpenSSL 'trusted certificate' format, identified with "BEGIN TRUSTED CERTIFICATE", which contains purpose information.

Arch's current update-ca-trust creates /etc/ca-certificates/extracted/cadir with 'trust extract --format=openssl-directory', and populates /etc/ssl/certs with symlinks to all the files in it.

the 'openssl-directory' format creates 'trusted certificate' files, not basic PEM files. So by doing things this way, you've introduced a change from the previous behaviour that I'm not entirely sure was intentional.

/etc/ssl/certs acts as part of OpenSSL's default trust store in Arch; you build OpenSSL with '--openssldir=/etc/ssl', and OpenSSL uses OPENSSLDIR/certs (read as a CApath) as part of its trust store. In Fedora we're just currently considering the implications of switching OpenSSL to using its 'trusted certificate' format in its default certificate store, and we're not 100% sure it's safe to do yet - one of our experts in the area has the initial thought that it may not be safe for applications which don't specify a purpose when validating certificates. I'm currently looking into that.

Aside from the question of what effect changing from basic PEM certificates to 'trusted certificates' in OpenSSL's default store has on OpenSSL, there's another consideration. Apps built against another SSL/TLS library or accessing trust store directly for some reason might assume /etc/ssl/certs will be a directory of basic PEM certificates - because this is what it is, and has been forever, on Debian. Populating it with 'trusted certificate' files, especially before Debian does, could possibly have unexpected consequences. GnuTLS, for instance, has the ability to use a directory of files as a trust store (since 3.3.6), but cannot handle OpenSSL's 'trusted certificate' format.

You do *also* create a /etc/ssl/cert.pem, which OpenSSL will use as the *other* part of its trust store - it uses OPENSSLDIR/cert.pem (as a CAfile) as well as OPENSSLDIR/certs . Thus you have a somewhat interesting and I think unique default OpenSSL trust store: it will be using both /etc/ssl/cert.pem and the contents of /etc/ssl/certs , both of which are output by update-ca-trust but in different forms. /etc/ssl/certs contains 'trusted certificates', /etc/ssl/cert.pem contains basic PEM certificates, different --filter parameters are used in each case. I don't think any other distro uses such a setup, and it could _conceivably_ possibly lead to unexpected OpenSSL behaviour, I guess. There's also obviously a large degree of duplication there, though I'm not sure how much that matters in performance terms in practice.

Your /etc/ssl/cert.pem is a double symbolic link, which seems a bit unnecessary (it's a symlink to /etc/ssl/certs/ca-certificates.crt , which is in turn a symlink to /etc/ca-certificates/extracted/tls-ca-bundle.pem ).

You create a /etc/ssl/certs/ . AFAIK outside of Arch this file exists only in Fedora/RHEL, and it's a bit of an odd duck there. Debian does not have such a file. /etc/ssl is not canonical for Fedora/RHEL, we just carry /etc/ssl/certs as a symlink to /etc/pki/tls/certs as some kind of attempt at 'third party compatibility' - see and . Allegedly having a /etc/ssl/certs/ca-bundle.crt somehow helped whatever version of the Firefox Flash plugin existed in 2010. I've poked around, and there's really no software that uses /etc/ssl/certs/ca-bundle.crt or /etc/ssl/certs/ so far as I can tell, just a few stray forum references. You don't create a /etc/ssl/certs/ca-bundle.crt - if anything I'd expect that'd be _slightly_ more likely to be used than /etc/ssl/certs/ , but I'm not sure it's worth bothering with either.

If you wanted to provide Fedora-style files for compatibility, the most likely ones are /etc/pki/tls/certs/ca-bundle.crt and /etc/pki/tls/cert.pem as PEM bundles and /etc/pki/tls/certs/ as an OpenSSL extended bundle.
This task depends upon

Closed by  Eli Schwartz (eschwartz)
Sunday, 07 January 2018, 07:14 GMT
Reason for closing:  Implemented
Comment by Jan Alexander Steffens (heftig) - Thursday, 02 April 2015, 20:07 GMT
Thank you for your thoughts.

> It seems a bit unfortunate to choose different primary locations for the extracted trust stores

Yeah, I agree. I probably should have stuck with the RH paths instead of creating yet another new scheme. I'm not sure changing them again now is worth it.

> The other thoughts I had are all about /etc/ssl/certs

It seems OpenSSL prefers using cert.pem. It doesn't look at the certs dir if it finds the certificate in the bundle.

With p11-kit 0.23 and ca-certificates-utils 20150402, /etc/ssl/certs/ now uses the classic PEM format again, so it matches cert.pem and we're back to Debian compatibility here.

> Your /etc/ssl/cert.pem is a double symbolic link

IIRC this is because at one point I had no tls-ca-bundle.crt and extracted to /etc/ssl/certs/ca-certificates.crt directly.

Now ca-certificates.crt is linked by update-ca-trust. I could put ca-certificates.crt into the package instead, but then we would require user intervention to upgrade. Pacman does not have a mechanism like RPM's shadow files. At the very least I can move it from update-ca-trust to the package post_install/pre_remove handling.

Anyway, cert.pem can point to tls-ca-bundle.crt directly.

> You create a /etc/ssl/certs/

It's still unused here. I think I'll move it to the other bundles in /etc/ca-certificates/extracted/, so /etc/ssl/certs should only contain symlinks.
Comment by Eli Schwartz (eschwartz) - Tuesday, 08 August 2017, 23:05 GMT
It looks like all the concerns given here have been addressed, can we close this?