FS#70326 - [arch-audit] not working when started by systemd

Attached to Project: Community Packages
Opened by Andrej Podzimek (andrej) - Wednesday, 07 April 2021, 03:22 GMT
Last edited by Buggy McBugFace (bugbot) - Saturday, 25 November 2023, 20:02 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Christian Rebischke (Shibumi)
Levente Polyak (anthraxx)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

arch-audit works when run manually, but fails when started by systemd. This has been happening since a recent update a few hours ago.

The logs report DNS resolution problems. Yet the machine doesn't have any such problems: Both systemd-resolve (--status) and dig work fine. Bind runs on the machine, which serves as an authoritative DNS as well as a caching resolver.

A manual resolution of the reported domain name (see logs below) yields both IPv6 and iPv4 addresses and the JSON file can be downloaded without issues (tested only with IPv6, who cares about IPv4 anyway).

Starting arch-audit (-u) manually under a normal user account works fine as well.

That^^^ said, my speculation would be that this is caused by one of the (tens of) restrictions specified in the systemd .service file for arch-audit.


Additional info:

* package version(s)

arch-audit 0.1.19-1
linux 5.11.11.arch1-1
systemd 248-3

* config and/or log files etc.

$ journalctl -u arch-audit:
[...]
systemd[1]: Starting Audit installed packages against known vulnerabilities...
arch-audit[980123]: Error: failed to get AVG json
arch-audit[980123]: Because: failed to fetch AVGs from URL
arch-audit[980123]: Because: Failed to send request
arch-audit[980123]: Because: error sending request for url (https://security.archlinux.org/all.json): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution
arch-audit[980123]: Because: error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution
arch-audit[980123]: Because: dns error: failed to lookup address information: Temporary failure in name resolution
arch-audit[980123]: Because: failed to lookup address information: Temporary failure in name resolution
systemd[1]: arch-audit.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: arch-audit.service: Failed with result 'exit-code'.
[...]

$ cat /etc/resolv.conf
nameserver ::1

$ egrep -v '^[[:space:]]*($|#)' /etc/systemd/resolved.conf
[Resolve]
DNS=::1#<<<censored (domain name)>>>
FallbackDNS=
DNSSEC=yes

$ systemd-resolve --status
Global
Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=yes/supported
resolv.conf mode: foreign
Current DNS Server: ::1#<<<censored (domain name)>>>
DNS Servers: ::1#<<<censored (domain name)>>>

Link 2 (charon1g0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=yes/supported
Current DNS Server: ::1
DNS Servers: ::1

Link 3 (charon10g0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=yes/supported
Current DNS Server: ::1
DNS Servers: ::1

[...] ~5 further interfaces; all the same

$ dig +short security.archlinux.org AAAA
2a01:4f9:c010:aa84::1

$ systemd-resolve security.archlinux.org
security.archlinux.org: 2a01:4f9:c010:aa84::1
95.217.239.55

-- Information acquired via protocol DNS in 17.8ms.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: network

<offtopic>BTW, this looks concerning: "Data is authenticated: no;" A security-related domain without DNSSEC? Srsly? It's unrelated to the problem, but worth pointing out nonetheless.</offtopic>

* link to upstream bug report, if any

N/A

Steps to reproduce:

This fails: # systemctl restart arch-audit
Also the timer fails.
This works: $ arch-audit -u
$ arch-audit
This task depends upon

Closed by  Buggy McBugFace (bugbot)
Saturday, 25 November 2023, 20:02 GMT
Reason for closing:  Moved
Additional comments about closing:  https://gitlab.archlinux.org/archlinux/p ackaging/packages/arch-audit/issues/1
Comment by Andrej Podzimek (andrej) - Wednesday, 07 April 2021, 03:49 GMT
Alright, I've identified the troublesome two lines in /usr/lib/systemd/system/arch-audit.service:

IPAddressDeny=localhost
RestrictAddressFamilies=AF_INET AF_INET6

Each of those restrictions is OK when used in isolation, but the *combination* of the two causes DNS resolution to fail. This is because getaddrinfo() (or whatever is used under the hood):

1. cannot contact systemd-resolved on the well-known socket /run/systemd/resolve/io.systemd.Resolve, because the AF_UNIX address family is banned.
2. cannot contact Bind on ::1 directly (which is what systemd-resolved would ultimately do), because "localhost" addresses, which includes ::1/128, are banned.

With both the default and the fallback failing, we have a DNS resolution failure!

The workaround is to create a file called (e.g.) /etc/systemd/system/arch-audit.service.d/workaround.conf (and then run systemctl daemon-reload). There are at least 3 possible workarounds (contents of workaround.conf):

Workaround No. 1: Allow AF_UNIX (which enables DNS resolution via systemd-resolved). (You may want to disable AF_INET though; no need to allow a deprecated 32b address experiment from 1975.)

[Service]
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6

Workaround No. 2: Allow IPv6 localhost (which enables DNS resolution via Bind on localhost):

[Service]
IPAddressAllow=::1/128

Workaround No. 3: Disallow IPv4 localhost only (who wants IPv4 anyway); but leave IPv6 localhost allowed:

[Service]
IPAddressDeny=
IPAddressDeny=127.0.0.0/8

Any of these 3 workarounds makes it work again. I'm using a combination of the first two (plus IPv4 disabled) for the time being:

[Service]
RestrictAddressFamilies=AF_UNIX AF_INET6
IPAddressAllow=::1/128

This way the address resolution library can just pick one of the mechanisms (systemd-resolved or Bind) as it sees fit. Not sure if there's any extra risk in doing that.
Comment by Andrej Podzimek (andrej) - Wednesday, 07 April 2021, 04:07 GMT
A summary is that there are 2 problems:

1. On *all* systems the restrictions ban DNS resolution using sytemd-resolved (i.e., this option always fails).
2. Specifically on systems that run a DNS server on localhost (::1), direct DNS resolution (a fallback option, in a sense) is banned as well.

What is tried first and what is a fallback is governed by /etc/nsswitch.conf. In my case, for example:

hosts: files mymachines myhostname resolve [!UNAVAIL=return] dns

So, from a performance / latency standpoint, the best workaround is No. 1 above, because that way getaddrinfo() doesn’t have to go through a failed attempt to contact systemd-resolved first and fallback to direct DNS afterwards; systemd-resolved will just work.
Comment by Andrej Podzimek (andrej) - Wednesday, 07 April 2021, 04:17 GMT
Here’s the upstream unit file: https://github.com/ilpianista/arch-audit/blob/master/contrib/systemd/arch-audit.service
This was the change that caused trouble: https://github.com/ilpianista/arch-audit/commit/57602024f4d3b1f32b8020574b589596f82639de

The unit file has *not* been changed recently. But the package upgrade is recent:

[2021-04-04T23:25:34+0200] [ALPM] upgraded arch-audit (0.1.18-1 -> 0.1.19-1)

That^^^ explains why it worked fine on my system up until a few days ago.
Comment by Buggy McBugFace (bugbot) - Tuesday, 08 August 2023, 19:11 GMT
This is an automated comment as this bug is open for more then 2 years. Please reply if you still experience this bug otherwise this issue will be closed after 1 month.

Loading...