Arch Linux

Cannot test right now. I pushed a version 5.4.2-2 in cty-testing.

Ask upstream to consider carrying arch's service file? (as well as the following)
https://github.com/grafana/grafana/blob/master/packaging/deb/systemd/grafana-server.service
https://github.com/grafana/grafana/blob/master/packaging/rpm/systemd/grafana-server.service

I'm not fan of upstream hosting downstream specific contents. I'll not make a move in that direction.

Those "downstream specific contents" are not downstream specific! The only specific thing about them is the location of a SysVInit-compatible shell script designed to pass optional options to override the builtin defaults on the program command line.

Please discuss with upstream, the idea of consolidating this messy set of copy-pasted files with extremely trivial variations, since the sysconfig file variables are only ever used in order to change the location of the --config file.

It makes no sense **even on Debian or Fedora** for the sysconfig/default file to set the default locations for things that are already specified in grafana.ini, nor does it make sense to have the pidfile be configurable in that manner when systemd has mandatory XDG_RUNTIME_DIR intended for this explicit use case.

I agree and I also think, in that case, that upstream could merge these files because the differences have no reason to exists (Type=, EnvironmentFile=, and ExecStart= option).

Eli, If you have time to start discussion with them, do it. You can act on my behalf, or, moreover, if you are interested to take over maintainership of grafana, let me known.

The server stopped work.
```
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=eror msg="Failed to start session" logger=con
text error="open /usr/share/grafana/data/sessions/d/6/d621c876cb06c63f: read-only file system"
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/login status=302 remote_addr=127.0.0.1 time_ms=12 size=35 referer=https://localhost/monitoring/login
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=eror msg="Failed to start session" logger=context error="open /usr/share/grafana/data/sessions/0/a/0a8c315abf9936a0: read-only file system"
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/ status=302 remote_addr=[::1] time_ms=0 size=40 referer=https://localhost/monitoring/login
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=eror msg="Failed to start session" logger=context error="open /usr/share/grafana/data/sessions/c/f/cfff0c0f78eadbc0: read-only file system"
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/login status=302 remote_addr=127.0.0.1 time_ms=6 size=35 referer=https://localhost/monitoring/login
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=eror msg="Failed to start session" logger=context error="open /usr/share/grafana/data/sessions/5/d/5df761d3e70ef6f7: read-only file system"
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/ status=302 remote_addr=[::1] time_ms=0 size=40 referer=https://localhost/monitoring/login
Dec 26 12:54:33 santa grafana-server[4983]: t=2018-12-26T12:54:33+0000 lvl=eror msg="Failed to start session" logger=context error="open /usr/share/grafana/data/sessions/3/2/32526567a33da992: read-only file system"
Dec 26 12:54:34 santa grafana-server[4983]: t=2018-12-26T12:54:34+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/login status=302 remote_addr=127.0.0.1 time_ms=7 size=35 referer=https://localhost/monitoring/login
```

this is because of `ProtectSystem=strict`

I think also adding

LogsDirectory=grafana
StateDirectory=grafana

to the systemd unit fixes this.

FYI, the following configuration suggested by slashME solved my issues:

LogsDirectory=grafana
StateDirectory=grafana

The log dir is in /usr/ which is wrong in many ways
Dec 31 09:32:43 calcium grafana-server[22809]: FileLogWriter("/usr/share/grafana/data/log/grafana.log"): Rotate: rename /usr/share/grafana/data/log/grafana.log /usr/share/grafana/data/log/grafana.log.2018-12-31.002: read-only file system

/usr/share/grafana/data is a symlink to /var/lib/grafana, and further .../log is a symlink to /var/log/grafana. There's no data under /usr it actually modifies

The current version in community can't even create its database on first startup, removing ProtectSystem=strict solves the problem. Maybe adding some ReadWritePaths also works, but I didn't try that...

I think the best solution is to move from the symlink tricks to a proper defaults path in the build process. Looks like custom.ini is here to achieve that.
I'm curious about the LogsDirectory/StateDirectory directives fixing the path access. I wanna test that.

Finally, custom.ini is not read when --config-file is set on command line.
So, I patched the default.ini with few sed lines (instead of adding our default path on the Execstart=).

I'm balanced between:
- switch from ProtectSystem=strict to ProtectSystem=full.
Because Grafana is legitimate to write on the filesystem where it is configured, except on protected paths (i.e: /etc, /usr, /home, /dev, etc).
So, if you want your data in /srv/grafana or /mnt/nfs/grafana, that perfectly allowed by only changing the grafana configuration.
But, if you wan to do dangerous things, like storing you data in /usr/share/grafana/data, you have to deal with a higher level security system.

- Keep ProtectSystem=strict and allow read-write on specific directories (i.e: /var/{lib,log}/grafana).
This offer, by default, a stronger configuration of grafana by allowing him to write only in its default directories.
So, it's offer a additional layer of security on grafana code to not overwrite data in others directory, which may not contains grafana data.
But, the grafana configuration file is no more the (only) place where we define grafana paths, as long as they are legitimate, so it's not a free security.

We don't have an Arch policy about which part of the filesystem should be set read-only or not with daemon. If you have opinions about this let me known.
To fix the broken situation, I'll pushed a package with correct default paths and ProtectSystem=full until there is more discussion.