FS#68763 - [influxdb] Update to influxdb 2.0.2

Attached to Project: Community Packages
Opened by Damjan Georgievski (damjan) - Friday, 27 November 2020, 19:42 GMT
Last edited by Morten Linderud (Foxboron) - Saturday, 06 March 2021, 12:37 GMT
Task Type Feature Request
Category Packages
Status Closed
Assigned To Morten Linderud (Foxboron)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description: the influxdb 2.0 project had substantial project changes, including the build process and repository organization.
Now it has Chronograph and Kapacitor integrated, and an additional new query language "Flux".

I've updated the PKGBUILD for the 2.x release (attached).

Note:
In my version I've also removed influxdb.tmpfiles and influxdb.sysusers, and added a influxdb.service that runs as a DynamicUser=yes service, with StateDirectory=influxdb (attached).
If that's not desired, the package can be reverted to the current setup (I personally find dynamic user service better, more secure and easier to manage).


ps.
if it's more convenient, I can open a PR on github too.

This task depends upon

Closed by  Morten Linderud (Foxboron)
Saturday, 06 March 2021, 12:37 GMT
Reason for closing:  Fixed
Additional comments about closing:  Updated to 2.x release
Comment by Damjan Georgievski (damjan) - Friday, 27 November 2020, 19:44 GMT
Forgot to update the sha256sum of the influxdb.service file, it should be a22ba54f10fa98ec4e81e864c0c3181e53dedd9e23e1d112ac4ad22c4fa4d9ee
Comment by Morten Linderud (Foxboron) - Saturday, 28 November 2020, 12:17 GMT
Did you try building this before making this bug report?
Comment by Damjan Georgievski (damjan) - Saturday, 28 November 2020, 12:26 GMT
> Did you try building this before making this bug report?

yes, I'm running it currently
Comment by Morten Linderud (Foxboron) - Saturday, 28 November 2020, 12:29 GMT
They fucked up the Makefile (again) so running with -j$(nproc) creates race conditions with targets depending on the yarn build.
Comment by Damjan Georgievski (damjan) - Saturday, 28 November 2020, 12:34 GMT
ah, I did not try that, I haven't set MAKEFLAGS in /etc/makepkg.conf
Comment by Morten Linderud (Foxboron) - Saturday, 28 November 2020, 12:50 GMT
This wasn't built in a clean chroot? cargo is missing, some llvm/clang so-name as well. So this doesn't build as presented.
Comment by Damjan Georgievski (damjan) - Saturday, 28 November 2020, 13:17 GMT
> This wasn't built in a clean chroot?

indeed, I haven't fixed the deps (except replacing npm with yarn). thanks for the review, I'll post a second draft soon (have some chores to run currently).

Comment by Damjan Georgievski (damjan) - Saturday, 28 November 2020, 19:39 GMT
Dependencies updated.

The only thing I'm not sure of is `backup=('etc/influxdb/influxdb.conf')` - the file was present in the old package, but not in the new - should I leave it?

   PKGBUILD (1.4 KiB)
Comment by Damjan Georgievski (damjan) - Tuesday, 15 December 2020, 17:03 GMT
v2.0.3 has been released https://github.com/influxdata/influxdb/releases/tag/v2.0.3
updated PKGBUILD attached
   PKGBUILD (1.4 KiB)
Comment by Morten Linderud (Foxboron) - Tuesday, 15 December 2020, 17:07 GMT
Been on the back burner because the test suite doesn't work on my end. I'll try pick this up this month but no promises :)
Comment by Jurek Olden (oldenj) - Saturday, 02 January 2021, 12:17 GMT
Per https://docs.influxdata.com/influxdb/v2.0/reference/config-options/ the 'INFLUXD_CONFIG_PATH' environment variable specifies an optional config directory.
Maybe it is a good idea to set Environment="INFLUXD_CONFIG_PATH=/etc/influxdb" in the unit file and even include a configuration file with the options commented out.
This way you can easily set your tls-cert and other stuff from the usual location.

It's also worth noting that InfluxDB creates a new, version 2 compatible, config file from the old config file.
I upgraded with 'sudo -u influxdb influxd --bolt-path /var/lib/influxdb/influxd.bolt --engine-path /var/lib/influxdb/engine upgrade' which gave me /etc/influxdb/config.toml with options such as tls-cert, tls-key, bolt-path, engine-path etc.
I guess people that are upgrading will expect this config file to be used.
Comment by Damjan Georgievski (damjan) - Tuesday, 05 January 2021, 19:51 GMT
sure,`Environment="INFLUXD_CONFIG_PATH=/etc/influxdb"` works even when /etc/influxdb doesn't exist (I'm running it without a config file at all), so that's acceptable
Comment by Morten Linderud (Foxboron) - Wednesday, 20 January 2021, 20:41 GMT
Blockers:

* Test suite is failing even after cloning the test data.
* -j1 is needed because of a racey build system. Everything takes a while to build.

Need to sit down and unwrangle parts of this upstream or pray a minor release suddenly fixed this.

I pushed a WIP pkgbuild
https://github.com/Foxboron/archlinux-pkgbuilds/blob/b0f4375264ce7ac0220f67f131e1466146040105/influxdb/PKGBUILD
Comment by Damjan Georgievski (damjan) - Thursday, 21 January 2021, 13:10 GMT
you're missing the .service unit file in the directory.
Comment by Morten Linderud (Foxboron) - Thursday, 21 January 2021, 13:14 GMT
Yes, it's WIP. I never reach the `package()` function because of the failing tests.
Comment by Damjan Georgievski (damjan) - Friday, 12 February 2021, 12:15 GMT
2.0.4 is out now,
https://github.com/influxdata/influxdb/releases/tag/v2.0.4

it fails to compile again :(
Comment by Damjan Georgievski (damjan) - Thursday, 25 February 2021, 13:29 GMT
* updated to 2.0.4
* force -j1 to stop parallel builds
* removed the bzr make dependency, it wasn't needed
   PKGBUILD (1.4 KiB)
Comment by Morten Linderud (Foxboron) - Thursday, 25 February 2021, 13:36 GMT
The interesting part is if you get the test to pass or not frankly.
Comment by Damjan Georgievski (damjan) - Thursday, 25 February 2021, 13:50 GMT
ah yes, forgot to mention, yes the tests did pass for me!

do they not work for you?
Comment by Morten Linderud (Foxboron) - Thursday, 25 February 2021, 14:10 GMT
Right, because you just run `go test ./...` and not the complete test suite. Hmmmm

I did get an email pointing out some issues.


- https://github.com/influxdata/influxdb/issues/16463
- https://github.com/influxdata/influxdb/issues/20294
- https://github.com/influxdata/influxdb/issues/20718

- https://github.com/influxdata/influxdb/labels/area%2Ftests

I'll just push with running `go test ./...` and leave UI testing for upstream if I can't get this to work soon'ish
Comment by Damjan Georgievski (damjan) - Thursday, 25 February 2021, 14:13 GMT
> Right, because you just run `go test ./...` and not the complete test suite. Hmmmm

Ahhh, I guess I copied that from the original PKGBUILD, didn't think to update it 🤦
Comment by Morten Linderud (Foxboron) - Thursday, 25 February 2021, 14:14 GMT
Omg! They did actually this release! Amazing!

I'll try get this shipped in day or two since it takes a little while to build.
Comment by Morten Linderud (Foxboron) - Thursday, 25 February 2021, 18:24 GMT
I have built and released influxdb 2.0.4. Thank you so much for the help over the past few months!

I realized i forgot the config env variable change so I'll push a fixup with that and any other stuff that might crop up over the next days :)
Comment by Antony Male (canton7) - Saturday, 27 February 2021, 17:00 GMT
In case it helps anyone else, these are the steps I needed to go through to get influxdb working again, after the upgrade broke it:

1. systemctl stop influxdb.service
2. influxd --bolt-path /var/lib/private/influxdbv2/influxd.bolt --engine-path /var/lib/private/influxdbv2/engine --v2-config-path /etc/influxdb/config.toml upgrade
3. Move /var/lib/private/influxdb2 to /var/lib/private/influxdb
4. Edit /etc/influxdb/config.toml and change bolt-path and engine-path to start with /var/lib/influxdb
5. systemctl edit influxdb.service and add [Service] Environment=INFLUXD_CONFIG_PATH=/etc/influxdb
6. systemctl start influxdb.service

I'm not convinced that that's the best way to do the upgrade (or even a correct way!). Some guidance would have been nice: normally packages print instructions when manual intervention is required, but here everything silently stopped working, which is disappointing. The wiki is also out of date.
Comment by Morten Linderud (Foxboron) - Saturday, 27 February 2021, 17:04 GMT
I get the frustration but you should see there is a 1.8.3 -> 2.0.4 upgrade in the update and handle it accordingly. Feel free to add the steps you listed to the wiki!

The INFLUXD_CONFIG_PATH was forgotten but will be added in a patch release soon'ish.
Comment by Antony Male (canton7) - Saturday, 27 February 2021, 17:14 GMT
I stand by what I said: normally packages at least print a warning when user intervention is required. Silent breakage on Arch due to its rolling release system is mercifully rare, despite Arch's reputation!

Sure, in an ideal world every user would google every package whose major version changed in an upgrade to see whether any manual intervention was required. Most won't. That's why an upgrade warning, and some easy-to-find instructions on how to upgrade, are nice things to provide, rather than every user wasting a chunk of their weekend trying to find the right magic incantation.

If you're confident that the steps I took are correct (they seem clunky, but whole DynamicUser thing makes running influxd as the influxdb user outside of the context of a service annoying), then I'll happily update the wiki.
Comment by Morten Linderud (Foxboron) - Saturday, 27 February 2021, 17:22 GMT
> Silent breakage on Arch due to its rolling release system is mercifully rare, despite Arch's reputation!

That is because larger breakage between major releases doesn't happen a lot. Not because we do anything with them though. In this case it did however.
Comment by Morten Linderud (Foxboron) - Saturday, 27 February 2021, 18:47 GMT
I forgot the last part.

>If you're confident that the steps I took are correct (they seem clunky, but whole DynamicUser thing makes running influxd as the influxdb user outside of the context of a service annoying), then I'll happily update the wiki.

Yes, please do. You can omit the service editing since it will be fixed. But rest looks good and I'll update with any other issues if they get reported.

Thanks!

Comment by Jurek Olden (oldenj) - Saturday, 27 February 2021, 18:49 GMT
I think while it might not be a maintainers responsibility to notify users, Antony does have a point. Adding a notification, as a courtesy, shouldn't be too much of a bother. By the way, i had a package (samba) break completely even without the update being major, i would have been real happy about a warning there.

Anyway, thanks a lot for your work, it's much appreciated.
Comment by Morten Linderud (Foxboron) - Saturday, 27 February 2021, 19:14 GMT
I can't figure out what is going to break between major version numbers for users. The best would be release notes you have access to anyway and further clutters the installation messages.

Samba breaking is nothing new and it's samba. Friends don't let friends use samba.

If people actually care they can send me an email and I'll happily drop it into `[community-testing]` before hitting `[community]`.
Comment by Jurek Olden (oldenj) - Saturday, 27 February 2021, 19:18 GMT
Fair enough. And if i could ditch samba i would, but sometimes you can't really choose.
Comment by Jakub Klinkovský (lahwaacz) - Saturday, 27 February 2021, 20:35 GMT
How do you migrate the "[[collectd]]" section from the old /etc/influxdb/influxdb.conf to the new config.toml? Is it even supported? Does anybody have a working setup?
Comment by Damjan Georgievski (damjan) - Sunday, 28 February 2021, 10:24 GMT
> How do you migrate the "[[collectd]]"

afaik you use telegraf for that
Comment by Morten Linderud (Foxboron) - Saturday, 06 March 2021, 12:37 GMT
Closing this as it's gone a week :)

Loading...