FS#70633 - [timescaledb] old library version is needed during the first promscale restart after an upgrade

Attached to Project: Community Packages
Opened by Andrej Podzimek (andrej) - Wednesday, 28 April 2021, 23:54 GMT
Last edited by George Rawlinson (rawlinsong) - Sunday, 04 July 2021, 21:18 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Sven-Hendrik Haase (Svenstaro)
George Rawlinson (rawlinsong)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

After updating from 2.1.1 to 2.2.0, promscale started failing due to a PostgreSQL error.
The error was caused by missing timescaledb-2.1.1.so (the old library version).
The old library file is needed temporarily, once, so that promscale can upgrade the timescaledb in its PostgreSQL database(s).

The workaround is ugly:
1. Build 2.1.1 from AUR
2. Manually copy (only) timescaledb{,-tsl}-2.1.1.so from the package archive into /usr/lib/postgresql.
3. Restart promscale once (and check if the version update message mentioned below is present.
4. Delete /usr/lib/postgresql/timescaledb{,-tsl}-2.1.1.so and restart promscale again to check that it works with just the new version.

Additional info:

* package version(s)

timescaledb 2.2.0-1
promscale 0.3.0-1
promscale_extension 0.1.2-7
postgresql 13.2-2


* config and/or log files etc.

When only timescaledb-2.2.0.so is available:
promscale[578194]: level=error ts=2021-04-28T23:30:36.123Z caller=runner.go:43 msg="aborting startup due to error" err="could not install timescaledb: problem fetching available version: ERROR: could not access file \"$libdir/timescaledb-2.1.1\": No such file or directory (SQLSTATE 58P01)"

Once the old library becomes available, i.e., both timescaledb-2.1.1.so and timescaledb-2.2.0.so exist:
promscale[578213]: level=info ts=2021-04-28T23:30:41.680Z caller=extension.go:258 msg="successfully updated extension" extension_name=timescaledb old_version=2.1.1 new_version=2.2.0

After that^^^ the old library version can be removed; promscale will work fine with the new version (only) after the automatic update.

* link to upstream bug report, if any

N/A; haven't found anything on GitHub (or haven't searched thoroughly enough).

Steps to reproduce:

* Update timescaledb.
* Restart promscale.

I'm not entirely sure what the best solution would be:
(a) a permanent coexistence of the two library version (which would make the PKGBUILD somewhat ugly, but it would work)
(b) something like postgresql-old-upgrade (e.g. timescaledb-old-upgrade) to provide (upon demand) the old library files (only), without file conflicts against the latest version
This task depends upon

Closed by  George Rawlinson (rawlinsong)
Sunday, 04 July 2021, 21:18 GMT
Reason for closing:  Fixed
Comment by Sven-Hendrik Haase (Svenstaro) - Thursday, 29 April 2021, 17:23 GMT
Crap, I didn't expect this somehow but it makes sense! I'm probably going to roll with timescaledb-old-upgrade.
Comment by George Rawlinson (rawlinsong) - Monday, 03 May 2021, 01:40 GMT
The readme for `timescaledb-backup`[0] describes this particular limitation, so upstream is aware that it is a current pain-point.

For option B; I've attached a draft PKGBUILD to this bug, it'll be adjusted eventually.

[0]: https://github.com/timescale/timescaledb-backup
Comment by Sven-Hendrik Haase (Svenstaro) - Friday, 11 June 2021, 22:58 GMT
Yo George, now that you're a full TU, wanna work on this?
Comment by George Rawlinson (rawlinsong) - Friday, 11 June 2021, 23:54 GMT
Yup!

What do you think of changing timescaledb to a split package providing both timescaledb and timescaledb-old-upgrade?

The source can be the git repository, and the versioned shared libraries can be built.

timescaledb: compile with current postgresql; list of past libraries compiled and placed with current postgresql library directory
timescaledb-old-upgrade: list of past versions compiled with postgresql-old-upgrade and placed in postgresql-old-upgrade library directory

Apologies for the crappy message, I’m currently on my phone. D:
Comment by George Rawlinson (rawlinsong) - Friday, 11 June 2021, 23:55 GMT
I’ve got a proof of concept for this; I’ll attach it to this bug report once I get home.
Comment by Andrej Podzimek (andrej) - Saturday, 12 June 2021, 00:11 GMT
> timescaledb-old-upgrade: list of past versions compiled with postgresql-old-upgrade and placed in postgresql-old-upgrade library directory

This^^^ wouldn’t work. It needs to be compiled against the current PostgreSQL in both cases, because timescaledb versions are independent of PostgreSQL versions. The migration (done by promscale automatically) requires that both timescaledb versions be loaded into one and the same (current) PostgreSQL instance at the same time.

Not sure how this should be integrated with postgresql-old-upgrade:

(a) Provide all 4 combinations, which sounds ugly and cumbersome (and I can’t imagine the package naming). :-D

(b) If only there was something like DKMS for PostgreSQL, so that the timescaledb extension’s sources would be installed somewhere and the extension itself would only get compiled upon demand… But I don’t think this exists.

(c) Or there could be a custom system of “upgrade hooks” in postgresql-old-upgrade to rebuild all installed PostgreSQL extensions against legacy PostgreSQL upon demand. But that idea sounds error-prone and requires a DKMS-like installation of source code anyway.
Comment by George Rawlinson (rawlinsong) - Saturday, 12 June 2021, 02:00 GMT
Somewhat relevant: https://docs.timescale.com/timescaledb/latest/how-to-guides/update-timescaledb/

EDIT: If TimescaleDB can operate independently of PostgreSQL versions, is there actually a need for an -old-upgrade package?
Comment by Andrej Podzimek (andrej) - Saturday, 12 June 2021, 02:38 GMT
> If TimescaleDB can operate independently of PostgreSQL versions, is there actually a need for an -old-upgrade package?

Possibly, because for a brief moment (for one start of promscale per timescaledb update), promscale requires two timescaledb versions to coexist at the same time.

An alternative would be to _always_ include the previous version of the extension in the package. It’s basically just two files. That would make things (sort of) seamless.

(There could be an equivalent version for postgresql-old-upgrade, also with two timescaledb versions in it (same ones as in the default postgresql). That way PostgreSQL and timescaledb could be updated independently or both at once and, no matter what, the update would work.)

The PKGBUILD would be somewhat ugly though, because it would need to work with two different versions of timescaledb while building the package (and only take the ~2 relevant files from the old one).
Comment by Sven-Hendrik Haase (Svenstaro) - Sunday, 13 June 2021, 00:51 GMT
George I'll leave this one to you. Do what you consider the best solution.
Comment by George Rawlinson (rawlinsong) - Sunday, 04 July 2021, 21:17 GMT
The most recent release of timescaledb (2.3.0-2) has the last 5 shared libraries for this scenario, seems like upstream does the same for their Debian/Ubuntu packages.

Loading...