FS#68903 - [tracker3] testsuite breaks with sqlite 3.34.0
Attached to Project:
Arch Linux
Opened by Jan Alexander Steffens (heftig) - Wednesday, 09 December 2020, 10:25 GMT
Last edited by Jan Alexander Steffens (heftig) - Saturday, 12 December 2020, 12:52 GMT
Opened by Jan Alexander Steffens (heftig) - Wednesday, 09 December 2020, 10:25 GMT
Last edited by Jan Alexander Steffens (heftig) - Saturday, 12 December 2020, 12:52 GMT
|
Details
I tried to package tracker3 3.0.2; tracker-fd-test from the
test suite fails with:
Tracker-CRITICAL **: 02:39:41.823: SQLite error: database disk image is malformed (errno: Success) Apparently sqlite3_step returns SQLITE_CORRUPT. When building with sqlite 3.33.0-2, the test suite passes. To enable debug output, add `--buildtype debug` to arch-meson and run the tests with `TRACKER_DEBUG=sqlite,sql-statements`. This shows the last statement executed: Tracker-Message: 10:22:32.985: Executing update: 'INSERT INTO "main".fts5 (fts5, rowid , "nco:phoneNumber", "nfo:fontFamily", "nmm:artistName", "nfo:tableOfContents", "nfo:fileName", "nfo:genre", "nmm:genre", "nco:emailAddress", "nie:keyword", "nmm:category", "nco:nameAdditional", "nco:nameFamily", "nco:nameGiven", "nco:contactGroupName", "nco:fullname", "nco:nickname", "nco:country", "nco:county", "nco:district", "nco:extendedAddress", "nco:locality", "nco:pobox", "nco:postalcode", "nco:region", "nco:streetAddress", "nco:imID", "nco:imNickname", "nie:comment", "nie:description", "nie:plainTextContent", "nie:subject", "nie:title", "nao:description", "nao:prefLabel", "nco:department", "nco:role", "nco:title", "nco:note") SELECT 'delete', rowid , "nco:phoneNumber", "nfo:fontFamily", "nmm:artistName", "nfo:tableOfContents", "nfo:fileName", "nfo:genre", "nmm:genre", "nco:emailAddress", "nie:keyword", "nmm:category", "nco:nameAdditional", "nco:nameFamily", "nco:nameGiven", "nco:contactGroupName", "nco:fullname", "nco:nickname", "nco:country", "nco:county", "nco:district", "nco:extendedAddress", "nco:locality", "nco:pobox", "nco:postalcode", "nco:region", "nco:streetAddress", "nco:imID", "nco:imNickname", "nie:comment", "nie:description", "nie:plainTextContent", "nie:subject", "nie:title", "nao:description", "nao:prefLabel", "nco:department", "nco:role", "nco:title", "nco:note" FROM "main".fts_view WHERE rowid = 100002' |
This task depends upon
Closed by Jan Alexander Steffens (heftig)
Saturday, 12 December 2020, 12:52 GMT
Reason for closing: Fixed
Additional comments about closing: tracker3 3.0.2-1
Saturday, 12 December 2020, 12:52 GMT
Reason for closing: Fixed
Additional comments about closing: tracker3 3.0.2-1
https://www.sqlite.org/src/timeline?advm=0&udc=1&y=t or https://www.sqlite.org/src/timeline?udc=1&ss=j&n=100&y=ci&advm=0
https://www.sqlite.org/src/timeline?udc=1&ss=j&n=100&y=ci&advm=0
The complete list of check-ins going from 3.33.0 to 3.34.0 can be seen here: https://www.sqlite.org/src/timeline?from=version-3.33.0&to=version-3.34.0&y=ci
If you are uncomfortable using the canonical Fossil repository for SQLite, you can pull the sources from the GitHub mirror: https://github.com/sqlite/sqlite
For each version of SQLite that you pull to test, you can build the "sqlite3.c" and "sqlite3.h" source files by running "./configure && make sqlite3.c".
86f477edaa17767b39c7bae5b67cac8580f7a8c1 is the first bad commit https://www.sqlite.org/src/info/b79f19edfd33c2a7
commit 86f477edaa17767b39c7bae5b67cac8580f7a8c1
Author: dan <dan@noemail.net>
Date: Fri Sep 11 15:01:49 2020 +0000
Catch fts5 index corruption caused by issuing 'delete' commands with incorrect data earlier in some cases. Also fix a couple of test script problems.
FossilOrigin-Name: b79f19edfd33c2a75f936c352668e14e81f35acf4f07edc27a21f941a7304b38
I guess the problem is that inserting a 'delete' command with all possible columns but where values that weren't actually inserted are NULL isn't handled correctly?
Dan Kennedy.
I can't seem to run the test suite locally. Following the instructions here:
https://gitlab.gnome.org/GNOME/tracker
has the "meson ./build --prefix=$HOME/opt/tracker -Dtracker_core=subproject" line fail with:
The Meson build system
Version: 0.45.1
Source dir: /home/dan/tmp/tracker3/tracker-miners
Build dir: /home/dan/tmp/tracker3/tracker-miners/build
Build type: native build
meson.build:126:18: ERROR: Expecting eol got id.
if optimization in ['0', 'g']
^
A full log can be found at /home/dan/tmp/tracker3/tracker-miners/build/meson-logs/meson-log.txt
You'll need to get an updated version of meson. Actually I'm a bit surprised you could even configure this as I thought meson should raise an error saying you need a new version of meson...
Note that meson can be run directly from the source tree as meson.py, or installed via pip
To ease things a bit, I've left a dump of all SQL queries issued in that test at https://people.gnome.org/~carlosg/tracker-fd-test-sql-commands.txt, with some minor editions so it doesn't rely on functions, tokenizers, collation, virtual tables and other code defined by Tracker.
That file does still reproduce the issue with `cat tracker-fd-test-sql-commands.txt | sqlite3` here.
If I grep the tracker-fd-test-sql-commands.txt for "fts5", I can see the table being created, the rank function being set, one row being inserted and then two separate 'delete' commands - neither of which look to match the insert. So the index is corrupt. Prior to 3.34.0, this was silently ignored, or reported on when the index was queried. But now the SQLITE_CORRUPT is returned for the second delete.
Dan.
I FTR think this was silently ignored. Tracker had in the past other FTS table consistency errors, and I remember FTS5 integrity-check command coming up clean after fixing.