FS#80126 - [squid] coredumps after update to squid 6.4-1

Attached to Project: Arch Linux
Opened by BAD+MAD (mat_weiss) - Wednesday, 01 November 2023, 07:15 GMT
Last edited by Toolybird (Toolybird) - Friday, 03 November 2023, 05:26 GMT
Task Type Bug Report
Category Upstream Bugs
Status Closed
Assigned To Sergej Pupykin (sergej)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Get many coredumps after update to squid 6.4-1.
No coredumps after downgrade to squid 6.3-1.


squid[945]: FATAL: assertion failed: stmem.cc:98: "lowestOffset () <= target_offset" current master transaction: master148
systemd-coredump[981]: [🡕] Process 945 (squid) of user 15 dumped core.

Stack trace of thread 945:
#0 0x00007f793908783c n/a (libc.so.6 + 0x8e83c)
#1 0x00007f7939037668 raise (libc.so.6 + 0x3e668)
#2 0x00007f793901f4b8 abort (libc.so.6 + 0x264b8)
#3 0x00005650da6aa0fe xassert (squid + 0x4e30fe)
#4 0x00005650da4a3de5 _ZN7mem_hdr12freeDataUptoEl (squid + 0x2dcde5)
#5 0x00005650da4af14e _ZN10StoreEntry10trimMemoryEb (squid + 0x2e814e)
#6 0x00005650da4c125c _ZN10StoreEntry7swapOutEv (squid + 0x2fa25c)
#7 0x00005650da4b6c96 _ZN10StoreEntry14invokeHandlersEv (squid + 0x2efc96)
#8 0x00005650da4a7e57 _ZN10StoreEntry5writeE13StoreIOBuffer (squid + 0x2e0e57)
#9 0x00005650da535a86 _ZN6Client30handleMoreAdaptedBodyAvailableEv (squid + 0x3>
#10 0x00005650da369caf n/a (squid + 0x1a2caf)
#11 0x00005650da58f491 _ZN14AsyncCallQueue4fireEv (squid + 0x3c8491)
#12 0x00005650da3785ac _ZN9EventLoop7runOnceEv (squid + 0x1b15ac)
#13 0x00005650da47ad09 _Z9SquidMainiPPc (squid + 0x2b3d09)
#14 0x00005650da31e647 main (squid + 0x157647)
#15 0x00007f7939020cd0 n/a (libc.so.6 + 0x27cd0)
#16 0x00007f7939020d8a __libc_start_main (libc.so.6 + 0x27d8a)
#17 0x00005650da323b75 _start (squid + 0x15cb75)
ELF object binary architecture: AMD x86-64

squid[944]: FATAL: assertion failed: Controller.cc:940: "TheRoot"

systemd-coredump[1163]: [🡕] Process 944 (squid) of user 15 dumped core.

Stack trace of thread 944:
#0 0x00007ff993fbf83c n/a (libc.so.6 + 0x8e83c)
#1 0x00007ff993f6f668 raise (libc.so.6 + 0x3e668)
#2 0x00007ff993f574b8 abort (libc.so.6 + 0x264b8)
#3 0x000055935ccda0fe xassert (squid + 0x4e30fe)
#4 0x000055935cd60645 _ZN5Store4RootEv (squid + 0x569645)
#5 0x000055935cad921a _ZN10StoreEntry16destroyMemObjectEv (squid + 0x2e221a)
#6 0x000055935cad94c9 _Z17destroyStoreEntryPv (squid + 0x2e24c9)
#7 0x000055935cd696c2 hashFreeItems (squid + 0x5726c2)
#8 0x000055935cd63399 _ZN5Store10ControllerD1Ev (squid + 0x56c399)
#9 0x000055935cd6378b n/a (squid + 0x56c78b)
#10 0x00007ff993f71cc6 n/a (libc.so.6 + 0x40cc6)
#11 0x00007ff993f71e10 exit (libc.so.6 + 0x40e10)
#12 0x000055935caaae94 _Z9SquidMainiPPc (squid + 0x2b3e94)
#13 0x000055935c94e647 main (squid + 0x157647)
#14 0x00007ff993f58cd0 n/a (libc.so.6 + 0x27cd0)
#15 0x00007ff993f58d8a __libc_start_main (libc.so.6 + 0x27d8a)
#16 0x000055935c953b75 _start (squid + 0x15cb75)
ELF object binary architecture: AMD x86-64



I found upstream:

squid-users@lists.squid-cache.org&q=subject:%22%5C%5Bsquid%5C-users%5C%5D+%5C%5Bext%5C%5D+Re%5C%3A+Squid+6.4+assertion+errors%5C%3A+FATAL%5C%3A+assertion+failed%5C%3A+stmem.cc%5C%3A98%5C%3A+%5C%22lowestOffset+%5C%28%5C%29+%3C%3D+target_offset%5C%22+current+master+transaction%5C%3A+master655+%5C%28backtrace%5C%29%5C%5D%22&o=newest"> https://www.mail-archive.com/search?l=squid-users@lists.squid-cache.org&q=subject:%22%5C%5Bsquid%5C-users%5C%5D+%5C%5Bext%5C%5D+Re%5C%3A+Squid+6.4+assertion+errors%5C%3A+FATAL%5C%3A+assertion+failed%5C%3A+stmem.cc%5C%3A98%5C%3A+%5C%22lowestOffset+%5C%28%5C%29+%3C%3D+target_offset%5C%22+current+master+transaction%5C%3A+master655+%5C%28backtrace%5C%29%5C%5D%22&o=newest
This task depends upon

Closed by  Toolybird (Toolybird)
Friday, 03 November 2023, 05:26 GMT
Reason for closing:  Fixed
Additional comments about closing:  squid 6.4-3
Comment by BAD+MAD (mat_weiss) - Wednesday, 01 November 2023, 07:20 GMT Comment by BAD+MAD (mat_weiss) - Wednesday, 01 November 2023, 07:27 GMT Comment by BAD+MAD (mat_weiss) - Wednesday, 01 November 2023, 07:40 GMT Comment by Sergej Pupykin (sergej) - Wednesday, 01 November 2023, 21:18 GMT
please try squid-6.4-2
Comment by BAD+MAD (mat_weiss) - Thursday, 02 November 2023, 06:30 GMT
the update to version 6.4-2 did not solve the problem, I switch back to version 6.3-1.

pacman -S squid

Warnung: squid-6.4-2 ist aktuell -- Reinstalliere
Abhängigkeiten werden aufgelöst …
Nach in Konflikt stehenden Paketen wird gesucht …

Pakete (1) squid-6.4-2

Gesamtgröße der installierten Pakete: 8,53 MiB
Größendifferenz der Aktualisierung: 0,00 MiB

:: Installation fortsetzen? [J/n]
(1/1) Schlüssel im Schlüsselbund werden geprüft [#######################################################] 100%
(1/1) Paket-Integrität wird überprüft [#######################################################] 100%
(1/1) Paket-Dateien werden geladen [#######################################################] 100%
(1/1) Auf Dateikonflikte wird geprüft [#######################################################] 100%
(1/1) Verfügbarer Festplattenspeicher wird ermittelt [#######################################################] 100%
:: Paketänderungen werden verarbeitet …
(1/1) Reinstalliert wird squid [#######################################################] 100%
:: Post-transaction-Hooks werden gestartet …
(1/4) Creating system user accounts...
(2/4) Reloading system manager configuration...
(3/4) Creating temporary files...
(4/4) Arming ConditionNeedsUpdate...

------------------------------------------------------------------------------

systemctl restart squid

------------------------------------------------------------------------------

squid -v

Squid Cache: Version 6.4
Service Name: squid

This binary uses OpenSSL 3.1.4 24 Oct 2023. configure options: '--prefix=/usr' '--sbindir=/usr/bin' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--localstatedir=/var' '--with-logdir=/var/log/squid' '--with-pidfile=/run/squid.pid' '--enable-auth' '--enable-auth-basic' '--enable-auth-ntlm' '--enable-auth-digest' '--enable-auth-negotiate' '--enable-removal-policies=lru,heap' '--enable-storeio=aufs,ufs,diskd,rock' '--enable-delay-pools' '--with-openssl' '--enable-snmp' '--enable-linux-netfilter' '--enable-ident-lookups' '--enable-useragent-log' '--enable-cache-digests' '--enable-referer-log' '--enable-htcp' '--enable-carp' '--enable-epoll' '--with-large-files' '--enable-arp-acl' '--with-default-user=proxy' '--enable-async-io' '--enable-truncate' '--enable-icap-client' '--enable-ssl-crtd' '--disable-arch-native' '--disable-strict-error-checking' '--enable-wccpv2' 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/squid/src=/usr/src/debug/squid -flto=auto' 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto' 'CXXFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -g -ffile-prefix-map=/build/squid/src=/usr/src/debug/squid -flto=auto'

------------------------------------------------------------------------------

squid[151738]: FATAL: assertion failed: stmem.cc:98: "lowestOffset () <= target_offset" current master transaction: master500

systemd-coredump[151755]: [🡕] Process 151738 (squid) of user 15 dumped core.

Stack trace of thread 151738:
#0 0x00007fa7475bf83c n/a (libc.so.6 + 0x8e83c)
#1 0x00007fa74756f668 raise (libc.so.6 + 0x3e668)
#2 0x00007fa7475574b8 abort (libc.so.6 + 0x264b8)
#3 0x000055ef8c25d13e xassert (squid + 0x4e313e)
#4 0x000055ef8c056f05 _ZN7mem_hdr12freeDataUptoEl (squid + 0x2dcf05)
#5 0x000055ef8c06226e _ZN10StoreEntry10trimMemoryEb (squid + 0x2e826e)
#6 0x000055ef8c07451c _ZN10StoreEntry7swapOutEv (squid + 0x2fa51c)
#7 0x000055ef8c069f56 _ZN10StoreEntry14invokeHandlersEv (squid + 0x2eff56)
#8 0x000055ef8c05af77 _ZN10StoreEntry5writeE13StoreIOBuffer (squid + 0x2e0f77)
#9 0x000055ef8c0e8ce6 _ZN6Client30handleMoreAdaptedBodyAvailableEv (squid + 0x36ece6)
#10 0x000055ef8bf1ceef n/a (squid + 0x1a2eef)
#11 0x000055ef8c142491 _ZN14AsyncCallQueue4fireEv (squid + 0x3c8491)
#12 0x000055ef8bf2b7ec _ZN9EventLoop7runOnceEv (squid + 0x1b17ec)
#13 0x000055ef8c02df29 _Z9SquidMainiPPc (squid + 0x2b3f29)
#14 0x000055ef8bed1647 main (squid + 0x157647)
#15 0x00007fa747558cd0 n/a (libc.so.6 + 0x27cd0)
#16 0x00007fa747558d8a __libc_start_main (libc.so.6 + 0x27d8a)
#17 0x000055ef8bed6b75 _start (squid + 0x15cb75)
ELF object binary architecture: AMD x86-64

squid[151759]: Starting Squid Cache version 6.4 for x86_64-pc-linux-gnu...
squid[151759]: Service Name: squid

------------------------------------------------------------------------------
Comment by BAD+MAD (mat_weiss) - Thursday, 02 November 2023, 09:33 GMT Comment by loqs (loqs) - Thursday, 02 November 2023, 12:09 GMT Comment by BAD+MAD (mat_weiss) - Thursday, 02 November 2023, 12:18 GMT Comment by loqs (loqs) - Thursday, 02 November 2023, 13:11 GMT
That is why I was suggesting you added it and tested if that resolves the issue.
Comment by BAD+MAD (mat_weiss) - Thursday, 02 November 2023, 13:19 GMT
I'm not the maintainer. And I don't have any abs tools installed on the productive systems.
Comment by BAD+MAD (mat_weiss) - Thursday, 02 November 2023, 13:31 GMT
I'll try it later on an testsystem.
Comment by loqs (loqs) - Thursday, 02 November 2023, 13:34 GMT Comment by BAD+MAD (mat_weiss) - Thursday, 02 November 2023, 13:52 GMT
No coredumps since 15 min after patching with https://github.com/squid-cache/squid/commit/c11ee3d0812e7041f395073cf3b2f368f1caf26d.patch

A) OK on the testsystem by building my own package
B) OK on the productive system by using your package squid-6.4-2.1-x86_64.pkg.tar.zst

Thanks
Comment by Sergej Pupykin (sergej) - Thursday, 02 November 2023, 22:49 GMT Comment by BAD+MAD (mat_weiss) - Friday, 03 November 2023, 05:24 GMT
Problem solved with the squid-6.4-3 package.