FS#50520 - [glibc] glibc compilation fails when fortify is not in default flags

Attached to Project: Arch Linux
Opened by Tim Savannah (kata198) - Friday, 26 August 2016, 04:43 GMT
Last edited by Allan McRae (Allan) - Sunday, 09 April 2017, 04:33 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Allan McRae (Allan)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

glibc compilation fails when fortify is not in default flags, and likely also requires advanced CPU featuresets to be enabled (like in -march=native on my i5-3230M, including specifically ssse3 and avx).

The issue is that "__chk_fail" method does not get defined when the fortify options are not present in cflags.

This causes problems because of this commit:

commit c365e615f7429aee302f8af7bf07ae262278febb
Author: H.J. Lu <hjl.tools@gmail.com>
Date: Mon Mar 28 13:13:36 2016 -0700

Implement x86-64 multiarch mempcpy in memcpy

Implement x86-64 multiarch mempcpy in memcpy to share most of code. It
reduces code size of libc.so.

[BZ #18858]
* sysdeps/x86_64/multiarch/Makefile (sysdep_routines): Remove
mempcpy-ssse3, mempcpy-ssse3-back, mempcpy-avx-unaligned
and mempcpy-avx512-no-vzeroupper.
* sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S (MEMPCPY_CHK):
New.
(MEMPCPY): Likewise.
* sysdeps/x86_64/multiarch/memcpy-avx512-no-vzeroupper.S
(MEMPCPY_CHK): New.
(MEMPCPY): Likewise.
* sysdeps/x86_64/multiarch/memcpy-ssse3-back.S (MEMPCPY_CHK): New.
(MEMPCPY): Likewise.
* sysdeps/x86_64/multiarch/memcpy-ssse3.S (MEMPCPY_CHK): New.
(MEMPCPY): Likewise.
* sysdeps/x86_64/multiarch/mempcpy-avx-unaligned.S: Removed.
* sysdeps/x86_64/multiarch/mempcpy-avx512-no-vzeroupper.S:
Likewise.
* sysdeps/x86_64/multiarch/mempcpy-ssse3-back.S: Likewise.
* sysdeps/x86_64/multiarch/mempcpy-ssse3.S: Likewise.


Steps to reproduce:

Use an i5 or other modern processor with the following CFLAGS/CXXFLAGS in /etc/makepkg.conf:

CFLAGS="-mtune=native -march=native -O3 -pipe"
CXXFLAGS="$CFLAGS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro"


and run "makepkg".

You will get errors like this:

gcc ../sysdeps/x86_64/multiarch/memcpy-ssse3.S -c -U_FORTIFY_SOURCE -I../include -I/usr/src/arch/glibc/src/glibc-build/string -I/usr/src/arch/glibc/src/glibc-build -I../sysdeps/unix/sysv/linux/x86_64/64 -I../sysdeps/unix/sysv/linux/x86_64 -I../sysdeps/unix/sysv/linux/x86 -I../sysdeps/unix/sysv/linux/wordsize-64 -I../sysdeps/x86_64/nptl -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux -I../sysdeps/nptl -I../sysdeps/pthread -I../sysdeps/gnu -I../sysdeps/unix/inet -I../sysdeps/unix/sysv -I../sysdeps/unix/x86_64 -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/x86_64/64 -I../sysdeps/x86_64/fpu/multiarch -I../sysdeps/x86_64/fpu -I../sysdeps/x86/fpu/include -I../sysdeps/x86/fpu -I../sysdeps/x86_64/multiarch -I../sysdeps/x86_64 -I../sysdeps/x86 -I../sysdeps/ieee754/ldbl-96 -I../sysdeps/ieee754/dbl-64/wordsize-64 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/ieee754/flt-32 -I../sysdeps/wordsize-64 -I../sysdeps/ieee754 -I../sysdeps/generic -I.. -I../libio -I. -nostdinc -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/include -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/include-fixed -isystem /usr/include -D_LIBC_REENTRANT -include /usr/src/arch/glibc/src/glibc-build/libc-modules.h -DMODULE_NAME=libc -include ../include/libc-symbols.h -DPIC -DASSEMBLER -Werror=undef -Wa,--noexecstack -o /usr/src/arch/glibc/src/glibc-build/string/memcpy-ssse3.o -MD -MP -MF /usr/src/arch/glibc/src/glibc-build/string/memcpy-ssse3.o.dt -MT /usr/src/arch/glibc/src/glibc-build/string/memcpy-ssse3.o
gcc ../sysdeps/x86_64/multiarch/memmove-ssse3.S -c -U_FORTIFY_SOURCE -I../include -I/usr/src/arch/glibc/src/glibc-build/string -I/usr/src/arch/glibc/src/glibc-build -I../sysdeps/unix/sysv/linux/x86_64/64 -I../sysdeps/unix/sysv/linux/x86_64 -I../sysdeps/unix/sysv/linux/x86 -I../sysdeps/unix/sysv/linux/wordsize-64 -I../sysdeps/x86_64/nptl -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux -I../sysdeps/nptl -I../sysdeps/pthread -I../sysdeps/gnu -I../sysdeps/unix/inet -I../sysdeps/unix/sysv -I../sysdeps/unix/x86_64 -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/x86_64/64 -I../sysdeps/x86_64/fpu/multiarch -I../sysdeps/x86_64/fpu -I../sysdeps/x86/fpu/include -I../sysdeps/x86/fpu -I../sysdeps/x86_64/multiarch -I../sysdeps/x86_64 -I../sysdeps/x86 -I../sysdeps/ieee754/ldbl-96 -I../sysdeps/ieee754/dbl-64/wordsize-64 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/ieee754/flt-32 -I../sysdeps/wordsize-64 -I../sysdeps/ieee754 -I../sysdeps/generic -I.. -I../libio -I. -nostdinc -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/include -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/include-fixed -isystem /usr/include -D_LIBC_REENTRANT -include /usr/src/arch/glibc/src/glibc-build/libc-modules.h -DMODULE_NAME=libc -include ../include/libc-symbols.h -DPIC -DASSEMBLER -Werror=undef -Wa,--noexecstack -o /usr/src/arch/glibc/src/glibc-build/string/memmove-ssse3.o -MD -MP -MF /usr/src/arch/glibc/src/glibc-build/string/memmove-ssse3.o.dt -MT /usr/src/arch/glibc/src/glibc-build/string/memmove-ssse3.o
../sysdeps/x86_64/multiarch/memcpy-ssse3.S: Assembler messages:
../sysdeps/x86_64/multiarch/memcpy-ssse3.S:65: Error: operand type mismatch for `jb'


On 3 files. This is likely a bug for glibc itself, but sometimes they take a long time so maybe we can cover it here, or at least leave this open and thus search-able for others to find the solution.

I've attached a patch (also duplicated below) which effectively replaces the call to the missing fortify function with a "nop" (NO-OP) instruction. This allows compilation to complete under these conditions.

Maybe we can add a comment to the official PKGBUILD like:

# Uncomment the following patch if you have ssse3/avx enabled on your processor and in CFLAGS, and fortify is disabled
# cat ${startdir}/fix-compile-without-fortify-and-ssse3_or_avx.patch | patch -p1

Here is the patch (also in attachment):

diff --git a/sysdeps/x86_64/multiarch/memcpy-ssse3-back.S b/sysdeps/x86_64/multiarch/memcpy-ssse3-back.S
index b4890f4..44d2f98 100644
--- a/sysdeps/x86_64/multiarch/memcpy-ssse3-back.S
+++ b/sysdeps/x86_64/multiarch/memcpy-ssse3-back.S
@@ -48,8 +48,10 @@
.section .text.ssse3,"ax",@progbits
#if !defined USE_AS_MEMPCPY && !defined USE_AS_MEMMOVE
ENTRY (MEMPCPY_CHK)
- cmpq %rdx, %rcx
+/* cmpq %rdx, %rcx
jb HIDDEN_JUMPTARGET (__chk_fail)
+*/
+ nop
END (MEMPCPY_CHK)

ENTRY (MEMPCPY)
@@ -61,8 +63,10 @@ END (MEMPCPY)

#if !defined USE_AS_BCOPY
ENTRY (MEMCPY_CHK)
- cmpq %rdx, %rcx
+/* cmpq %rdx, %rcx
jb HIDDEN_JUMPTARGET (__chk_fail)
+*/
+ nop
END (MEMCPY_CHK)
#endif

diff --git a/sysdeps/x86_64/multiarch/memcpy-ssse3.S b/sysdeps/x86_64/multiarch/memcpy-ssse3.S
index 1ca88c0..1e6183d 100644
--- a/sysdeps/x86_64/multiarch/memcpy-ssse3.S
+++ b/sysdeps/x86_64/multiarch/memcpy-ssse3.S
@@ -48,8 +48,10 @@
.section .text.ssse3,"ax",@progbits
#if !defined USE_AS_MEMPCPY && !defined USE_AS_MEMMOVE
ENTRY (MEMPCPY_CHK)
- cmpq %rdx, %rcx
+/* cmpq %rdx, %rcx
jb HIDDEN_JUMPTARGET (__chk_fail)
+*/
+ nop
END (MEMPCPY_CHK)

ENTRY (MEMPCPY)
@@ -61,8 +63,11 @@ END (MEMPCPY)

#if !defined USE_AS_BCOPY
ENTRY (MEMCPY_CHK)
+/*
cmpq %rdx, %rcx
jb HIDDEN_JUMPTARGET (__chk_fail)
+*/
+ nop
END (MEMCPY_CHK)
#endif

diff --git a/sysdeps/x86_64/multiarch/memset-avx512-no-vzeroupper.S b/sysdeps/x86_64/multiarch/memset-avx512-no-vzeroupper.S
index 9687df0..79afd42 100644
--- a/sysdeps/x86_64/multiarch/memset-avx512-no-vzeroupper.S
+++ b/sysdeps/x86_64/multiarch/memset-avx512-no-vzeroupper.S
@@ -29,8 +29,10 @@
.section .text.avx512,"ax",@progbits
#if defined PIC
ENTRY (MEMSET_CHK)
- cmpq %rdx, %rcx
+/* cmpq %rdx, %rcx
jb HIDDEN_JUMPTARGET (__chk_fail)
+*/
+ nop
END (MEMSET_CHK)
#endif




Additional info:
* package version(s)
* config and/or log files etc.


Steps to reproduce:
This task depends upon

Closed by  Allan McRae (Allan)
Sunday, 09 April 2017, 04:33 GMT
Reason for closing:  Upstream
Comment by Tim Savannah (kata198) - Friday, 26 August 2016, 04:48 GMT
Note this patch could probably be written as an additional conditional, checking _FORTIFY_SOURCE , which is undefined (via -U_FORTIFY_SOURCE) with my CFLAGS.

Likely when I submit this upstream to the glibc folks, they may prefer an implementation like that. Thing is, the archlinux PKGBUILD undefines them and redefines the fortify flags at a later point, which should swap -U_FORTIFY_SOURCE with -D_FORTIFY_SOURCE. I'm not sure how this would interact with the checking of that flag, probably not at all, but it would take a long time to do multiple full compiles and test, whereas I have already tested the given patch.
Comment by Tim Savannah (kata198) - Friday, 26 August 2016, 04:54 GMT
For reference, here is the implementation using "ifdef" that I will likely submit upstream. I have tested that it allows compilation to complete on those files, but I have NOT tested it as extensive as the original patch, and whether it is affected by our PKGBUILD's swapping in/out of the fortify CFLAGS.
Comment by Tim Savannah (kata198) - Friday, 26 August 2016, 05:02 GMT
I have submitted upstream here using the second patch (the one posted in comments): https://sourceware.org/bugzilla/show_bug.cgi?id=20518

Comment by Jon Gjengset (Jonhoo) - Friday, 16 December 2016, 14:42 GMT
  • Field changed: Percent Complete (100% → 0%)
This has been marked as WORKSFORME upstream, but the build still fails on Arch (without kata198's patch).

Loading...