FS#46392 - [clang] regression: cannot enable any sanitizer (ASAN, UBsan, etc.)

Attached to Project: Arch Linux
Opened by Peter Wu (Lekensteyn) - Tuesday, 22 September 2015, 09:34 GMT
Last edited by Evangelos Foutras (foutrelis) - Monday, 28 September 2015, 09:56 GMT
Task Type Bug Report
Category Packages: Testing
Status Closed
Assigned To Evangelos Foutras (foutrelis)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:
Since upgrading from clang 3.6 to 3.7, I cannot build programs with ASAN. It misses files:

/usr/bin/ld: cannot find /usr/bin/../lib/clang/3.7.0/lib/linux/libclang_rt.asan-x86_64.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation.
/usr/bin/ld: cannot find /usr/bin/../lib/clang/3.7.0/lib/linux/libclang_rt.msan-x86_64.a: No such file or directory
/usr/bin/ld: cannot find /usr/bin/../lib/clang/3.7.0/lib/linux/libclang_rt.ubsan_standalone-x86_64.a: No such file or directory
/usr/bin/ld: cannot find /usr/bin/../lib/clang/3.7.0/lib/linux/libclang_rt.tsan-x86_64.a: No such file or directory
/usr/bin/ld: cannot find /usr/bin/../lib/clang/3.7.0/lib/linux/libclang_rt.lsan-x86_64.a: No such file or directory
...

This upstream bug suggests that cmake is favored over autotools (https://llvm.org/bugs/show_bug.cgi?id=22757), would that fix the issue?

Additional info:
* package version: clang 3.7.0-1


Steps to reproduce:
echo 'int main(){ return 0;}' > main.c
clang -o /dev/null main.c -fsanitize=address
clang -o /dev/null main.c -fsanitize=undefined
clang -o /dev/null main.c -fsanitize=memory
clang -o /dev/null main.c -fsanitize=thread
clang -o /dev/null main.c -fsanitize=leak
... you get the idea.
This task depends upon

Closed by  Evangelos Foutras (foutrelis)
Monday, 28 September 2015, 09:56 GMT
Reason for closing:  Fixed
Additional comments about closing:  clang 3.7.0-2
Comment by Evangelos Foutras (foutrelis) - Tuesday, 22 September 2015, 17:25 GMT
I tested building LLVM 3.7 with CMake but the resulting packages were huge; this size increase as due to all binaries being statically linked instead of linking to libLLVM. LLVM 3.8 should work better in this regard and we might be able to switch over to CMake at that point.
Comment by Peter Wu (Lekensteyn) - Thursday, 24 September 2015, 16:19 GMT
This issue also affect Debian which uses autotools https://llvm.org/bugs/show_bug.cgi?id=22757.

Upstream is very keen on moving to CMake, even for 3.7:
http://clang-developers.42468.n3.nabble.com/RFC-Dropping-support-for-building-sanitizers-with-autotools-td4043923.html

Please reconsider converting the current setup to cmake, removing sanitizer support is quite a huge regression and would block me from using Clang until it is fixed.

Can this bug be re-opened so that any required work can be tracked?
Comment by Evangelos Foutras (foutrelis) - Thursday, 24 September 2015, 16:24 GMT
Reopening since you did the work to resolve the conflicts that prevented a straight backport of LLVM_LINK_LLVM_DYLIB. (Thanks!)

I'll try your proposed changes (http://sprunge.us/VPMj) in the following couple of days.
Comment by Peter Wu (Lekensteyn) - Thursday, 24 September 2015, 16:53 GMT
As sprunge.us will lose the changes fast, I'll attach the needed changes here. It applies cleanly on the current release_37 branch.
   patch.sh (1.3 KiB)
Comment by Evangelos Foutras (foutrelis) - Thursday, 24 September 2015, 20:01 GMT
My earlier testing indicates that the patch doesn't apply cleanly to the 3.7.0 tag; there were 3 hunks that failed I think.
Comment by Peter Wu (Lekensteyn) - Thursday, 24 September 2015, 21:57 GMT
Here are the patches that were generated from git (5/6 is needed for 6/6). They were created using the cherry-pick commands from that patch.sh.
Comment by Evangelos Foutras (foutrelis) - Friday, 25 September 2015, 17:31 GMT
llvm/clang 3.7.0-2 in [testing] is now built with CMake.
Comment by Peter Wu (Lekensteyn) - Saturday, 26 September 2015, 14:17 GMT
The cmake builds are missing some tools.

These binaries are missing from clang (don't know what they are used for):
/usr/bin/clang-tblgen
/usr/bin/c-index-test

These are missing from clang-tools-extra:
/usr/bin/clang-query (<-- I miss this)
/usr/bin/pp-trace

Discovered this when looking at the curious size differences for clang-tools-extra:
Package (46) Old Version New Version Net Change Download Size
testing/clang 3.7.0-1 3.7.0-2 14.66 MiB 25.15 MiB
testing/clang-analyzer 3.7.0-1 3.7.0-2 0.00 MiB 0.06 MiB
testing/clang-tools-extra 3.7.0-1 3.7.0-2 -24.29 MiB 12.93 MiB
testing/llvm 3.7.0-1 3.7.0-2 -0.04 MiB 16.13 MiB
testing/llvm-libs 3.7.0-1 3.7.0-2 0.71 MiB 9.24 MiB

Do you have build logs available? I just started my build, but it may take a while. My first attempt with options=(debug) and -ggdb used over 16G and failed at 80% due to resource issues. The build process produced a src/llvm-3.7.0.src/build/bin/clang-query binary though.
Comment by Evangelos Foutras (foutrelis) - Saturday, 26 September 2015, 15:04 GMT
Those tools simply don't have install targets. It appears to be intentional, although a follow up post partially disagrees:

http://lists.llvm.org/pipermail/cfe-dev/2015-June/043318.html
http://lists.llvm.org/pipermail/cfe-dev/2015-June/043329.html
Comment by Peter Wu (Lekensteyn) - Saturday, 26 September 2015, 16:36 GMT
clang-query allows you to perform complicated queries on code. Here is an example documenting how to find malloc callers: https://stackoverflow.com/q/29380948/427545

With a bit more effort, you can also match parameters (possibly after expanding macros) and have queries like "find all callers of malloc with a size smaller than X" or "find all callers of proto_tree_add_item which is located under an if(tree) block".

Debian has also included this binary (with the clang package, not clang-tools-extra). There seems enough support/usage of this tool to warrant its inclusion.

echo 'install(TARGETS clang-query RUNTIME DESTINATION bin)' >>
tools/clang/tools/extra/clang-query/tool/CMakeLists.txt
Comment by Evangelos Foutras (foutrelis) - Saturday, 26 September 2015, 17:04 GMT
Debian's packages continue to ship clang-query because they haven't switched to CMake yet.

I'm not against including clang-query but please bring this up with upstream as well.
Comment by Peter Wu (Lekensteyn) - Sunday, 27 September 2015, 10:06 GMT
Missing tools (incl. clang-query) was reported before at
https://llvm.org/bugs/show_bug.cgi?id=24046

Should I create a new bug for the clang-query tool?
The original sanitizer issue is fixed now (so this bug can be marked as fixed).
Comment by Evangelos Foutras (foutrelis) - Sunday, 27 September 2015, 10:36 GMT
Sending a patch (that enables installation of clang-query) might yield more useful results compared to a new bug report. [1]

I'll try to include clang-query in future package releases; hopefully in addition to a fix for lldb which isn't working too well at the moment. [2]

[1] http://llvm.org/docs/DeveloperPolicy.html#making-and-submitting-a-patch
[2] https://llvm.org/bugs/show_bug.cgi?id=24953
Comment by Evangelos Foutras (foutrelis) - Monday, 28 September 2015, 09:55 GMT
clang-tools-extra 3.7.0-3 will include clang-query and I've also posted a diff upstream for consideration. [1]

[1] http://reviews.llvm.org/D13206

Loading...