cool, compile it with debug symbols and get proper debug traces.

https://wiki.archlinux.org/index.php/Debug_-_Getting_Traces#One_package_settings_only

do that for libva and vdpau-video

Is this backtrace enough ?

(gdb) bt
#0 0x0000000000005316 in ?? ()
#1 0x00007ffff587d4f5 in floorf () from /lib/libm.so.6
#2 0x00007ffff7de807c in _dl_relocate_object ()
from /lib/ld-linux-x86-64.so.2
#3 0x00007ffff7dee546 in dl_open_worker () from /lib/ld-linux-x86-64.so.2
#4 0x00007ffff7dea2a6 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#5 0x00007ffff7dede8a in _dl_open () from /lib/ld-linux-x86-64.so.2
#6 0x00007ffff689af26 in ?? () from /lib/libdl.so.2
#7 0x00007ffff7dea2a6 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#8 0x00007ffff689b4cf in ?? () from /lib/libdl.so.2
#9 0x00007ffff689afc1 in dlopen () from /lib/libdl.so.2
#10 0x00007ffff79c00d0 in ?? () from /usr/lib/libva.so.1
#11 0x00007ffff79c0b9b in vaInitialize () from /usr/lib/libva.so.1
#12 0x00000000004015d5 in ?? ()
#13 0x00007ffff70e238d in __libc_start_main () from /lib/libc.so.6
#14 0x0000000000402a2d in ?? ()
#15 0x00007fffffffdaf8 in ?? ()
#16 0x00000000ffffffff in ?? ()
#17 0x0000000000000003 in ?? ()
#18 0x00007fffffffdf29 in ?? ()
---Type <return> to continue, or q <return> to quit---
#19 0x00007fffffffdf3d in ?? ()
#20 0x00007fffffffdf40 in ?? ()
#21 0x0000000000000000 in ?? ()

use this package: http://pkgbuild.com/~ioni/libva/ and try again

abique@lotus:~> gdb /usr/bin/h264encode
Reading symbols from /usr/bin/h264encode...done.
(gdb) r -o toto.mp4
Starting program: /usr/bin/h264encode -o toto.mp4
[Thread debugging using libthread_db enabled]
libva: VA-API version 0.32.0
Xlib: extension "XFree86-DRI" missing on display ":0.0".
libva: va_getDriverName() returns 0
libva: Trying to open /usr/lib/dri/nvidia_drv_video.so

Program received signal SIGSEGV, Segmentation fault.
0x0000000000005316 in ?? ()
(gdb) bt
#0 0x0000000000005316 in ?? ()
#1 0x00007ffff58724f5 in floorf () from /lib/libm.so.6
#2 0x00007ffff7de807c in _dl_relocate_object ()
from /lib/ld-linux-x86-64.so.2
#3 0x00007ffff7dee546 in dl_open_worker () from /lib/ld-linux-x86-64.so.2
#4 0x00007ffff7dea2a6 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#5 0x00007ffff7dede8a in _dl_open () from /lib/ld-linux-x86-64.so.2
#6 0x00007ffff6895f26 in ?? () from /lib/libdl.so.2
#7 0x00007ffff7dea2a6 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#8 0x00007ffff68964cf in ?? () from /lib/libdl.so.2
#9 0x00007ffff6895fc1 in dlopen () from /lib/libdl.so.2
#10 0x00007ffff79bb2ff in va_openDriver (dpy=0x681680,
driver_name=0x681e50 "nvidia") at va.c:194
#11 0x00007ffff79bc01a in vaInitialize (dpy=0x681680,
major_version=0x7fffffffda0c, minor_version=0x7fffffffda10) at va.c:397
#12 0x0000000000402e04 in main (argc=3, argv=0x7fffffffdb08)
at h264encode_common.c:361
(gdb)

I've the very same problem, vlc says "No accelerated IMDCT transform found", but ffmpeg (downgraded to 20111211-1, the latest cannot play videos at all) says avcodec is using VA API for hw decoding o_O

# vainfo

Starting program: /usr/bin/vainfo
[Thread debugging using libthread_db enabled]
libva: VA-API version 0.32.0
Xlib: extension "XFree86-DRI" missing on display ":0.0".
libva: va_getDriverName() returns 0
libva: Trying to open /usr/lib/dri/nvidia_drv_video.so

Program received signal SIGSEGV, Segmentation fault.
0x0000000000005316 in ?? ()
(gdb) bt
#0 0x0000000000005316 in ?? ()
#1 0x00007ffff58784f5 in floorf () from /lib/libm.so.6
#2 0x00007ffff7de807c in _dl_relocate_object () from /lib/ld-linux-x86-64.so.2
#3 0x00007ffff7dee546 in dl_open_worker () from /lib/ld-linux-x86-64.so.2
#4 0x00007ffff7dea2a6 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#5 0x00007ffff7dede8a in _dl_open () from /lib/ld-linux-x86-64.so.2
#6 0x00007ffff70d5f26 in ?? () from /lib/libdl.so.2
#7 0x00007ffff7dea2a6 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#8 0x00007ffff70d64cf in ?? () from /lib/libdl.so.2
#9 0x00007ffff70d5fc1 in dlopen () from /lib/libdl.so.2
#10 0x00007ffff7bc52ff in va_openDriver (dpy=0x60f720, driver_name=0x60fef0 "nvidia") at va.c:194
#11 0x00007ffff7bc601a in vaInitialize (dpy=0x60f720, major_version=0x7fffffffdf88, minor_version=0x7fffffffdf8c) at va.c:397
#12 0x0000000000400b80 in main (argc=1, argv=0x7fffffffe088) at vainfo.c:116

@wonder: could you share your PKGBUILD for the debug version of your libva/vdpau? i'm unable to have gdb print "driver_name" debug stuff, it looks like it still optimizes the build and want to learn how you did that, i followed the wiki but it just optimize out the values.. (EDIT: can get gdb to print stuff with -O0, not -O1 but that's not quite good since code paths could be very different)

post with -O0. Also what video card do you have?

The results i posted above were produced with the library compiled with -O0: i've three nVidia cards, a GT220, a GT430 and a 9400GT, all the machines present the same problem and the same stack trace.

Should probably be assigned to glibc maint. s_floorf.S might need to go as well.

Does it work with http://dev.archlinux.org/~allan/glibc-2.15-3.1-x86_64.pkg.tar.xz ?

@einar, you are correct, that glibc package did it, no more segfaults, thank you!

Just an update, now VLC keeps segfaulting trying to use libva:

$ vlc
VLC media player 1.1.13 The Luggage (revision exported)
Blocked: call to unsetenv("DBUS_ACTIVATION_ADDRESS")
Blocked: call to unsetenv("DBUS_ACTIVATION_BUS_TYPE")
[0x1969120] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Blocked: call to setlocale(6, "")
Blocked: call to setlocale(6, "")

(process:12234): Gtk-WARNING **: Locale not supported by C library.
Using the fallback 'C' locale.
m_el[mi_level] == NULL
arrrrrrrrrrrrrg Up cannot escape itself
m_el[mi_level] == NULL
arrrrrrrrrrrrrg Up cannot escape itself
libva: VA-API version 0.32.0
Xlib: extension "XFree86-DRI" missing on display ":0.0".
libva: va_getDriverName() returns 0
libva: Trying to open /usr/lib/dri/nvidia_drv_video.so
libva: va_openDriver() returns 0
Segmentation fault

what ffmpeg version do you have?

pacman -Q ffmpeg

I was about to post it along with vlc:


Name : ffmpeg
Version : 20120111-1

Name : vlc
Version : 1.1.13-1

Mmm.. tried to downgrade ffmpeg to 20111211-1, 20111123-1 and 20111108-1, all segfault.
20111003-1 just says "no suitable decoder module for fourcc `h264`"..
This seems all pretty odd since ffplay from the ffmpeg-20120111-1 package just works fine o_O

EDIT: tried to downgrade vlc to 1.1.12-5/1.1.12-1 with no luck, all segfaults (1.1.12-1 output the same no decoder error as ffmpeg-20111003-1)

Here is a backtrace with a debug version (-O0 and --enable-debug) of vlc-1.1.13-1

#0 0x00007ffff718fece in _int_malloc () from /lib/libc.so.6
#1 0x00007ffff7192345 in malloc () from /lib/libc.so.6
#2 0x00007fffc717bf11 in CopyInitCache (cache=0x7fffc00013d0, width=1280) at copy.c:296
#3 0x00007fffc717acfd in CreateSurfaces (p_va=0x7fffc00012b0, pp_hw_ctx=0x7fffccc513c8, pi_chroma=0x7fffccc557b8, i_width=1280, i_height=534) at vaapi.c:313
#4 0x00007fffc717aeef in Setup (p_external=0x7fffc00012b0, pp_hw_ctx=0x7fffccc513c8, pi_chroma=0x7fffccc557b8, i_width=1280, i_height=534) at vaapi.c:353
#5 0x00007fffc7175620 in vlc_va_Setup (va=0x7fffc00012b0, hw=0x7fffccc513c8, output=0x7fffccc557b8, width=1280, height=534) at va.h:42
#6 0x00007fffc7177c4c in ffmpeg_GetFormat (p_codec=0x7fffccc50fe0, pi_fmt=0x7fffc64d32b0) at video.c:1213
#7 0x00007fffc616fcb0 in ?? () from /usr/lib/libavcodec.so.53
#8 0x00007fffc617333f in ?? () from /usr/lib/libavcodec.so.53
#9 0x00007fffc6173f5d in ?? () from /usr/lib/libavcodec.so.53
#10 0x00007fffc635cc62 in avcodec_decode_video2 () from /usr/lib/libavcodec.so.53
#11 0x00007fffc71766cd in DecodeVideo (p_dec=0x7fffccc555a8, pp_block=0x7fffc42b7da0) at video.c:555
#12 0x00007ffff78ea8c3 in DecoderDecodeVideo (p_dec=0x7fffccc555a8, p_block=0x7fffccc1f710) at input/decoder.c:1466
#13 0x00007ffff78eb488 in DecoderProcessVideo (p_dec=0x7fffccc555a8, p_block=0x7fffccc1f710, b_flush=false) at input/decoder.c:1835
#14 0x00007ffff78eba6a in DecoderProcess (p_dec=0x7fffccc555a8, p_block=0x7fffccc1f710) at input/decoder.c:2007
#15 0x00007ffff78e927d in DecoderThread (p_this=0x7fffccc555a8) at input/decoder.c:892
#16 0x00007ffff794e39f in thread_entry (data=0x7fffccc07b20) at misc/threads.c:58
#17 0x00007ffff7688e7a in start_thread () from /lib/libpthread.so.0
#18 0x00007ffff71efdbd in clone () from /lib/libc.so.6
#19 0x0000000000000000 in ?? ()

That backtrace is entirely unrelated to the original report.