FS#28882 - [xorg] 1.12.0-1 segfault as VMware guest

Attached to Project: Arch Linux
Opened by Wil (nullptr) - Tuesday, 13 March 2012, 01:32 GMT
Last edited by Andreas Radke (AndyRTR) - Thursday, 15 March 2012, 06:14 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Andreas Radke (AndyRTR)
Architecture x86_64
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

When running as a VMware guest, Xorg 1.12.0-1 segfaults during module loading. It looks like it might be due to vmware_drv.so loading libvgahw.so as a dependency. In any case, it currently isn't possible to run Arch with X as a VMware guest, which is a showstopper for me (and I imagine some others).

It looks like someone else reported something similar last month at https://bbs.archlinux.org/viewtopic.php?pid=1061728#p1061728, despite the thread title.

I've attached a log of the crash.
This task depends upon

Closed by  Andreas Radke (AndyRTR)
Thursday, 15 March 2012, 06:14 GMT
Reason for closing:  Fixed
Additional comments about closing:  see comments and make sure you load the module!
Comment by Wil (nullptr) - Tuesday, 13 March 2012, 02:24 GMT
Perhaps this is a slightly more helpful backtrace. It was gathered using gdb directly on Xorg from an ssh session.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) i thr
Id Target Id Frame
* 1 Thread 0x7ffff7fc9880 (LWP 1295) "Xorg" 0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00007ffff3bfdea1 in vgaHWSaveColormap () from /usr/lib/xorg/modules/libvgahw.so
#2 0x00007ffff3bff7ad in vgaHWSave () from /usr/lib/xorg/modules/libvgahw.so
#3 0x00007ffff4415043 in ?? () from /usr/lib/xorg/modules/drivers/vmware_drv.so
#4 0x00000000004735cc in InitOutput ()
#5 0x0000000000422c0d in ?? ()
#6 0x00007ffff61df38d in __libc_start_main () from /lib/libc.so.6
#7 0x00000000004230cd in _start ()
(gdb)
Comment by Wil (nullptr) - Tuesday, 13 March 2012, 02:29 GMT
And, here's a core dump from that session.
Comment by Wil (nullptr) - Tuesday, 13 March 2012, 02:59 GMT
I did a bit of analysis of the core dump. It looks like vgaHWSaveColormap is passed a vgaHWPtr that has some NULL function pointers, including (at least) writeDacMask, writeDacReadAddr, and readDacData. The crash happens when vgaHWSaveColormap tries to call hwp->writeDacMask at hw/xfree86/vgahw/vgaHW.c:1075 in the xorg-server source distribution.

It looks like vmware_drv.so doesn't have debugging symbols available, but from looking at src/vmware.c in the source distribution for xf86-video-vmware, it seems pretty likely that vgaHWSave is getting called from VMWARESave, which is getting called from VMWAREPreInit. I'm not enough of an X expert to know where pScrn (which is later cast to the vgaHWPtr) should be picking up those missing function pointers (maybe when the vgahw module is loaded?), but hopefully this gives someone somewhere something to go on.
Comment by Jessehk (Jessehk) - Tuesday, 13 March 2012, 03:46 GMT
Same thing here. I've also attached my log. I tried running with the new svga-dri package for 3D support and also without it (uninstalled). Seg fault in both cases. Just a note: I was able to get X11 working again by specifying the "vesa" driver xorg.conf instead of the vmware driver. Performance is awful, though.
Comment by Lee.MaRS (leemars) - Tuesday, 13 March 2012, 05:11 GMT
Same thing here. Is it better to rollback to previous version?
Comment by Robert David (robjdavid) - Tuesday, 13 March 2012, 11:19 GMT
I'm seeing the same thing here, in my case running under KVM. I've tried using the vmware (supported under KVM as "vmvga") and the cirrus drivers, with the same result.

Rolling back to the previous version (via ARM) fixes it for me.
Comment by Jelle van der Waa (jelly) - Tuesday, 13 March 2012, 11:46 GMT
As always report bugs that are not caused by packages UPSTREAM,. https://bugs.freedesktop.org/

@Wil, arch packages don't have debug symbols you could rebuild these packages and report your information.. upstream. https://bugs.freedesktop.org/

@jessehk, vesa has always been slow.
Comment by Wil (nullptr) - Tuesday, 13 March 2012, 13:38 GMT
@jelly, while I appreciate your point of view, the end result of the package upgrade is that X is catastrophically broken for multiple Arch users. Isn't there value in having that documented here instead of having people simply wonder what the hell is going on when X suddenly no longer comes up after restarting?

In any case, thanks for the pointer to freedesktop's bugtracker. I will report what I've found there.
Comment by Jelle van der Waa (jelly) - Tuesday, 13 March 2012, 15:44 GMT
The devs probably didn't know about the bug, since the bug has been reported when the driver went to [extra], so no one in [testing] sadly used it.
Comment by Wil (nullptr) - Tuesday, 13 March 2012, 16:02 GMT
Looks like that is in fact the case. For those that are interested, the upstream report is at https://bugs.freedesktop.org/show_bug.cgi?id=47280.

In the meantime, I can echo Robert's comment that rolling back via ARM works. The versions you want are:

xorg-server 1.11.4-1
xorg-server-common 1.9.2-2
xf86-input-vmmouse 12.7.0-3
xf86-input-evdev 2.6.0-4
xf86-input-keyboard 1.6.1-1
xf86-input-mouse 1.7.1-2
xf86-video-vmware 11.1.0-1
Comment by Frederik Vos (inktvis75) - Wednesday, 14 March 2012, 15:49 GMT
I have the same problem, still not able to fix it, but so far i can see now:
- you'll need svga-dri installed and vmwgfx in /etc/rc.d MODULES
- you'll need 32mb video memory
Comment by Andreas Radke (AndyRTR) - Wednesday, 14 March 2012, 17:32 GMT
Please test new version 12.0.1 with 3D support.
Comment by Robert David (robjdavid) - Wednesday, 14 March 2012, 22:53 GMT
Tested against latest version of xf86-video-vmware, version 12.0.1-1. Looks like it made no obvious difference. I've attached a core dump file and the Xorg log file.
Comment by Jessehk (Jessehk) - Thursday, 15 March 2012, 02:05 GMT
It works! Robert, you need to load the vmwgfx module in rc.conf by adding it to MODULES as Frederik said.
Anybody else?
Comment by Wil (nullptr) - Thursday, 15 March 2012, 03:51 GMT
Pulling the latest packages from testing and making sure that vmwgfx is loaded seems to work here as well.
Comment by Lee.MaRS (leemars) - Thursday, 15 March 2012, 03:56 GMT
It also works here, but it's too slow and with high cpu usage. Does it still have problem?

Loading...