Using the nvidia binary driver with Xen on Debian etch
Posted by TristanSchmelcher on Fri 26 Jan 2007 at 09:28
I recently set up Xen 3.0.3 on Debian etch using the great guide here from this site. However, if (like me) you use the binary Nvidia X.org driver rebooting into your new XenLinux kernel your X.org server will fail to start. If you read its error output, you will see that this is because it cannot find the nvidia kernel module. Here we'll show how to fix things.
The simple solution is to switch from the proprietary nvidia X.org driver to the nv driver. However, if (like me) you have become emotionally attached to the proprietary nvidia driver, then it is possible to make it work on a Xen kernel. Essentially, what we'll do is build the nvidia kernel module from source, with a patch, and target it to your new XenLinux kernel.
First, on your Xen kernel, as root, do:
apt-get install module-assistant nvidia-kernel-source \ gcc linux-headers-$(uname -r)
Also, download this patch. Save it in a temporary directory somewhere. I'll call it "tmp". (Replace it with whatever directory you actually use.)
Note: if you don't yet have networking working on your Xen kernel, you can instead do the above under your normal kernel and then switch, but make sure that you substitute "$(uname -r)" with the result that you'd get on your Xen kernel. (e.g., if your normal kernel would say 2.6.18-3-amd64, then your Xen kernel would say 2.6.18-3-xen-amd64.)
After the apt-get, you will have a new file called /usr/src/nvidia-kernel-source.tar.gz. Extract it to tmp:
cd tmp && tar xzf /usr/src/nvidia-kernel-source.tar.gz
This will create a directory called "modules". Within it will be a directory called "nvidia-kernel", and within that will be two, one of them called "nv". Go there:
cd modules/nvidia-kernel/nv
Now use the patch that you downloaded, as follows:
patch -p1 < tmp/patch-nv-1.0-9625-xenrt.txt
You may see some errors about hunks failing. I hope you know how the "patch" command works, because you're going to need to do a bit of manual work. The problem is just due to the fact that the patch file was meant for a different version of the nvidia kernel module. To fix the problem, look at the rejected hunks in the .rej file to see what change the patch command was unable to make. Then look at the file(s) it tried to change, locate the parts that look most similar to the text that the patch command was looking for, and do the change manually yourself. For the version of the nvidia kernel module that etch was using at the time I wrote this (8776), one hunk fails in os-interface.c, but it is fairly easy to locate the spot to change (it's at line 564).
After you're done that, it's time to make the module:
MODULE_LOC=tmp/modules module-assistant --unpack-once build nvidia
And now install it:
module-assistant install nvidia
Now just reboot, and X.org should work with the nvidia driver on your XenLinux kernel.
Note: for the original thread about the patch used here, see this link.
Otherwise I ran the following script in the nvidia build directory. (For 64bit, will need to be modified for a 32bit system... the colrm part and the ld -m)
for sym in xen_tlb_flush force_evtchn_callback xen_features; do
export $sym=$(egrep " $sym\$" /boot/System.map-2.6.16.29-xen | colrm 17)
done
ld -m elf_x86_64 --defsym xen_tlb_flush=0x$xen_tlb_flush --defsym force_evtchn_callback=0x$force_evtchn_callback --defsym xen_features=0x$xen_features -r -o nvidia.ko nvidia.o nvidia.mod.o
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I get tainted kernel message and rejected module install when I try this. Damn ;-(
I don't get any errors from the module compile as far as I can c but a lot of warning... When I try to load the nvidia module and start X it will fail and say module not present.
Any Hints?
Regards
Thomas
[ Parent | Reply to this comment ]
uname -r
ls -al /usr/src
ls -al /lib/modules/$(uname -r)/nvidia
ls -al /dev/nvidia*
dmesg
Also, are you using Debian etch (testing), or something else?
[ Parent | Reply to this comment ]
Any thoughts?
Rob
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
uname -r gives me 2.6.16.33-xen.
I did apt-get linux-headers-2.6.16-2 with no immediate success.
Do you think creating a symlink would work here? Or should I just remove the version I've got and get the xen kernel from the Debian package repositories?
Thanks.
Rob
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Rob
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
uname -r
ls -al /usr/src
ls -al /lib/modules/$(uname -r)/nvidia
ls -al /dev/nvidia*
dmesg
[ Parent | Reply to this comment ]
http://www.nvnews.net/vbulletin/attachment.php?attachmentid=25476 &d=1178418866
If you use that file instead of the one linked to in my original article, then you can completely ignore the paragraph that starts out with "You may see some errors about hunks failing ...".
(Note though that if you are using the current Debian unstable, then you should NOT use this file, because Debian unstable is using a different NVIDIA driver version and the patch file given in the article is the right one to use.)
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
OS : Debian etch
kernel : 2.6.18-4-xen-686
Video card : 01:00.0 VGA compatible controller: nVidia Corporation NV43 [GeForce 6600/GeForce 6600 GT] (rev a2)
I have tried to apply the patch as described and it is applied sucessfully. But when I reboot the
computer I still get the empty black screen. X looks started but no display, here is what I get on /var/log/messages :
May 16 08:55:19 tatooine kernel: NVRM: loading NVIDIA Linux x86 Kernel Module 1.0-8776 Mon Oct 16 21:56:04 PDT 2006
May 16 08:55:19 tatooine kernel: Modules linked in: nvidia ppdev lp button ac battery ipv6 bridge loop snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_emu10k1 bt878 snd_seq_dummy snd_seq_oss snd_seq_midi tuner tvaudio snd_seq_midi_event bttv video_buf firmware_class snd_seq ir_common snd_bt87x snd_pcm_oss snd_mixer_oss compat_ioctl32 i2c_algo_bit btcx_risc tveeprom snd_rawmidi snd_ac97_codec snd_ac97_bus i2c_core emu10k1_gp snd_seq_device videodev snd_util_mem v4l1_compat snd_pcm snd_timer parport_pc snd_hwdep snd soundcore snd_page_alloc v4l2_common gameport parport serial_core rtc floppy serio_raw intel_agp shpchp pci_hotplug psmouse pcspkr agpgart tsdev evdev ext3 jbd mbcache dm_mirror dm_snapshot dm_mod ide_generic usbhid ide_cd cdrom ide_disk ehci_hcd ohci_hcd e100 mii uhci_hcd usbcore piix generic ide_core thermal processor fan
May 16 08:55:19 tatooine kernel: EIP: 0061:[<eed24114>] Tainted: P VLI
May 16 08:55:19 tatooine kernel: EFLAGS: 00010082 (2.6.18-4-xen-686 #1)
May 16 08:55:19 tatooine kernel: [<eed24235>] __nv_enable_pat_support+0xa9/0x1a9 [nvidia]
May 16 08:55:19 tatooine kernel: [<ee29f62a>] nvidia_init_module+0x62a/0x75c [nvidia]
May 16 08:55:19 tatooine kernel: [<c01337a1>] sys_init_module+0x16c3/0x1846
May 16 08:55:19 tatooine kernel: [<c0104883>] syscall_call+0x7/0xb
May 16 08:55:21 tatooine kernel: <4>printk: 143017 messages suppressed.
May 16 08:55:21 tatooine kernel: 4gb seg fixup, process dhcdbd (pid 2933), cs:ip 73:b7dd4620
May 16 08:55:24 tatooine kernel: agpgart: Found an AGP 2.0 compliant device at 0000:00:00.0.
May 16 08:55:24 tatooine kernel: agpgart: Putting AGP V2 device at 0000:00:00.0 into 4x mode
May 16 08:55:24 tatooine kernel: agpgart: Putting AGP V2 device at 0000:01:00.0 into 4x mode
May 16 08:55:24 tatooine kernel: NVRM: Trying to sleep during raised irql!!
May 16 08:55:24 tatooine kernel: NVRM: are we holding a lock?
May 16 08:55:24 tatooine kernel: NVRM: skipping os_delay
Since I am not really familar with the patch command I have used your alternate patch file at : http://www.nvnews.net/vbulletin/attachment.php?attachmentid=25476 &d=1178418866
tatooine:/usr/src/modules/nvidia-kernel/nv# patch -p1 < /root/tmp/patch-nv-1.0-8776-xenrt.txt
patching file nv.c
patching file nv-linux.h
patching file nv-vm.c
patching file os-agp.c
patching file os-interface.c
tatooine:/usr/src/modules/nvidia-kernel/nv# cd ..
tatooine:/usr/src/modules/nvidia-kernel# cd ..
tatooine:/usr/src/modules# cd ..
tatooine:/usr/src# MODULE_LOC=/usr/src/modules module-assistant --unpack-once build nvidia
Extracting the package tarball, /usr/src/nvidia-kernel-source.tar.gz, please wait...
Done with /usr/src/nvidia-kernel-2.6.18-4-xen-686_1.0.8776-4+2.6.18.dfsg.1- 12etch2_i386.deb .
and then :
tatooine:/usr/src# module-assistant install nvidia
Selecting previously deselected package nvidia-kernel-2.6.18-4-xen-686.
(Reading database ... 85718 files and directories currently installed.)
Unpacking nvidia-kernel-2.6.18-4-xen-686 (from .../nvidia-kernel-2.6.18-4-xen-686_1.0.8776-4+2.6.18.dfsg.1-12etc h2_i386.deb) ...
Setting up nvidia-kernel-2.6.18-4-xen-686 (1.0.8776-4+2.6.18.dfsg.1-12etch2) ...
tatooine:/usr/src# reboot
Here is my device section is xorg.conf:
Section "Device"
Identifier "nVidia Corporation NV43 [GeForce 6600]"
Driver "nvidia"
BusID "PCI:1:0:0"
Option "UseFBDev" "true"
EndSection
thanks in advance
[ Parent | Reply to this comment ]
So i have uncompressed nvidia-kernel-source.tar.gz again in a clean directory and now X is working with a xen-kernel !!!!
Many thanks for your patch
bye
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
The problem might be partly due to the fact that you extracted the nvidia-kernel-source.tar.gz file to /usr/src. That's the same place that module-assistant itself normally extracts it to on its own when using other module-assistant commands. So if you didn't run the commands exactly as you've posted above (e.g., if you ran another module-assistant command between the "patch -p1 ..." and "MODULE_LOC=... " lines), then you might have accidentally made module-assistant overwrite the patched files with the originals.
So my first recommendation would be to retry the procedure using a different directory for "tmp". Say, /root/tmp. You will have to pass "-f" to both module-assistant commands in order to force them to redo the process. So the commands to run would be:
cd /root/tmp
tar xzf /usr/src/nvidia-kernel-source.tar.gz
cd modules/nvidia-kernel/nv
patch -p1 < /root/tmp/patch-nv-1.0-8776-xenrt.txt
MODULE_LOC=/root/tmp/modules module-assistant --unpack-once -f build nvidia
module-assistant -f install nvidia
If that doesn't fix things, then send me an email with the new contents of /var/log/messages, and also attach the files nv.c, nv-linux.h, nv-vm.c, os-agp.c, and os-interface.c from /root/tmp/modules/nvidia-kernel/nv. (My email address is in my user profile.)
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
# cd /usr/src
# export IGNORE_XEN_PRESENCE=1
# wget http://us.download.nvidia.com/XFree86/Linux-x86_64/100.14.19/NVID IA-Linux-x86_64-100.14.19-pkg2.run
# sh NVIDIA-Linux-x86_64-100.14.19-pkg2.run
Awesome! Simple and functional. All on a "2.6.18-xen #11 SMP Thu Sep 20 14:35:59 EDT 2007 x86_64" running Xen 3.1 custom build.
[ Parent | Reply to this comment ]
apt-get install module-assistant nvidia-kernel-source gcc linux-headers-$(uname -r)
cd /tmp && tar xzf /usr/src/nvidia-kernel-source.tar.gz
cd /root
Download this version of driver nvidia NVIDIA-Linux-x86-100.14.11-pkg1.run
Download a patch (http://www.nvnews.net/vbulletin/showthread.php?t=95483) for this driver in folder /root
sh ./NVIDIA-Linux-x86-100.14.11-pkg1.run --extract-only
cd /root/NVIDIA-Linux-x86-100.14.11-pkg1/usr/src/nv
patch -p1 < /root/NVIDIA_xen-9746.patch.gz
cp -r /root/NVIDIA-Linux-x86-100.14.11-pkg1/usr/src/nv/* /tmp/modules/nvidia-kernel/nv
MODULE_LOC=/tmp/modules module-assistant --unpack-once build nvidia
module-assistant install nvidia
reboot et ca fonctionne.
[ Parent | Reply to this comment ]