Using the nvidia binary driver with Xen on Debian etch

Posted by TristanSchmelcher on Fri 26 Jan 2007 at 09:28

Tags: , , ,

I recently set up Xen 3.0.3 on Debian etch using the great guide here from this site. However, if (like me) you use the binary Nvidia X.org driver rebooting into your new XenLinux kernel your X.org server will fail to start. If you read its error output, you will see that this is because it cannot find the nvidia kernel module. Here we'll show how to fix things.

The simple solution is to switch from the proprietary nvidia X.org driver to the nv driver. However, if (like me) you have become emotionally attached to the proprietary nvidia driver, then it is possible to make it work on a Xen kernel. Essentially, what we'll do is build the nvidia kernel module from source, with a patch, and target it to your new XenLinux kernel.

First, on your Xen kernel, as root, do:

apt-get install module-assistant nvidia-kernel-source \
   gcc linux-headers-$(uname -r)

Also, download this patch. Save it in a temporary directory somewhere. I'll call it "tmp". (Replace it with whatever directory you actually use.)

Note: if you don't yet have networking working on your Xen kernel, you can instead do the above under your normal kernel and then switch, but make sure that you substitute "$(uname -r)" with the result that you'd get on your Xen kernel. (e.g., if your normal kernel would say 2.6.18-3-amd64, then your Xen kernel would say 2.6.18-3-xen-amd64.)

After the apt-get, you will have a new file called /usr/src/nvidia-kernel-source.tar.gz. Extract it to tmp:

cd tmp && tar xzf /usr/src/nvidia-kernel-source.tar.gz

This will create a directory called "modules". Within it will be a directory called "nvidia-kernel", and within that will be two, one of them called "nv". Go there:

cd modules/nvidia-kernel/nv

Now use the patch that you downloaded, as follows:

patch -p1 < tmp/patch-nv-1.0-9625-xenrt.txt

You may see some errors about hunks failing. I hope you know how the "patch" command works, because you're going to need to do a bit of manual work. The problem is just due to the fact that the patch file was meant for a different version of the nvidia kernel module. To fix the problem, look at the rejected hunks in the .rej file to see what change the patch command was unable to make. Then look at the file(s) it tried to change, locate the parts that look most similar to the text that the patch command was looking for, and do the change manually yourself. For the version of the nvidia kernel module that etch was using at the time I wrote this (8776), one hunk fails in os-interface.c, but it is fairly easy to locate the spot to change (it's at line 564).

After you're done that, it's time to make the module:

MODULE_LOC=tmp/modules module-assistant --unpack-once build nvidia

And now install it:

module-assistant install nvidia

Now just reboot, and X.org should work with the nvidia driver on your XenLinux kernel.

Note: for the original thread about the patch used here, see this link.

 

 


Posted by wuzzeb (64.5.xx.xx) on Fri 26 Jan 2007 at 18:12
Last time I did this (admitibly on ubuntu dapper on a desktop machine), I needed to export xen_tlb_flush symbol from the kernel. Does the debian xen kernel package include the patch to export this symbol, or is this still a required step?

Otherwise I ran the following script in the nvidia build directory. (For 64bit, will need to be modified for a 32bit system... the colrm part and the ld -m)

for sym in xen_tlb_flush force_evtchn_callback xen_features; do
    export $sym=$(egrep " $sym\$" /boot/System.map-2.6.16.29-xen | colrm 17)
done
ld -m elf_x86_64 --defsym xen_tlb_flush=0x$xen_tlb_flush --defsym force_evtchn_callback=0x$force_evtchn_callback --defsym xen_features=0x$xen_features -r -o nvidia.ko nvidia.o nvidia.mod.o

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (65.95.xx.xx) on Fri 26 Jan 2007 at 19:57
Debian must include a patch for that (at least for the 2.6.18-3-xen-amd64 kernel), because I did not need to do that step.

[ Parent | Reply to this comment ]

Posted by Anonymous (212.214.xx.xx) on Wed 14 Feb 2007 at 16:52
Hmmm...


I get tainted kernel message and rejected module install when I try this. Damn ;-(

I don't get any errors from the module compile as far as I can c but a lot of warning... When I try to load the nvidia module and start X it will fail and say module not present.

Any Hints?

Regards
Thomas

[ Parent | Reply to this comment ]

Posted by Anonymous (70.48.xx.xx) on Wed 14 Feb 2007 at 20:43
Getting lots of warnings during the compile is normal, as is the tainted kernel message. They shouldn't be preventing the module from loading. Does the pre-built module work in your non-Xen kernel? And did you have any problems patching the source? Also, please post the exact error messages you get when loading the module and starting X, and also the output from the following commands when running in Xen:

uname -r
ls -al /usr/src
ls -al /lib/modules/$(uname -r)/nvidia
ls -al /dev/nvidia*
dmesg

Also, are you using Debian etch (testing), or something else?

[ Parent | Reply to this comment ]

Posted by Anonymous (84.166.xx.xx) on Sat 14 Apr 2007 at 20:14
Can't find linux-headers-2.6.16.33-xen anywhere...

Any thoughts?



Rob

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (65.95.xx.xx) on Sat 14 Apr 2007 at 21:31
I take it that your problem is at the "apt-get install" step? The kernel version in the package name would be 2.6.16-33 if anything, not 2.6.16.33, but I'll assume that that's just a typo in your message. In any event, I don't see a headers package with either name in the Debian repositories, nor even in snapshot.debian.net. Where exactly did you get your Xen kernel? Is it custom-compiled or something? If it's not from the Debian package repositories, you won't be able to get the right headers with apt-get. If you don't have them and don't know where to get them, then I'd recommend upgrading to a pre-built kernel from the Debian package repositories. That way you can then just download _its_ headers package.

[ Parent | Reply to this comment ]

Posted by Anonymous (84.166.xx.xx) on Mon 16 Apr 2007 at 19:19
Yeah, that's the step. I downloaded xen-3.0.4_1-install-x86_32.tgz (the most recent) from their website. It installed vmlinux-2.6.16.33-xen (really. Not a typo.) in /boot along with symlinks named vmlinux-2.6.16-xen and vmlinux-2.6-xen that both point to the original file vmlinux-2.6.16.33-xen.
uname -r gives me 2.6.16.33-xen.
I did apt-get linux-headers-2.6.16-2 with no immediate success.
Do you think creating a symlink would work here? Or should I just remove the version I've got and get the xen kernel from the Debian package repositories?

Thanks.


Rob

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (65.95.xx.xx) on Mon 16 Apr 2007 at 19:45
Ahh, I see, that explains it. Yeah, you could try a symlink from /usr/src/linux-headers-2.6.16.33-xen to /usr/src/linux-headers-2.6.16-2 (if you've installed it from, like, the snapshot archives or something). But installing a Xen kernel from the Debian repositories is quite painless, so I'd recommend that, personally.

[ Parent | Reply to this comment ]

Posted by Anonymous (84.166.xx.xx) on Mon 16 Apr 2007 at 20:23
Thanks again. I'll let ya know how it works out.



Rob

[ Parent | Reply to this comment ]

Posted by Anonymous (84.250.xx.xx) on Sat 5 May 2007 at 11:45
It doesn't work. :( Errors with building, x.org wont start.

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (154.20.xx.xx) on Sat 5 May 2007 at 19:57
If you give me the details of the errors, I can help you work through a solution. Post the errors that you get when building, and also the outputs of these commands:

uname -r
ls -al /usr/src
ls -al /lib/modules/$(uname -r)/nvidia
ls -al /dev/nvidia*
dmesg

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (154.20.xx.xx) on Sun 6 May 2007 at 03:20
For users that are finding the patching process to be too complex, I have prepared an alternate patch file that can be used instead and will not require you to manually edit any files. Get it here:

http://www.nvnews.net/vbulletin/attachment.php?attachmentid=25476 &d=1178418866

If you use that file instead of the one linked to in my original article, then you can completely ignore the paragraph that starts out with "You may see some errors about hunks failing ...".

(Note though that if you are using the current Debian unstable, then you should NOT use this file, because Debian unstable is using a different NVIDIA driver version and the patch file given in the article is the right one to use.)

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (154.20.xx.xx) on Thu 10 May 2007 at 04:32
Actually, my last remark in that comment is incorrect. If you are using unstable then you do not need a patch at all. Just run "apt-get install module-assistant && module-assistant auto-install nvidia" from Xen and it will set everything up.

[ Parent | Reply to this comment ]

Posted by Anonymous (195.68.xx.xx) on Wed 16 May 2007 at 08:07
hi,

OS : Debian etch
kernel : 2.6.18-4-xen-686
Video card : 01:00.0 VGA compatible controller: nVidia Corporation NV43 [GeForce 6600/GeForce 6600 GT] (rev a2)

I have tried to apply the patch as described and it is applied sucessfully. But when I reboot the
computer I still get the empty black screen. X looks started but no display, here is what I get on /var/log/messages :

May 16 08:55:19 tatooine kernel: NVRM: loading NVIDIA Linux x86 Kernel Module 1.0-8776 Mon Oct 16 21:56:04 PDT 2006
May 16 08:55:19 tatooine kernel: Modules linked in: nvidia ppdev lp button ac battery ipv6 bridge loop snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_emu10k1 bt878 snd_seq_dummy snd_seq_oss snd_seq_midi tuner tvaudio snd_seq_midi_event bttv video_buf firmware_class snd_seq ir_common snd_bt87x snd_pcm_oss snd_mixer_oss compat_ioctl32 i2c_algo_bit btcx_risc tveeprom snd_rawmidi snd_ac97_codec snd_ac97_bus i2c_core emu10k1_gp snd_seq_device videodev snd_util_mem v4l1_compat snd_pcm snd_timer parport_pc snd_hwdep snd soundcore snd_page_alloc v4l2_common gameport parport serial_core rtc floppy serio_raw intel_agp shpchp pci_hotplug psmouse pcspkr agpgart tsdev evdev ext3 jbd mbcache dm_mirror dm_snapshot dm_mod ide_generic usbhid ide_cd cdrom ide_disk ehci_hcd ohci_hcd e100 mii uhci_hcd usbcore piix generic ide_core thermal processor fan
May 16 08:55:19 tatooine kernel: EIP: 0061:[<eed24114>] Tainted: P VLI
May 16 08:55:19 tatooine kernel: EFLAGS: 00010082 (2.6.18-4-xen-686 #1)
May 16 08:55:19 tatooine kernel: [<eed24235>] __nv_enable_pat_support+0xa9/0x1a9 [nvidia]
May 16 08:55:19 tatooine kernel: [<ee29f62a>] nvidia_init_module+0x62a/0x75c [nvidia]
May 16 08:55:19 tatooine kernel: [<c01337a1>] sys_init_module+0x16c3/0x1846
May 16 08:55:19 tatooine kernel: [<c0104883>] syscall_call+0x7/0xb
May 16 08:55:21 tatooine kernel: <4>printk: 143017 messages suppressed.
May 16 08:55:21 tatooine kernel: 4gb seg fixup, process dhcdbd (pid 2933), cs:ip 73:b7dd4620
May 16 08:55:24 tatooine kernel: agpgart: Found an AGP 2.0 compliant device at 0000:00:00.0.
May 16 08:55:24 tatooine kernel: agpgart: Putting AGP V2 device at 0000:00:00.0 into 4x mode
May 16 08:55:24 tatooine kernel: agpgart: Putting AGP V2 device at 0000:01:00.0 into 4x mode
May 16 08:55:24 tatooine kernel: NVRM: Trying to sleep during raised irql!!
May 16 08:55:24 tatooine kernel: NVRM: are we holding a lock?
May 16 08:55:24 tatooine kernel: NVRM: skipping os_delay

Since I am not really familar with the patch command I have used your alternate patch file at : http://www.nvnews.net/vbulletin/attachment.php?attachmentid=25476 &d=1178418866

tatooine:/usr/src/modules/nvidia-kernel/nv# patch -p1 < /root/tmp/patch-nv-1.0-8776-xenrt.txt
patching file nv.c
patching file nv-linux.h
patching file nv-vm.c
patching file os-agp.c
patching file os-interface.c
tatooine:/usr/src/modules/nvidia-kernel/nv# cd ..
tatooine:/usr/src/modules/nvidia-kernel# cd ..
tatooine:/usr/src/modules# cd ..
tatooine:/usr/src# MODULE_LOC=/usr/src/modules module-assistant --unpack-once build nvidia
Extracting the package tarball, /usr/src/nvidia-kernel-source.tar.gz, please wait...
Done with /usr/src/nvidia-kernel-2.6.18-4-xen-686_1.0.8776-4+2.6.18.dfsg.1- 12etch2_i386.deb .

and then :

tatooine:/usr/src# module-assistant install nvidia
Selecting previously deselected package nvidia-kernel-2.6.18-4-xen-686.
(Reading database ... 85718 files and directories currently installed.)
Unpacking nvidia-kernel-2.6.18-4-xen-686 (from .../nvidia-kernel-2.6.18-4-xen-686_1.0.8776-4+2.6.18.dfsg.1-12etc h2_i386.deb) ...
Setting up nvidia-kernel-2.6.18-4-xen-686 (1.0.8776-4+2.6.18.dfsg.1-12etch2) ...

tatooine:/usr/src# reboot

Here is my device section is xorg.conf:

Section "Device"
Identifier "nVidia Corporation NV43 [GeForce 6600]"
Driver "nvidia"
BusID "PCI:1:0:0"
Option "UseFBDev" "true"
EndSection

thanks in advance

[ Parent | Reply to this comment ]

Posted by Anonymous (195.68.xx.xx) on Wed 16 May 2007 at 08:37
it is working now, I had tried to patch in /usr/src/modules/nvidia-kernel directoty but this directory had been used before for another patch.

So i have uncompressed nvidia-kernel-source.tar.gz again in a clean directory and now X is working with a xen-kernel !!!!

Many thanks for your patch

bye

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (154.20.xx.xx) on Thu 17 May 2007 at 05:26
lol, whoops, I didn't notice your follow-up and wrote a reply anyways. :) Glad to hear you got it working.

[ Parent | Reply to this comment ]

Posted by TristanSchmelcher (154.20.xx.xx) on Thu 17 May 2007 at 05:23
Hmm, that is rather peculiar. It looks like you followed the guide correctly, but from the contents of your /var/log/messages it seems that the nvidia driver is attempting to use a thing called PAT, which the patch disables because it doesn't work on Xen. That suggests that somehow the patch didn't get used in the end.

The problem might be partly due to the fact that you extracted the nvidia-kernel-source.tar.gz file to /usr/src. That's the same place that module-assistant itself normally extracts it to on its own when using other module-assistant commands. So if you didn't run the commands exactly as you've posted above (e.g., if you ran another module-assistant command between the "patch -p1 ..." and "MODULE_LOC=... " lines), then you might have accidentally made module-assistant overwrite the patched files with the originals.

So my first recommendation would be to retry the procedure using a different directory for "tmp". Say, /root/tmp. You will have to pass "-f" to both module-assistant commands in order to force them to redo the process. So the commands to run would be:

cd /root/tmp
tar xzf /usr/src/nvidia-kernel-source.tar.gz
cd modules/nvidia-kernel/nv
patch -p1 < /root/tmp/patch-nv-1.0-8776-xenrt.txt
MODULE_LOC=/root/tmp/modules module-assistant --unpack-once -f build nvidia
module-assistant -f install nvidia

If that doesn't fix things, then send me an email with the new contents of /var/log/messages, and also attach the files nv.c, nv-linux.h, nv-vm.c, os-agp.c, and os-interface.c from /root/tmp/modules/nvidia-kernel/nv. (My email address is in my user profile.)

[ Parent | Reply to this comment ]

Posted by Anonymous (89.0.xx.xx) on Thu 30 Aug 2007 at 15:37
worth to mention the 100.14.11 nvidia driver can be built against Xen kernels without patches if the IGNORE_XEN_PRESENCE environment variable is set to a non-zero value. Tested on a amd_64 Athlon Etch setup with nvidia installer [dom0]. thread at nvnews : http://www.nvnews.net/vbulletin/showthread.php?t=95483

[ Parent | Reply to this comment ]

Posted by Anonymous (24.128.xx.xx) on Wed 26 Sep 2007 at 18:38
Even better the current (as of today) driver NVIDIA-Linux-x86_64-100.14.19 builds the Kernel module after:

# cd /usr/src
# export IGNORE_XEN_PRESENCE=1
# wget http://us.download.nvidia.com/XFree86/Linux-x86_64/100.14.19/NVID IA-Linux-x86_64-100.14.19-pkg2.run
# sh NVIDIA-Linux-x86_64-100.14.19-pkg2.run

Awesome! Simple and functional. All on a "2.6.18-xen #11 SMP Thu Sep 20 14:35:59 EDT 2007 x86_64" running Xen 3.1 custom build.

[ Parent | Reply to this comment ]

Posted by Anonymous (82.243.xx.xx) on Fri 19 Oct 2007 at 18:37
This the procedure that use for installing nvidia driver in debian etch. It's not professionnal but it's works.

apt-get install module-assistant nvidia-kernel-source gcc linux-headers-$(uname -r)

cd /tmp && tar xzf /usr/src/nvidia-kernel-source.tar.gz

cd /root

Download this version of driver nvidia NVIDIA-Linux-x86-100.14.11-pkg1.run

Download a patch (http://www.nvnews.net/vbulletin/showthread.php?t=95483) for this driver in folder /root

sh ./NVIDIA-Linux-x86-100.14.11-pkg1.run --extract-only

cd /root/NVIDIA-Linux-x86-100.14.11-pkg1/usr/src/nv

patch -p1 < /root/NVIDIA_xen-9746.patch.gz

cp -r /root/NVIDIA-Linux-x86-100.14.11-pkg1/usr/src/nv/* /tmp/modules/nvidia-kernel/nv

MODULE_LOC=/tmp/modules module-assistant --unpack-once build nvidia

module-assistant install nvidia

reboot et ca fonctionne.

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 258 votes ~ 1 comments )