Rocks 7 Dracut-Initque Timeout ... dev/root does not exist

[iaan]

Hello all,

I've been trying to upgrade a cluster from centos 6 to rocks 7 (centos 7) for a few days now and can't seem to get the compute nodes to install. I'm trying to install my compute nodes using PXE but keep getting dracut initqueue timeout error, and the warning that dev/root does not exist.

Code: Select all

Warning: dracut-initqueue timeout - starting timeout scripts
...
Warning: could not boot.
Warning: /dev/root does not exist

I have gone through many of the posts on this forum and tried to fix it using all solutions. I have attempted to modify the default pxelinux cfg file, regenerate the initramfs using dracut -f in rescue mode from a CD, and many other things. I am completely lost as for what could be causing this error. It seems like DHCP is working fine but It just won't kickstart.

When I look in rdsosreport, there isn't much information of use. I do see some errors surrounding ISCSI such as:

Code: Select all

iscid(644): can't open InitiatorName configuration file /etc/iscsi/initiatorname.iscsi

Does anyone have any advice? I have been stumped for too long. I will gladly post more information if required. Thanks.

[TrevorH]

That invariably means that it was unable to retrieve the initramfs and/or unpack it. Check that you've got enough RAM available - minimum is 1280MB, more helps.

Are you using iSCSI? If not then you can safely ignore iscsi related error messages.

Make sure you put the original initramfs from the DVD back on your PXE server. It doesn't need to be rebuilt and the chances are that if you do so then it will result in the very error messages that you are seeing.

[iaan]

Thanks for the quick reply Trevor!

I have about 6 gb of ram free so I'm clear on that front. Can you elaborate on what you mean in about moving the initramfs to the PXE server? I'm a bit of a centos rookie

[TrevorH]

Well, you said you'd rebuilt it and that may well be why it doesn't boot so I would recommend going back to the original DVD version of both kernel and initrd.img (they have to match). You can find them in the images/pxeboot/ directory on the CentOS boot media.

When you PXE boot, your network card will contact the DHCP server which then contains a 'next server' that points to your tftp server. It will then search down through $tftproot/pxelinux.cfg/ looking for files that match its MAC address, removing one digit at a time from the righthand end of it until it matches or drops through and uses the pxelinux.cfg/default file if no match is found. So if your MAC address is 123456789abc then it will look for pxelinux.cfg/123456789abc then pxelinux.cfg/1234568789ab then 123456789a, all the way down to "1" and stop on the first match or use the file called "default" if there is no match. Your pxelinux config file then has a menu that tells it the files to retrieve - kernel and initrd.img. It'll pull those over to the target machine and then load the initrd and boot the kernel.

So the error you are getting is one that's usually caused by either the wrong initrd.img being fetched, a failure to fetch it at all, the contents being corrupted or the image it has fetched was larger than the default ram disk size.

You'll want a pxelinux.cfg file that contains entries like this:

Code: Select all

LABEL CentOS 7 x86_64 Kickstart
        MENU LABEL CentOS 7.6 x86_64 Kickstart
        KERNEL images/centos/x86_64/7.6/vmlinuz
        APPEND initrd=images/centos/x86_64/7.6/initrd.img ramdisk_size=100000 ip=dhcp repo=http://192.168.x.y/mirrors/CentOS-7-x86_64/  ks=http://192.168.x.y/mirrors/seven4.ks text

[iaan]

This is the default rocks 7 pxelinux.cfg file that I am using:

Code: Select all

default rocks
prompt 0
label rocks
        kernel vmlinuz-7.0-x86_64
        append inst.ks.sendmac inst.sshd
        lang= devfs=nomount selinux=0
        ipv6.disable=1 
        rocks=client 
        ksdevice=enp133s0f1 
        inst.repo=http://127.0.0.1/install/rocks-dist/x86_64 
        inst.ks=http://localhost/fetchRocksKS.py
        inst.updates=http://127.0.0.1/install/rocks-dist/x86_64/images/updates.img
        rocks.ks=https://10.1.1.1/install/sbin/kickstart.cgi 
        tracker.trackers=10.1.1.1 initrd=initrd.img-7.0-x86_64

How does that look?

[TrevorH]

Utter rubbish?

I think you'll need to ask the Rocks people for support. It's sort of partly based on CentOS but we have no idea what they've done to it.

My initial thoughts when I read that were "don't all the options need to be on the same line as the APPEND statement with no CR/LF etc."

Then inst.repo pointing to 127.0.0.1? How is that going to work? That's itself and I doubt very much if there is an http server integrated into the boot process! Same thing for inst.updates= and inst.ks=. I really really hope that vmlinuz-7.0-x86_64 is not a CentOS 7.0 kernel as that is over 5 years out of date and positively riddled with security vulnerabilities.

We also never recommend anyone disables selinux.

[iaan]

Supposedly there is an http server associated with it. As for the line breaks in the append, those just happened when I copied and pasted for some reason. I guess I'll ask the folks over at rocks themselves. The point of rocks from my understanding is that it is meant to make setting up clusters easy but it's only given me problems because it uses older versions of centos 7

[TrevorH]

Personally I wouldn't use anything that doesn't supply timely security updates so unless they're shipping a version based on 7.6, I would avoid it.

[iaan]

Current release is based on centos 7.4. If I can't get rocks working I will probably just ditch rocks and setup the cluster myself using the newest release.

[iaan]

SOLVED: It looks like my problem was caused by an old nvidia network card that no longer has drivers supported in centos 7. PXE boot kickstarts well for centos 6 but not for 7. If anyone has a similar issue, check card compatibility.