network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Issues related to configuring your network
User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/18 12:42:50

After upgrading CentOS 6.3 machine with 4 network interfaces on VMWare into CentOS 7.5, the network interfaces order got mixed. for example - eth0 becomes eth1.

On the upgraded CentOS 7.5 VM, I saw some errors in udevd saying:
Error changing net interface name 'eth3' to 'eth2': File exists
Error changing net interface name 'eth2' to 'eth1': File exists
Error changing net interface name 'eth0' to 'eth3': File exists
Error changing net interface name 'eth1' to 'eth0': File exists

I saw the following solution in RHEL https://access.redhat.com/solutions/1422203 but it didn't help.

- Issue happens on machines hosted on VMware
- we use net.ifnames=0 biosdevname=0 in the centos 7.5 grub file, for using the old network interface naming.
- on centos 6.3 we have the file /etc/udev/rules.d/69-vmxnet3-net.rules with the following data. this file is forwarded into the upgraded system and tries to change the nics order to the correct one:
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:0b:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:13:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:1b:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:04:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:0c:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:14:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth5"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:1c:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth6"
SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:05:00.0" DRIVERS=="vmxnet3", ATTR{type}=="1", KERNEL=="eth*", NAME="eth7"

Can you please help me setting the upgrade not to mix up the network interfaces order?
Thanks,
Mor.
Attachments
Capture (1).JPG
Capture (1).JPG (108.02 KiB) Viewed 5994 times

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by TrevorH » 2019/04/18 12:49:34

You really need to keep your systems up to date a bit more. CentOS 6.3 is from mid-2012 so you've not updated that for 7 years! You've also already started badly with CentOS 7 as the current version is not 7.5 but 7.6. I'd recommend checking for updates weekly, not every 5 years!

What happens if you remove your udev rules and add HWADDR=xx:yy:zz:aa:bb:cc to each of the /etc/sysconfig/network-scripts/ifcfg-eth* files? Obviously changing the MAC address to the one you want to be that device.

I alos presume you didn't actually *upgrade* from 6.3 to 7.5 as there is no supported upgrade path from 6.x to 7. It's a reinstall.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/18 14:38:48

Thank you for your reply.
You are totally correct, our product was based on a custom CentOS 6.3 for a long time and it a long while for upgrading.
Since moving to 7.5 it is planned to update every short while.
I tried your advice and removed the /etc/udev/rules.d/69-vmxnet3-net.rules and setting HWADDR on each ifcfg* file, and I get the same error and situation after rebooting the system.
"systemctl status systemd-udevd.service" outputs the same as with the rule file (similar to the attached image):
Error changing net interface name 'eth0' to 'eth3': File exists
Error changing net interface name 'eth2' to 'eth1': File exists
Error changing net interface name 'eth1' to 'eth0': File exists
Error changing net interface name 'eth3' to 'eth2': File exists

How can I change the network interfaces to their correct order?
Thanks.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by TrevorH » 2019/04/18 14:47:43

I suspect that your initramfs for the current kernel includes a copy of the udev rules that you deleted from the system. Rebuild it with dracut or just yum reinstall kernel-3.10.0-.957.10.1.el7.x86_64
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/21 06:14:42

TrevorH wrote:
2019/04/18 12:49:34
I alos presume you didn't actually *upgrade* from 6.3 to 7.5 as there is no supported upgrade path from 6.x to 7. It's a reinstall.
You are correct about it. The thing I am doing is kind of "export data -> installing new OS -> import data"
TrevorH wrote:
2019/04/18 14:47:43
I suspect that your initramfs for the current kernel includes a copy of the udev rules that you deleted from the system. Rebuild it with dracut or just yum reinstall kernel-3.10.0-.957.10.1.el7.x86_64
What do you mean? how can this happen? I am installing a new OS image, could it be that the initramfs is changed? the kernel I am using is not entirely vanilla, it contains some small parameter changes in. Does it matter on which machine it is compiled on?

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by TrevorH » 2019/04/21 11:04:09

The initramfs for each kernel is built when you install the package and is tailored for the hardware present on your machine at the time. It also picks up and includes various config files from /etc and puts them into the initramfs and if those files have subsequently been modified then you need to rebuild the initramfs file.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/21 11:57:20

Got you, thank you for the reply.
I tried your theory, so I removed /etc/initrams-$(uname-r).img file , and created a new one using "mkinitrd -f -v /boot/initrd-$(uname -r).img $(uname -r)".
Unfortunately, it changed nothing.
I tried doing that in multiple conditions: with/without net.ifnames=0 in the grub, with/without 69-vmxnet3-net.rules file in /etc/udev/rules.d.
After rebooting the machine with the new initramfs, In all of the situations - nothing has changed. Same error exists with the systemd-udevd service status, and the network interfaces order is still mixed.
Any idea?
Thanks.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by TrevorH » 2019/04/21 13:34:50

Remove your udev rules file. It's broken as it's trying to rename the interfaces to names that already exist.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

User avatar
moradiv
Posts: 21
Joined: 2019/04/18 09:34:53

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by moradiv » 2019/04/22 07:10:10

Same error.
I tried removing the udev file, and setting instead the "HWADDR" within ifcfg* files.
Rebooted the machine and got same situation: The network interfaces order is mixed and udevd service shows same error when trying to set the correct order.
This happens with the new created initramfs file, so I guess this is not related.

Any other idea?
Attachments
Capture.PNG
Capture.PNG (143.9 KiB) Viewed 5888 times

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: network interfaces order is changed after upgrading CentOS 6.3 to CentOS 7.5

Post by aks » 2019/04/22 07:57:20

I seem to recall that in CentOS 7 network order was not significant (which means it could change) - I blame systemd :)

From boot your network cards are setup and then you're trying to add them (not rename). It appears you've set the system to "force" the ethX naming convention.

I guess you'd have to remove all of them then add them one at a time. Or set one of them to be (say) eth9 and then shuffle about.

It would probably be easier if you simplified the number of cards, just for experimentation, until you've got something that works.

The file exists is the clue, I think udev is saying to you "I can not add ethX as ethX already exists".

I don't really understand why you need a specific order - I can think of some kind of aesthetics - or perhaps genuine layer2 traffic - which seems really silly these days.

If a packet is destined to a specific address, the network stack will know which interface to use (that's point of routing tables).

I guess if you had multiple networks all with the same subnet address ranges (I really feel for you if this is the case), you could always do something like "if the packet came in on this interface, send it back out the same interface" kind of thing - unless you're routing between the networks of course.

Post Reply