Possible networking issue in CentOS 7.6

General support questions
Post Reply
tri-mate
Posts: 1
Joined: 2019/05/26 20:34:19

Possible networking issue in CentOS 7.6

Post by tri-mate » 2019/05/26 21:04:30

I noticed a networking issue in CentOS 7.6. I am not sure whether this is particular to my computer or what.

It's a physical machine. It's a recently installed system. I even upgraded the kernel to the newest available one in the repositories.
[root@w ~]# uname -a
Linux w.mylab.local 3.10.0-957.12.2.el7.x86_64 #1 SMP Tue May 14 21:24:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@w ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

[root@w ~]# cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)
The way I am used to configuring the network interfaces in CentOS 7 is through /etc/sysconfig/network-scripts/ifcfg-<network_interface_name>.

I usually put something like this there:

Code: Select all

TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
NAME=enp2s0
DEVICE=enp2s0
ONBOOT=yes

IPADDR=10.5.5.50
PREFIX=24
GATEWAY=10.5.5.123

NOZEROCONF=yes
NMCONTROLLED=no
I usually disable NetworkManager and do everything manually with the ifcfg scripts.
Whenever I change something there I usually do:

Code: Select all

ip address flush dev <network_interface_name>
systemctl restart network
This used to work without issues in the past. Now it seems that something is causing problems.
I need to wait for 30-60s for the system to "realize that I configured something differently". Basically, the interface gets the new IP address, the link is in the up state, the routes are correct and present in the routing table, the only thing is that I need to wait 30-60s for the networking to "start working". Because I use a GUI I get a notification that something went wrong with the "kernel" package, I get a mail in /var/spool/mail/root every time I change something in these scripts and try to restart the network.

Here is an excerpt from /var/log/messages
May 26 13:12:38 w systemd: Started Hostname Service.
May 26 13:12:53 w dbus[4615]: [system] Activating service name='org.freedesktop.problems' (using serviceh
elper)
May 26 13:12:53 w dbus[4615]: [system] Successfully activated service 'org.freedesktop.problems'
May 26 13:13:31 w avahi-daemon[4640]: Withdrawing address record for 10.5.5.50 on enp2s0.
May 26 13:13:31 w avahi-daemon[4640]: Leaving mDNS multicast group on interface enp2s0.IPv4 with address
10.5.5.50.
May 26 13:13:31 w avahi-daemon[4640]: Interface enp2s0.IPv4 no longer relevant for mDNS.
May 26 13:13:31 w avahi-daemon[4640]: Withdrawing address record for fe80::329c:23ff:fe47:a83d on enp2s0.
May 26 13:13:37 w systemd: Stopping LSB: Bring up/down networking...
May 26 13:13:37 w network: Shutting down interface enp2s0: [ OK ]
May 26 13:13:37 w network: Shutting down loopback interface: [ OK ]
May 26 13:13:37 w systemd: Stopped LSB: Bring up/down networking.
May 26 13:13:37 w systemd: Starting LSB: Bring up/down networking...
May 26 13:13:37 w network: Bringing up loopback interface: [ OK ]
May 26 13:13:37 w kernel: r8169 0000:02:00.0 enp2s0: link down
May 26 13:13:37 w kernel: r8169 0000:02:00.0 enp2s0: link down
May 26 13:13:37 w kernel: IPv6: ADDRCONF(NETDEV_UP): enp2s0: link is not ready
May 26 13:13:41 w kernel: r8169 0000:02:00.0 enp2s0: link up
May 26 13:13:41 w kernel: IPv6: ADDRCONF(NETDEV_CHANGE): enp2s0: link becomes ready
May 26 13:13:41 w avahi-daemon[4640]: Joining mDNS multicast group on interface enp2s0.IPv4 with address
192.168.178.10.
May 26 13:13:41 w avahi-daemon[4640]: New relevant interface enp2s0.IPv4 for mDNS.
May 26 13:13:41 w avahi-daemon[4640]: Registering new address record for 192.168.178.10 on enp2s0.IPv4.
May 26 13:13:41 w network: Bringing up interface enp2s0: [ OK ]
May 26 13:13:41 w systemd: Started LSB: Bring up/down networking.
May 26 13:13:42 w avahi-daemon[4640]: Registering new address record for fe80::329c:23ff:fe47:a83d on enp
2s0.*.
May 26 13:14:17 w kernel: ------------[ cut here ]------------
May 26 13:14:17 w kernel: WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:356 dev_watchdog+0x248/0x260
May 26 13:14:17 w kernel: NETDEV WATCHDOG: enp2s0 (r8169): transmit queue 0 timed out
May 26 13:14:17 w kernel: Modules linked in: tcp_lp fuse iptable_filter iptable_raw iptable_mangle iptabl
e_security iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sunrpc dm_mirror
dm_region_hash dm_log dm_mod intel_pmc_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm i
TCO_wdt irqbypass iTCO_vendor_support snd_hda_codec_hdmi crc32_pclmul ghash_clmulni_intel aesni_intel snd
_hda_codec_realtek snd_hda_codec_generic lrw gf128mul glue_helper joydev ablk_helper snd_hda_intel pcspkr
snd_hda_codec cryptd sg snd_hda_core snd_hwdep snd_seq mei_me pinctrl_sunrisepoint pinctrl_intel mei acp
i_pad snd_seq_device snd_pcm snd_timer i2c_i801 snd soundcore ip_tables xfs libcrc32c sr_mod sd_mod cdrom
crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel nouveau serio_raw
May 26 13:14:17 w kernel: r8169 mii video mxm_wmi i2c_hid i2c_algo_bit drm_kms_helper wmi syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci libata drm_panel_orientation_quirks
May 26 13:14:17 w kernel: CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Not tainted 3.10.0-957.12.2.el7.x86_64 #1
May 26 13:14:17 w kernel: Hardware name: MSI MS-7996/H110M PRO-VH (MS-7996), BIOS 2.E0 08/11/2017
May 26 13:14:17 w kernel: Call Trace:
May 26 13:14:17 w kernel: <IRQ> [<ffffffffbab63041>] dump_stack+0x19/0x1b
May 26 13:14:17 w kernel: [<ffffffffba4976e8>] __warn+0xd8/0x100
May 26 13:14:17 w kernel: [<ffffffffba49776f>] warn_slowpath_fmt+0x5f/0x80
May 26 13:14:17 w kernel: [<ffffffffbaa66c38>] dev_watchdog+0x248/0x260
May 26 13:14:17 w kernel: [<ffffffffbaa669f0>] ? dev_deactivate_queue.constprop.26+0x60/0x60
May 26 13:14:17 w kernel: [<ffffffffba4a8048>] call_timer_fn+0x38/0x110
May 26 13:14:17 w kernel: [<ffffffffbaa669f0>] ? dev_deactivate_queue.constprop.26+0x60/0x60
May 26 13:14:17 w kernel: [<ffffffffba4aa4ad>] run_timer_softirq+0x24d/0x300
May 26 13:14:17 w kernel: [<ffffffffba4a0ff5>] __do_softirq+0xf5/0x280
May 26 13:14:17 w kernel: [<ffffffffbab7932c>] call_softirq+0x1c/0x30
May 26 13:14:17 w kernel: [<ffffffffba42e675>] do_softirq+0x65/0xa0
May 26 13:14:17 w kernel: [<ffffffffba4a1375>] irq_exit+0x105/0x110
May 26 13:14:17 w kernel: [<ffffffffbab7a6e8>] smp_apic_timer_interrupt+0x48/0x60
May 26 13:14:17 w kernel: [<ffffffffbab76df2>] apic_timer_interrupt+0x162/0x170
May 26 13:14:17 w kernel: <EOI> [<ffffffffba4c5d9d>] ? hrtimer_start_range_ns+0x1ed/0x3c0
May 26 13:14:17 w kernel: [<ffffffffba9aee37>] ? cpuidle_enter_state+0x57/0xd0
May 26 13:14:17 w kernel: [<ffffffffba9aee2d>] ? cpuidle_enter_state+0x4d/0xd0
May 26 13:14:17 w kernel: [<ffffffffba9aef8e>] cpuidle_idle_call+0xde/0x230
May 26 13:14:17 w kernel: [<ffffffffba4366de>] arch_cpu_idle+0xe/0xc0
May 26 13:14:17 w kernel: [<ffffffffba4fc65a>] cpu_startup_entry+0x14a/0x1e0
May 26 13:14:17 w kernel: [<ffffffffbab51117>] rest_init+0x77/0x80
May 26 13:14:17 w kernel: [<ffffffffbb1861c6>] start_kernel+0x44b/0x46c
May 26 13:14:17 w kernel: [<ffffffffbb185b7b>] ? repair_env_string+0x5c/0x5c
May 26 13:14:17 w kernel: [<ffffffffbb185120>] ? early_idt_handler_array+0x120/0x120
May 26 13:14:17 w kernel: [<ffffffffbb18572f>] x86_64_start_reservations+0x24/0x26
May 26 13:14:17 w kernel: [<ffffffffbb185885>] x86_64_start_kernel+0x154/0x177
May 26 13:14:17 w kernel: [<ffffffffba4000d5>] start_cpu+0x5/0x14
May 26 13:14:17 w kernel: ---[ end trace 114322db7dbf958e ]---
May 26 13:14:17 w kernel: r8169 0000:02:00.0 enp2s0: link up
May 26 13:14:18 w sh: abrt-dump-oops: Found oopses: 1
May 26 13:14:18 w sh: abrt-dump-oops: Creating problem directories
May 26 13:14:18 w sh: abrt-dump-oops: Not going to make dump directories world readable because PrivateReports is on
May 26 13:14:18 w abrt-server: Duplicate: core backtrace
May 26 13:14:18 w abrt-server: DUP_OF_DIR: /var/spool/abrt/oops-2019-05-24-09:32:40-9595-0
May 26 13:14:18 w abrt-server: Deleting problem directory oops-2019-05-26-13:14:18-18126-0 (dup of oops-2019-05-24-09:32:40-9595-0)
May 26 13:14:19 w abrt-server: Email address of sender was not specified. Would you like to do so now? If not, 'user@localhost' is to be used [y/N]
May 26 13:14:19 w abrt-server: Email address of receiver was not specified. Would you like to do so now? If not, 'root@localhost' is to be used [y/N]
May 26 13:14:19 w abrt-server: Sending an email...
May 26 13:14:19 w abrt-server: Sending a notification email to: root@localhost
May 26 13:14:19 w abrt-server: Email was sent to: root@localhost
May 26 13:14:19 w abrt-dump-oops: Reported 1 kernel oopses to Abrt
May 26 13:15:49 w avahi-daemon[4640]: Withdrawing address record for 192.168.178.10 on enp2s0.
May 26 13:15:49 w avahi-daemon[4640]: Leaving mDNS multicast group on interface enp2s0.IPv4 with address 192.168.178.10.
May 26 13:15:49 w avahi-daemon[4640]: Interface enp2s0.IPv4 no longer relevant for mDNS.
May 26 13:15:49 w avahi-daemon[4640]: Withdrawing address record for fe80::329c:23ff:fe47:a83d on enp2s0.
May 26 13:15:55 w systemd: Stopping LSB: Bring up/down networking...
May 26 13:15:55 w network: Shutting down interface enp2s0: [ OK ]
May 26 13:15:55 w network: Shutting down loopback interface: [ OK ]
May 26 13:15:55 w systemd: Stopped LSB: Bring up/down networking.
May 26 13:15:55 w systemd: Starting LSB: Bring up/down networking...
May 26 13:15:55 w network: Bringing up loopback interface: [ OK ]
May 26 13:15:55 w kernel: r8169 0000:02:00.0 enp2s0: link down
May 26 13:15:55 w kernel: r8169 0000:02:00.0 enp2s0: link down
May 26 13:15:55 w kernel: IPv6: ADDRCONF(NETDEV_UP): enp2s0: link is not ready
May 26 13:15:58 w kernel: r8169 0000:02:00.0 enp2s0: link up
May 26 13:15:58 w kernel: IPv6: ADDRCONF(NETDEV_CHANGE): enp2s0: link becomes ready
May 26 13:15:59 w avahi-daemon[4640]: Joining mDNS multicast group on interface enp2s0.IPv4 with address 10.5.5.50.
May 26 13:15:59 w avahi-daemon[4640]: New relevant interface enp2s0.IPv4 for mDNS.
May 26 13:15:59 w avahi-daemon[4640]: Registering new address record for 10.5.5.50 on enp2s0.IPv4.
May 26 13:15:59 w network: Bringing up interface enp2s0: [ OK ]
May 26 13:15:59 w systemd: Started LSB: Bring up/down networking.
May 26 13:16:00 w avahi-daemon[4640]: Registering new address record for fe80::329c:23ff:fe47:a83d on enp2s0.*.
May 26 13:16:34 w kernel: r8169 0000:02:00.0 enp2s0: link up
May 26 13:20:01 w systemd: Created slice User Slice of root.
May 26 13:20:01 w systemd: Started Session 15 of user root.
May 26 13:20:01 w systemd: Removed slice User Slice of root.
Am I the only one having this issue ? Did something change in CentOS ? This used to work without problems in older 7.X versions of CentOS.

amey101
Posts: 1
Joined: 2019/09/29 04:33:17

Re: Possible networking issue in CentOS 7.6

Post by amey101 » 2019/09/29 04:35:42

I am also facing the same issue, not much available on google ... Any solution?

tony_down_under
Posts: 83
Joined: 2019/08/07 01:50:24
Location: Perth, Australia but originally from Carshalton, Surrey

Re: Possible networking issue in CentOS 7.6

Post by tony_down_under » 2019/10/02 07:21:40

1. What happens if you do not edit any ifcfg script but instead, use the gui to set the network setting? - is it the same issue?

2. what network settings are you changing specifically?

3. I'm not sure we are aware of what the specific issue is.. Can you confirm what is the specific issue when you say "I need to wait for the computer to realise that I configured something differently"

Looking at the log output where it says "NETDEV WATCHDOG: enp2s0 (r8169): transmit queue 0 timed out" I googled it and a RedHat Bugzilla says it could be an issue with the nic. Do you have a usb network adapter you could test with (or another nic)? ref: https://bugzilla.redhat.com/show_bug.cgi?id=576058

Lastly, when I update network config on my systems I am only updating the ifcfg file and then doing a "systemctl restart network".

Also see this https://lists.debian.org/debian-user/20 ... 01003.html

My guess as to what is happening is, after network restart there is some issue between kernel and wire (either adapter itself, or kernel driver). Before your system will get network access, it will need to send ARP packets to resolve at very least the default gateways MAC address. IF you connect to other systems on the same subnet then also those MAC addresses. You can view the arp table with "arp -n". My bet is you'll have no entries during the issue. Just a guess.

tunk
Posts: 1205
Joined: 2017/02/22 15:08:17

Re: Possible networking issue in CentOS 7.6

Post by tunk » 2019/10/02 09:57:12

I don't know if it will make any difference, but you could try to update to CentOS 7.7.

dunch
Posts: 66
Joined: 2018/11/07 13:48:53
Location: Yorkshire

Re: Possible networking issue in CentOS 7.6

Post by dunch » 2019/10/02 11:26:39

I suspect you have a hardware issue here. If you haven't got another NIC handy you could try unloading/reloading the driver to give the card a poke but there's probably not much you can do.
Sorry I can't offer anything else.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Possible networking issue in CentOS 7.6

Post by TrevorH » 2019/10/02 14:46:04

Please post the output from lspci -nn | grep -i net so we can see the exact hardware in use.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

diksha
Posts: 2
Joined: 2019/10/05 12:44:07

Re: Possible networking issue in CentOS 7.6

Post by diksha » 2019/10/05 12:50:32

whats the output of # ip link command.

Post Reply