Two gateways with different set metrics - how?

Issues related to configuring your network
Post Reply
User avatar
AlainTH
Posts: 14
Joined: 2011/07/11 16:06:11
Location: France

Two gateways with different set metrics - how?

Post by AlainTH » 2019/08/20 13:36:13

Hello folks,

I'm begging for help here. Been spending hours researching how to solve what seemed like a simple problem: on a CentOS 7.6 host with two network interfaces on two different networks, how to have two default gateways with a set metric to favour one of these interfaces. Can't do it. Would anybody please tell me what I'm missing before I have no hair left?

First, I've tried using the METRIC=xxx setting in /etc/sysconfig/network-scripts-/ifcfg-<interface> files based on some advice found on the web. Obviously if that ever was supported, it's no longer. Next.

Then I've tried using policy-based routing following the tons of documentation that can be found online. Unfortunately very few mention adding metric values.

So, the full details (sorry it's long but I don't want to skip anything): this machine has two active interfaces enp10s4f0 (IP 10.149.247.23, prefix /24) and enp2s0f0 (IP 10.149.160.21, prefix /24).
It has two gateways: 10.149.247.254 and 10.149.160.254. I want 10.149.160.254 to be the "best" gateway, therefore with the lower metric.
I also want to avoid asymetric routing.
I can reach my goal of having different metrics on gateways by manually entering routes with "ip route add default ... dev ... metric" but I want it to be persistent.
So I've created two routing tables in /etc/iproute2/rt_tables:
247 enp10s4f0table
160 enp2s0f0table
And I've made the following rule and route files in /etc/sysconfig/network-scripts:

route-enp10s4f0:
10.149.247.0/24 dev enp10s4f0 src 10.149.247.23 table enp10s4f0table
default via 10.149.247.254 dev enp10s4f0 metric 110 table enp10s4f0table

rule-enp10s4f0:
from 10.149.247.23/32 table enp10s4f0table
to 10.149.247.23 table enp10s4f0table

route-enp2s0f0:
10.149.160.0/24 dev enp2s0f0 src 10.149.160.21 table enp2s0f0table
default via 10.149.160.254 dev enp2s0f0 table enp2s0f0table

rule-enp2s0f0:
from 10.149.160.21/32 table enp2s0f0table
to 10.149.160.21 table enp2s0f0table

Then I've used the proper incantations:
yum install NetworkManager-config-routing-rules
systemctl enable NetworkManager-dispatcher.service
systemctl start NetworkManager-dispatcher.service
...and reboot.

Obviously the configuration is read and processed properly but the metric is *not* applied to the 10.149.247.254 gateway:
# ip route show table enp10s4f0table
default via 10.149.247.254 dev enp10s4f0 metric 110
10.149.247.0/24 dev enp10s4f0 scope link src 10.149.247.23
# ip route show table enp2s0f0table
default via 10.149.160.254 dev enp2s0f0
# ip rule show
0: from all lookup local
32762: from all to 10.149.160.21 lookup enp2s0f0table
32763: from 10.149.160.21 lookup enp2s0f0table
32764: from all to 10.149.247.23 lookup enp10s4f0table
32765: from 10.149.247.23 lookup enp10s4f0table
32766: from all lookup main
32767: from all lookup default

But:
# ip route show
default via 10.149.247.254 dev enp10s4f0 proto static metric 100
default via 10.149.160.254 dev enp2s0f0 proto static metric 102
10.149.160.0/24 dev enp2s0f0 proto kernel scope link src 10.149.160.21 metric 102
10.149.247.0/24 dev enp10s4f0 proto kernel scope link src 10.149.247.23 metric 100
192.168.2.0/24 dev enp10s4f1 proto kernel scope link src 192.168.2.14 metric 101

[OK, I do have a third interface enp10s4f1 but this one has DEFROUTE=no and no GATEWAY= in its ifcfg file so I haven't mentioned it so far]

See? no metric 110 on the "default via 10.149.247.254 dev enp10s4f0..." line. And the result is the exact opposite to what I'm trying to achieve: enp10s4f0's default gateway has a lower metric than enp2s0f0's
I can confirm this by tracerouting to a host outside of these two networks, packets do get out through 10.149.247.254

I'm stuck at this point. Can't figure it out. Please kindly help if you can.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Two gateways with different set metrics - how?

Post by jlehtone » 2019/08/20 14:23:46

See section 'ipv4 setting' in:

Code: Select all

man nm-settings
I do use nmcli for configuration and don't care where/how it stores.

User avatar
AlainTH
Posts: 14
Joined: 2011/07/11 16:06:11
Location: France

Re: Two gateways with different set metrics - how?

Post by AlainTH » 2019/08/20 14:37:28

Thanks but I can't quite translate the table in this huge man page into what kind of nmcli command I should use.
Would you kindly elaborate a bit? do you mean that playing with these route-* and rule-* files is a dead end, not supported anymore or something like this? Most if not all advanced Linux routing tutorials I've found on the web seem to point to them...

EDIT: replying to myself. OK, I think I've figured it out:

Code: Select all

nmcli con modify enp2s0f0 ipv4.route-metric 100
nmcli con modify enp10s4f0 ipv4.route-metric 110
It actually seems to add a IPV4_METRIC=value to the ifcfg-<interface> files and appears to do the job. Much simpler than policy routing.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Two gateways with different set metrics - how?

Post by jlehtone » 2019/08/20 20:17:47

AlainTH wrote:
2019/08/20 14:37:28
do you mean that playing with these route-* and rule-* files is a dead end, not supported anymore or something like this?
Most if not all advanced Linux routing tutorials I've found on the web seem to point to them...
https://access.redhat.com/documentation ... networking
includes:
NetworkManager now supports configuring policy routing rules

Previously, users must set up policy routing rules outside of NetworkManager, for example by using the dispatcher script provided by the NetworkManager-dispatcher-routing-rules package. With this update, users can now configure rules as part of a connection profile. As a result, NetworkManager adds the rules when the profile is activated and removes the rules when the profile is deactivated
This shows that:
a) Policy based routing is still usable and used.
b) Before 7.7 (we are still at 7.6) one indeed had to edit the route-* and rule-* files directly.

An issue with tutorials (on various topics) is that they are unlikely to have written about current systems. (RHEL documentation is up to date.) On the other side, upstream documentation of e.g. systemd/firewalld/libvirt does not completely match RHEL/CentOS either, for it is "too new".

Post Reply