Posted by kul3ana on 2011/12/7 2:47:14

Just installed a new centos 6 box with multiple vlans and came across a routing issue. when I try to communicate to an interface that needs to be routed the machine never replies.

I have 2 vlans created on a trunked port. vlan158 and vlan159. vlan158 has 192.168.158.6/24 and vlan159 has 192.168.159.253/24.
The switch/router trunk port it is plugged into has both the vlan158 and vlan159 setup and is also the router for both subnets.

sitting on the 192.168.158.0 subnet I can ping 192.168.158.6 but I cannot ping 192.168.159.253.
sitting on the 192.168.159.0 subnet I can ping 192.168.159.253 but I cannot ping 192.168.158.6

This exact setup on the same physical machine works fine in centos 5, so something in centos 6 got introduced that is preventing this. Can anyone give me a clue as to where to look for the problem?

I've listed the ifcfg-eth0, ifcfg-vlan158, ifcfg-vlan159 files below. Let me know if you would like any other files to look at.

#ifcfg-eth0

DEVICE="eth0"
HWADDR="00:19:B9:2F:6E:71"
NM_CONTROLLED="no"
ONBOOT="yes"

------------------------------------------------------

#ifcfg-vlan158

VLAN="yes"
VLAN_NAME_TYPE="VLAN_PLUS_VID_NO_PAD"
DEVICE="vlan158"
PHYSDEV="eth0"
BOOTPROTO="static"
ONBOOT="yes"
TYPE="Ethernet"
IPADDR="128.171.158.6"
NETMASK="255.255.255.0"

------------------------------------------------------

#ifcfg-vlan159

VLAN="yes"
VLAN_NAME_TYPE="VLAN_PLUS_VID_NO_PAD"
DEVICE="vlan159"
PHYSDEV="eth0"
BOOTPROTO="static"
ONBOOT="yes"
TYPE="Ethernet"
IPADDR="128.171.159.253"
NETMASK="255.255.255.0"

Posted by jlehtone on 2011/12/7 9:23:36

I would add the
NM_CONTROLLED="no"
to every ifcfg-* just to be sure that NM is not involved.

Does either interface see any incoming ping, when they do fail?

There are at least two recent threads here about dual interfaces
and source routing. That might apply to you.

Posted by kul3ana on 2011/12/7 19:34:04

Thanks jlehtone. although the other source routing issues recently listed weren't exactly like
my situation, there was enough info in them to apply to my situation. The bottom line is that
it is a iproute2 issue. I implemented all my interfaces using iproute commands in the ifcfg.
Using the previous config files as is, I needed to add 4 more commands to add the 2 rules/tables.
These were :

ip route add default via 128.171.158.1 dev vlan158 table 1
ip route add default via 128.171.159.1 dev vlan159 table 2
ip rule add from 128.171.158.6 tab 1 priority 500
ip rule add from 128.171.159.253 tab 2 priority 501

So now the question is where do I put these commands so they get implemented at boot up.
I know I can put them in the rc.local file, but I figured there must be a proper place to implement
these iproute2 commands. i.e. /etc/iproute2/rt_tables...

Thanks,
Ross

Posted by jlehtone on 2011/12/7 20:31:19

For the routes
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s1-networkscripts-static-routes.html
with the "IP arguments format" should do.

With the rules rc.local might still be the only way:
http://stevejenkins.com/blog/2011/09/how-to-set-up-two-nics-on-different-subnets-with-static-routes-and-separate-default-gateways-that-remain-after-a-reboot/

Posted by kul3ana on 2011/12/8 2:54:39

So after figuring out that it was a iproute2 problem, I found the following link giving an example of
where to put all the route and rules and table information to be read at boot up.

http://lists.centos.org/pipermail/centos/2011-January/105072.html

So I created 4 files in /etc/sysconfig/network-scripts that implemented the following lines

ip route add default via 128.171.158.1 dev vlan158 table 1
ip route add default via 128.171.159.1 dev vlan159 table 2
ip rule add from 128.171.158.6 tab 1 priority 500
ip rule add from 128.171.159.253 tab 2 priority 501

route-vlan158:
default via 128.171.158.1 dev vlan158 table 1

route-vlan159:
default via 128.171.159.1 dev vlan159 table 2

rule-vlan158:
from 128.171.158.6 tab 1 priority 500

rule-vlan159:
from 128.171.159.253 tab 2 priority 501

The ip addresses specified in the rules files are the ip addresses that are defined on vlan158 and vlan159
and the ip addresses specified in the route-vlan files are the gateways for the respective subnets. After I
had all those in there, I just rebooted to check it and I can ping the 159.253 address from the 158 subnet
and vice versa.


Thank you jlehtone. I never would have figured this out without your hints.

Posted by AlanBartlett on 2011/12/8 2:58:43

Thank you for reporting back.

For posterity (and on your behalf) I have marked the initial post to this thread as [SOLVED].