Configuring a private iSCSI network on a second NIC

Issues related to configuring your network
Post Reply
delmer
Posts: 4
Joined: 2013/08/25 07:29:58

Configuring a private iSCSI network on a second NIC

Post by delmer » 2013/08/25 07:54:48

I am having issues configuring two separate subnets on two separate NICs. On eth0 I have a class A network with static IP's and no DHCP. eth0 connects to our production network. On eth1 I have a class C network with static IP's no DNS or DHCP AND no gateway since this is a iSCSI network. The iSCSI network consist of this host connected directly to a private switch via eth1 with the iSCSI target array and nothing else.

I have read up on static routes and have tried multiple configurations found online. The problem with most of the articles I have found is that they discuss multiple gateways on multiple NIC's. This seems very basic to me but im not sure what i am missing. Is static routes the appropriate solution? If so what is the appropriate syntax for this configuration?


[root@host ~]# uname -a
Linux host.com 2.6.18-348.16.1.el5 #1 SMP Wed Aug 21 04:00:25 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux

[root@host ~]# cat /etc/redhat-release
CentOS release 5.9 (Final)

[root@host ~]# ifconfig -a
eth0 Link encap:Ethernet HWaddr 1C:C1:DE:6C:A6:94
inet addr:XXX.XXX.XXX.XXX Bcast:XXX.XXX.XXX.XXX Mask:255.255.255.0
inet6 addr: fe80::1ec1:deff:fe6c:a694/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:22783 errors:0 dropped:0 overruns:0 frame:0
TX packets:18923 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4466451 (4.2 MiB) TX bytes:3616294 (3.4 MiB)
Interrupt:162 Memory:f4000000-f4012800

eth1 Link encap:Ethernet HWaddr 1C:C1:DE:6C:A6:96
inet addr:192.168.0.30 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::1ec1:deff:fe6c:a696/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:22 errors:0 dropped:0 overruns:0 frame:0
TX packets:68 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1408 (1.3 KiB) TX bytes:10374 (10.1 KiB)
Interrupt:170 Memory:f2000000-f2012800



[root@host ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
XXX.XXX.XXX.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
0.0.0.0 XXX.XXX.XXX.1 0.0.0.0 UG 0 0 0 eth0



root@host network-scripts]# cat ifcfg-eth0
# Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=none
USERCTL=yes
TYPE=Ethernet
NETMASK=255.255.255.0
IPADDR=xxx.xxx.xxx.xxx
GATEWAY=xxx.xxx.xxx.xxx
#PEERDNS=yes
IPV6INIT=no
HWADDR=1c:c1:de:6c:a6:94


[root@host network-scripts]# cat ifcfg-eth1
# Please read /usr/share/doc/initscripts-*/sysconfig.txt
# for the documentation of these parameters.
TYPE=Ethernet
DEVICE=eth1
HWADDR=1c:c1:de:6c:a6:96
BOOTPROTO=none
NETMASK=255.255.255.0
IPADDR=192.168.0.30
ONBOOT=yes
USERCTL=yes
IPV6INIT=no
#PEERDNS=yes

delmer
Posts: 4
Joined: 2013/08/25 07:29:58

Re: Configuring a private iSCSI network on a second NIC

Post by delmer » 2013/08/25 08:02:18

Apparently this got posted twice. Moderators feel free to delete one of these as I apparently can not.

Thanks

scottro
Forum Moderator
Posts: 2556
Joined: 2007/09/03 21:18:09
Location: NYC
Contact:

Configuring a private iSCSI network on a second NIC

Post by scottro » 2013/08/25 11:07:18

No problem, I'll delete the other one.

hawkmage
Posts: 162
Joined: 2011/12/17 00:00:11

Re: Configuring a private iSCSI network on a second NIC

Post by hawkmage » 2013/08/26 02:56:45

I assume that the iSCSI server you are accessing through eth1 is in the 192.168.0.0/24 subnet. If that is true then you do not need to add any static routes.

Are you unable to access the iSCSI server?

delmer
Posts: 4
Joined: 2013/08/25 07:29:58

Re: Configuring a private iSCSI network on a second NIC

Post by delmer » 2013/08/26 03:15:54

I cant say I know exactly what the problem was but everything is working now. I was not previously able to access anything on the 192.168.0.xxx/24 subnet until I tore done my bond interface, made some changes on the switch and rebooted the host.

I have now re-setup my 3 port bond using bonding mode 4 and channel-port groups on the switch and I can only ping two ports on the iSCSI target? I dont think thats expected behavior and when I disable channel-port groups on the switch i can then ping all 4 IPs. I think this may be because i do not have Link Aggregation Control Protocol enabled in my port channel and mode=4 probably requires that....

Ill be enabling LACP on my port channel shortly to see if I get the expected behavior.

hawkmage
Posts: 162
Joined: 2011/12/17 00:00:11

Re: Configuring a private iSCSI network on a second NIC

Post by hawkmage » 2013/08/26 21:51:02

OK, in your initial post you did not mention anything about using bonding. The interface names you posted indicated that you were using regular (non-bonded) interfaces.

You should ensure that the bond is working properly before looking at routing. For mode 4 (802.3ad or LACP) to work you do need it to be supported on the switch and available on the ports you have bonded.

Is the bonded interfere on the iSCSI initiator or target?

delmer
Posts: 4
Joined: 2013/08/25 07:29:58

Re: Configuring a private iSCSI network on a second NIC

Post by delmer » 2013/08/26 22:40:24

Correct, because at the time of the post I had already torn down the bond interface and was still having issues pinging through eth1.

The bond is on the iSCSI initiator and after recreating it and enabling LACP on the switch all seems to be working as expected.


I changed this:

interface GigabitEthernet0/41 *** eth1
switchport access vlan 3
switchport mode access
channel-group 5 mode on
!
interface GigabitEthernet0/42 *** eth2
switchport access vlan 3
switchport mode access
channel-group 5 mode on
!
interface GigabitEthernet0/43 *** eth3
switchport access vlan 3
switchport mode access
channel-group 5 mode on


To having the channel-group specifically call out "active" which enables LACP negotiation:

interface GigabitEthernet0/41
switchport access vlan 3
switchport mode access
channel-group 5 mode active
!
interface GigabitEthernet0/42
switchport access vlan 3
switchport mode access
channel-group 5 mode active
!
interface GigabitEthernet0/43
switchport access vlan 3
switchport mode access
channel-group 5 mode active

Post Reply