Firewalld & VPN Forwarding

Issues related to configuring your network
Post Reply
JoshuaPK
Posts: 10
Joined: 2014/07/20 20:12:26

Firewalld & VPN Forwarding

Post by JoshuaPK » 2019/05/12 22:00:49

I have a similar problem as described in this post: viewtopic.php?f=50&t=65945&p=277234

1. I have a physical host, that has several KVM virtual machines. The physical host's eth0 is on my 192.168.2 network. The .2 network is hard wired via gigabit switch.
2. The physical host also has an interface and network internal to KVM, virbr0, which is 192.168.4 network and used for all of the VM's.
3. I have a VM which acts as a VPN server. It gives out addresses in the 192.168.8 network.
4. For clients in the 192.168.8 network, they can reach servers in the .4 network. Also, servers in the .4 network are able to reach clients with open ports in the .8 network.
5. Clients in the .8 network can NOT reach anything on the .2 network, such as a Raspberry PI. Likewise, things on the .2 network can NOT reach anything on the .8 network. The gateway for .8 is properly configured as the .4 address of the VPN server.
6. If I turn off firewalld on the physical host, then clients in the .8 network CAN reach things in .2, and vice versa.
6.5. IP v4 forwarding is enabled in both the VPN VM and the physical host.
6.6. I have turned on masquerading both on the VPN VM as well as the physical host.
6.7. Enabling and/or disabling firewalld on the VPN VM does not change any of this behavior.
7. I have tried to put both virbr0 and eth0 in the same network Zone in firewalld. I have also tried to put them in different zones and explicitly configure firewalld. Nothing works.

Based on the post noted above, I added the following lines to the firewalld on the physical host. Now I can reach things on .2 from .8, but NOT vice versa.
firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i eth0 -o virbr0 -j ACCEPT
firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i virbr0 -o eth0 -j ACCEPT

Image

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Firewalld & VPN Forwarding

Post by jlehtone » 2019/05/13 07:45:14

Conseptually:

Code: Select all

LanA--RTR1--LanB--RTR2--LanC
* RTR1 has addresses RTR1a and RTR1b
* RTR2 has addresses RTR2b and RTR2c

No NAT/Masquerade anywhere.

* RTR1 has route to LanC via RTR2b
* RTR2 has route to LanA via RTR1b

* LanA members have routes:
- to LanB via RTR1a
- to LanC via RTR1a

* LanB members have routes:
- to LanA via RTR1b
- to LanC via RTR2b

* LanC members have routes:
- to LanA via RTR2c
- to LanB via RTR2c

Is that what you have and only problem being that one router filters too much forwarded traffic?


Do the VM's have to be in separate subnet? If the host would bridge rather than route, then you would have only two networks and the VPN VM would be the only router.

JoshuaPK
Posts: 10
Joined: 2014/07/20 20:12:26

Re: Firewalld & VPN Forwarding

Post by JoshuaPK » 2019/05/19 16:48:51

Yes, your thought process is correct. Due to security issues I can't simply bridge the interfaces- I'd rather keep things segregated.

To make things more interesting, when I have firewalld configured to log dropped packets, it doesn't log anything when I attempt to make a connection... even though it IS dropping the packet somewhere.

I've also attempted adding both interfaces to the 'trusted' zone, which theoretically shouldn't block anything. But it does.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Firewalld & VPN Forwarding

Post by jlehtone » 2019/05/20 08:00:01

JoshuaPK wrote:
2019/05/19 16:48:51
I've also attempted adding both interfaces to the 'trusted' zone, which theoretically shouldn't block anything. But it does.
You can (with relative ease) check what active rules you have about forwarding.
The built-in chain:

Code: Select all

# iptables -S FORWARD
-P FORWARD ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT
-A FORWARD -j FORWARD_direct
-A FORWARD -j FORWARD_IN_ZONES_SOURCE
-A FORWARD -j FORWARD_IN_ZONES
-A FORWARD -j FORWARD_OUT_ZONES_SOURCE
-A FORWARD -j FORWARD_OUT_ZONES
-A FORWARD -m conntrack --ctstate INVALID -j DROP
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
Nothing special there but the defaults. The actual rules are in subchains (and direct rules are checked first).

The various FORWARD_* chains:

Code: Select all

# iptables -S | grep FORWARD
split interfaces and sources to "zones".

The actual rules, if any, are deeper:

Code: Select all

# iptables -S | grep FWDI
# iptables -S | grep FWDO
Among them there are:

Code: Select all

-A FWDI_trusted -j ACCEPT
-A FWDO_trusted -j ACCEPT
Therefore, trusted to trusted should flow through.

Next step is to look at which rules are matched. For example:

Code: Select all

# iptables -vnL FORWARD
If you have no counted packets, then the packets never enter the netfilter. If there are packets, then you see which rules did they reach.

Plan B for me is to listen interface with tcpdump. If I see packet in incoming interface, then I shift to outgoing interface.
If the packet does not show there, then routing/filtering got it. If it shows, then replies should show too. Do the replies route back?

JoshuaPK
Posts: 10
Joined: 2014/07/20 20:12:26

Re: Firewalld & VPN Forwarding

Post by JoshuaPK » 2019/05/24 23:46:28

So I found a potential problem, but for the life of me I can't figure out why it's configured this way.

Code: Select all

-N FWDI_public
-N FWDI_public_allow
-N FWDI_public_deny
-N FWDI_public_log
-N FWDI_trusted
-N FWDI_trusted_allow
-N FWDI_trusted_deny
-N FWDI_trusted_log
-A FORWARD_IN_ZONES -i virbr2 -g FWDI_public
-A FORWARD_IN_ZONES -g FWDI_public
-A FORWARD_IN_ZONES_SOURCE -s 192.168.8.0/24 -j FWDI_trusted
-A FWDI_public -j FWDI_public_log
-A FWDI_public -j FWDI_public_deny
-A FWDI_public -j FWDI_public_allow
-A FWDI_public -p icmp -j ACCEPT
-A FWDI_trusted -j FWDI_trusted_log
-A FWDI_trusted -j FWDI_trusted_deny
-A FWDI_trusted -j FWDI_trusted_allow
-A FWDI_trusted -j ACCEPT
It looks like it's hitting the FWDI_trusted_log and FWDI_trusted_deny before it hits the FWDI_trusted_allow. FWDO is the same way. I know that libvirtd tampers with the firewall in a somewhat nonsensical manner, so I shut down all of my VM's and stopped libvirtd for this test. Otherwise I don't know why those deny rules are in there.

JoshuaPK
Posts: 10
Joined: 2014/07/20 20:12:26

Re: Firewalld & VPN Forwarding

Post by JoshuaPK » 2019/05/25 03:27:22

After a bit more research I found the actual cause of the problem. netfilter is blocking packets where the destination is the VPN network but the output interface (per the static route) is the virbr2. I attempted to add a direct rule, but it's putting the direct rule at the very bottom of the chain. It never reaches the direct rule because there's a rule in the middle of the chain that drops the packet.

Part of the challenge here is that the iptables -vnL command doesn't indicate which rule is described by each line. So this rule is what's blocking the packets:

Code: Select all

 1    60 REJECT     all  --  *      virbr2  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
The relevant configuration is this. The last rule should let any packet destined for 192.168.8.0/24 go through a "trusted" zone interface, which virbr2 is. But it appears the packet is never reaching that rule.

Code: Select all

-A FORWARD -d 192.168.4.0/24 -o virbr2 -j ACCEPT
-A FORWARD -s 192.168.4.0/24 -i virbr2 -j ACCEPT
-A FORWARD -i virbr2 -o virbr2 -j ACCEPT
-A FORWARD -o virbr2 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr2 -j REJECT --reject-with icmp-port-unreachable
........
-A FORWARD_IN_ZONES -i eno1 -j FWDI_trusted
-A FORWARD_IN_ZONES -i virbr2 -g FWDI_public
-A FORWARD_IN_ZONES -g FWDI_public
-A FORWARD_IN_ZONES_SOURCE -s 192.168.8.0/24 -j FWDI_trusted
-A FORWARD_OUT_ZONES -o eno1 -j FWDO_trusted
-A FORWARD_OUT_ZONES -o virbr2 -g FWDO_public
-A FORWARD_OUT_ZONES -g FWDO_public
-A FORWARD_OUT_ZONES_SOURCE -d 192.168.8.0/24 -j FWDO_trusted
Of course it's entirely possible I'm mis-reading a few things here. This is my first foray into firewalld.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Firewalld & VPN Forwarding

Post by jlehtone » 2019/05/31 12:03:50

I did show the default built-in chain in my previous post. This is from a machine, where libvirt has added a "network":

Code: Select all

# iptables -S FORWARD
-P FORWARD ACCEPT
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT
-A FORWARD -j FORWARD_direct
-A FORWARD -j FORWARD_IN_ZONES_SOURCE
-A FORWARD -j FORWARD_IN_ZONES
-A FORWARD -j FORWARD_OUT_ZONES_SOURCE
-A FORWARD -j FORWARD_OUT_ZONES
-A FORWARD -m conntrack --ctstate INVALID -j DROP
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

# iptables --lin -L FORWARD
Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  anywhere             192.168.122.0/24     ctstate RELATED,ESTABLISHED
2    ACCEPT     all  --  192.168.122.0/24     anywhere            
3    ACCEPT     all  --  anywhere             anywhere            
4    REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
5    REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
6    ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
7    ACCEPT     all  --  anywhere             anywhere            
8    FORWARD_direct  all  --  anywhere             anywhere            
9    FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere            
10   FORWARD_IN_ZONES  all  --  anywhere             anywhere            
11   FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere            
12   FORWARD_OUT_ZONES  all  --  anywhere             anywhere            
13   DROP       all  --  anywhere             anywhere             ctstate INVALID
14   REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited
The crucial point is that libvirt has added 5 rules before the default rules.

What libvirt usually adds is listed in https://libvirt.org/firewall.html
Alas, libvirt in CentOS 7 is version 4.5.0 and thus the upstream has more options than we do.

If libvirt adds rules when it starts and adds them before all other rules in FORWARD, then we cannot persistently have something early enough to be effective.

However, https://libvirt.org/formatnetwork.html describes

Code: Select all

<forward mode='open'>
that could avoid libvirt messing up our other rules.

One can nevertheless add rules via libvirt: https://libvirt.org/formatnwfilter.html

Post Reply