Firewall Rules not allowing VPN Traffic to pass through

Issues related to configuring your network
Post Reply
user12345
Posts: 4
Joined: 2019/07/30 12:52:30

Firewall Rules not allowing VPN Traffic to pass through

Post by user12345 » 2019/07/30 13:01:31

Hi all, would need some help.

I have a CentOS 7 Server sitting between two cisco routers. Between these routers is a site-to-site VPN connection, and CentOS 7 is configured as a router to forward/route packets from one end of the router to the other. However, when I try to ping from one end to another, it seems that the ping connection failed. Further debug and analysis reveals that it is possible that the CentOS 7 server is blocking VPN traffic.

The VPN Configuration is definitely correct since I have tested it without the CentOS 7 server in the middle, and it works. I have already enabled firewalld and configure ports to open with regards to ports used in Cisco VPN. I still have no idea what I am missing out. Maybe I am missing out something? Hope someone can reply me soon :)

Attached is my firewall rules on CentOS 7

[root@localhost log]# firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: ens33 ens37
sources:
services: ssh dhcpv6-client ipsec
ports: 500/udp 500/tcp 4500/udp 1723/udp 1723/tcp 4500/tcp 50/udp 1701/udp
protocols:
masquerade: yes
forward-ports:
source-ports:
icmp-blocks:
rich rules:

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Firewall Rules not allowing VPN Traffic to pass through

Post by jlehtone » 2019/07/30 21:59:07

Code: Select all

iptables -S
That shows the actual rules that are in the kernel.

Code: Select all

iptables -vnL
That shows same in different format, with statistics.

The point of interest is the chain FORWARD and the chains it calls.
* Have there been packets that have either passed or been blocked therein?
* What is allowed to be forwarded (aka routed)?

user12345
Posts: 4
Joined: 2019/07/30 12:52:30

Re: Firewall Rules not allowing VPN Traffic to pass through

Post by user12345 » 2019/07/31 02:42:34

Hi, thanks for your help. Unfortunately, I am not understanding what you have meant. I presume that as long as there are routes configured on the server to route packets from one router to another, then it should be fine? Pardon as I have little experience with iptables.

Another thing is that I am using firewalld, instead of iptables. Firewalld is the default firewall for CentOS 7.

Attached is the output of "iptables -vnL": https://pastebin.com/bt0upPbn

Another interesting thing to note, if you have done Cisco VPN before, is that there is an error on the ISAKMP Phase 1 SA policy proposal. The error states that the peer is not accepting the proposal. However, the peer stated is the directly connected IP address of the server, instead of the router. Therefore, I suspect that the router assumes that the peer (the other end of the VPN tunnel) is the CentOS server, but instead it should be the cisco router.

Looking forward to your reply.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Firewall Rules not allowing VPN Traffic to pass through

Post by jlehtone » 2019/07/31 07:56:27

user12345 wrote:
2019/07/31 02:42:34
I presume that as long as there are routes configured on the server to route packets from one router to another, then it should be fine?

Another thing is that I am using firewalld, instead of iptables. Firewalld is the default firewall for CentOS 7.
The actual, active filter rules are in the kernel memory.
Firewalld is a service. It has replaced the iptables service. Both load rules to kernel, but firewalld is more elaborate.

Both use same tool to read and write filter rules. That tool is iptables.
The iptables.service is a simple script that runs 'iptables' on boot to load rules from file to kernel.

The iptables commands that I did show do read and show the rules. That provides a different view than firewall-cmd.
(I do configure with firewalld, but still like to peek with iptables.)

Part of iptables -vnL FORWARD:

Code: Select all

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination        
   11  1898 ACCEPT     all  --  ens37  *       0.0.0.0/0            0.0.0.0/0          
   37  8109 ACCEPT     all  --  ens33  *       0.0.0.0/0            0.0.0.0/0          
    0     0 ACCEPT     all  --  ens37  *       0.0.0.0/0            0.0.0.0/0          
    0     0 ACCEPT     all  --  ens33  *       0.0.0.0/0            0.0.0.0/0          
Order of rules is important, but for now lets just note that
* 11 packets have arrived via ens37 and have been accepted to be routed
* 37 packets have arrived via ens33 and have been accepted to be routed

Part of iptables -vnL INPUT:

Code: Select all

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination        
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0          
   34  7280 INPUT_direct  all  --  *      *       0.0.0.0/0            0.0.0.0/0          
   34  7280 INPUT_ZONES_SOURCE  all  --  *      *       0.0.0.0/0            0.0.0.0/0          
   34  7280 INPUT_ZONES  all  --  *      *       0.0.0.0/0            0.0.0.0/0          
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
   34  7280 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited
34 packets have arrived to this CentOS 7 host. None of them did match any specific rule and therefore they were all rejected.

All the firewalld "services and ports" are only in the chain IN_public_allow. Those 34 packets did not match any of them.
The "IN" rules are not used for routed (forwarded) traffic.

I have machines that do route traffic. They don't have anything like "ACCEPT all -- ens37 *" in their FORWARD:

Code: Select all

$ iptables -S FORWARD
-P FORWARD ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT
-A FORWARD -j FORWARD_direct
-A FORWARD -j FORWARD_IN_ZONES_SOURCE
-A FORWARD -j FORWARD_IN_ZONES
-A FORWARD -j FORWARD_OUT_ZONES_SOURCE
-A FORWARD -j FORWARD_OUT_ZONES
-A FORWARD -m conntrack --ctstate INVALID -j DROP
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
Routes alone do not make a router. Every machine has routes. Whenever the system gets a packet, it makes a "routing decision":
* Is this for me? If yes, hand it to local process (through INPUT filter).
* If no, throw it out from interface that the routes say (through FORWARD filter).

However, forwarding requires that it is enabled in the kernel:

Code: Select all

$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
If that value is 0, you can enable it for this session with:

Code: Select all

sysctl net.ipv4.ip_forward=1
To have it persist after reboot, one has to write it to a file:

Code: Select all

echo "sysctl net.ipv4.ip_forward = 1" >> /etc/sysctl.d/90-enable-router.conf
(I don't actually have such file, for certain firewalld zone combinations set it for me.)


Lets recap:
1. A packet from "cisco-1" arrives from ens33 and has destination address "cisco-2"
2. Routes state that the packet has to be send out from ens37
3. ip_forward == 1, routing is enabled
4. Firewall rules in FORWARD allow "from ens33 to ens37"
5. Succee

The replies are handled similarly.

IMHO, comfy way to see the routes:

Code: Select all

ip ro

Oh my,

Code: Select all

masquerade: yes
You have both interfaces on zone public and you have enabled masquerade (aka SNAT) on zone public.

That changes the story:
1. A packet from "cisco-1" arrives from ens33 and has destination address "cisco-2"
2. Routes state that the packet has to be send out from ens37
3. ip_forward == 1, routing is enabled
4. Firewall rules in FORWARD allow "from ens33 to ens37"
5. SNAT: Sender address is set to IP of ens37.

* cisco-2 receives packet and replies to IP-of-ens37

1. A packet from "cisco-2" arrives from ens37 and has destination address "IP-of-ens37"
2. Masquerade notices reply and changes destination to "cisco-1"
3. Routes state that the packet has to be send out from ens33
4. ip_forward == 1, routing is enabled
5. Firewall rules in FORWARD allow "reply in existing connection"
6. SNAT: Sender address is set to IP of ens33.

* cisco-1 receives packet "from IP-of-ens33". That is unexpected. We did just call cisco-2. That does not reply.

Why do you have the MASQUERADE on?

cisco-1 should have effective route: "to cisco-2, send via IP-of-ens33"
cisco-2 should have effective route: "to cisco-1, send via IP-of-ens37"
With those the SNAT is not needed. Definitely not to both directions.

user12345
Posts: 4
Joined: 2019/07/30 12:52:30

Re: Firewall Rules not allowing VPN Traffic to pass through

Post by user12345 » 2019/07/31 12:38:24

Hey man, thanks for your detailed explanation!

Now I have a better understanding of how Iptables works, with regards to effectively using it to tell whether traffic is being passed through.

With your help, I think I have isolated the problem to be the masquerading option set for the ens33 and ens37 interfaces. By removing the masquerade option, the 2 cisco routers are able to ping one another with VPN tunnel establish between the two of them.

I have already configured traffic forwarding in the kernel, just that I could have mixed up the firewall rules. The reason I have set "masquerade" on both interfaces is because I have read an article that states re-routing of traffic in the CentOS server requries enabling "masquerading". Looks like the article could be wrong in terms of that.

Just to clarify with you, if the CentOS server is configuring properly to route the traffic, does it still need to utilize iptables configurations to forward or open ports needed for the VPN traffic? In my case, it seems that as long as the routes is configured correctly between the two routers, there is no need to do any port forwarding of sort. There is no explicit need to configure any postrouting/nat traffic since CentOS server is simply functioning as a router.

Please advise on this. Once again, thank you for detailed explanation. :D

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Firewall Rules not allowing VPN Traffic to pass through

Post by jlehtone » 2019/07/31 14:17:10

Masquerade is one form of network address translation (NAT). Router forwards traffic from one subnet to another subnet. That in itself requires no translation.

A home usually has subnet with private address range. You cannot forward to public net a packet that has private address in the "from" field. Nobody could reply to you. That is where the translation is necessary.


Lets say that you have machine A, machine B, and router R between them. A--R--B.
The B has HTTP server. Your browser on A sends packets to B. They go through the router R. B replies to A.
(For example, A is your desktop, B is www.centos.org, and we ignore the NAT.)
A has to know that B is behind R.
B has to know that A is behind R.
The R has to allow the HTTP traffic to go through. Yes, the firewall on the R has to allow forwarded traffic.

What was in your

Code: Select all

iptables -S FORWARD

In your case A is a cisco router, "browser" is "cisco VPN", B is a cisco router, and "HTTP server" is "cisco VPN".
From viewpoint of your CentOS "R" the VPN traffic is no different from HTTP traffic; A and B communicate.
(network protocol and source/destination ports of VPN differ from HTTP)
Inside the packets that cisco VPN's use will be your actual traffic, encrypted. The R won't see or care.

user12345
Posts: 4
Joined: 2019/07/30 12:52:30

Re: Firewall Rules not allowing VPN Traffic to pass through

Post by user12345 » 2019/08/01 12:30:39

Great explanation. I have understood much better in terms of NAT concepts.

Code: Select all

iptables -S FORWARD
-P FORWARD ACCEPT
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i ens37 -j ACCEPT
-A FORWARD -i ens33 -j ACCEPT
-A FORWARD -i ens37 -j ACCEPT
-A FORWARD -i ens33 -j ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT
-A FORWARD -j FORWARD_direct
-A FORWARD -j FORWARD_IN_ZONES_SOURCE
-A FORWARD -j FORWARD_IN_ZONES
-A FORWARD -j FORWARD_OUT_ZONES_SOURCE
-A FORWARD -j FORWARD_OUT_ZONES
-A FORWARD -m conntrack --ctstate INVALID -j DROP
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
Probably its because there are rules to forward traffic coming from both interfaces, therefore the VPN traffic is able to route from one end to the other.

Thanks for your help all these while!

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: Firewall Rules not allowing VPN Traffic to pass through

Post by aks » 2019/08/01 17:07:20

Didn't read all of the above, but a quick thought:

Cisco is notorious for proprietary networking stuff - especially with VPNs (they charge for clients), as does Juniper and a whole host of others.

Cursory glance suggests IKEv[1|2] - are you sure? Cisco labels something IKE and "adds" it's copyrighted and licensed crap on top.

In standardised IKEv1 NAT-T (which is what you probably want) is usually UDP port 4500.

I guess switch off the firewall and see if it still works.

Post Reply