eduardolucioac wrote: ↑2019/08/23 18:46:40
Who is the br0?
-> An interface bridged to ens33.
What is that for?
If ens33 is enslaved to br0, then typically the ens33 should have no IP address and the address for "outside" is on the br0.
eduardolucioac wrote: ↑2019/08/23 18:59:46
We may be able to solve the problem if we find out why
Indeed.
Lets look at the initial FORWARD again (could use
iptables -S FORWARD):
Code: Select all
# XXX
-A FORWARD -d 10.1.0.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 10.1.0.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable # YYY
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT
-A FORWARD -j FORWARD_direct # ZZZ
-A FORWARD -j FORWARD_IN_ZONES # WWW
-A FORWARD -m conntrack --ctstate INVALID -j DROP
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
I did leave out couple empty chains (that have no effect here).
* The "works" adds an allow rule at XXX
* The "direct" adds an allow rule at ZZZ
* The "forward_port" adds an allow rule at WWW
* The YYY rule drops packets that would go into 10.1.0.0/24 before ZZZ and WWW
However, the system default FORWARD does not look like that. It is about:
Code: Select all
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i lo -j ACCEPT
-A FORWARD -j FORWARD_direct # ZZZ
-A FORWARD -j FORWARD_IN_ZONES # WWW
-A FORWARD -m conntrack --ctstate INVALID -j DROP
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
The libvirt, when it does start "a network" (e.g. virbr0), injects rules (probably via firewalld):
Code: Select all
*filter
-A FORWARD -d 10.1.0.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 10.1.0.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
*nat
-A POSTROUTING -s 10.1.0.0/24 -d 224.0.0.0/24 -j RETURN
-A POSTROUTING -s 10.1.0.0/24 -d 255.255.255.255/32 -j RETURN
-A POSTROUTING -s 10.1.0.0/24 ! -d 10.1.0.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 10.1.0.0/24 ! -d 10.1.0.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 10.1.0.0/24 ! -d 10.1.0.0/24 -j MASQUERADE
These are at the top of FORWARD and POSTROUTING, before regular firewalld ruleset. Since the netfilter operates on "first match" principle, these "rule".
Personally, I use bridged mode rather than routed, so I have almost no experience with this. However, I do have a faint recollection of seeing this issue before.
How to go around that? The direct rules have "passthrough" option. If I did guess right, the priority
3 should place our rule just before the first reject by libvirt (you definitely want your rule after the "related,established" rule):
Code: Select all
firewall-cmd --direct --passthrough --add-rule ipv4 filter FORWARD 3 -o virbr0 -d 10.1.0.9 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT
firewall-cmd --permanent --zone=public --direct --add-rule ipv4 nat PREROUTING 0 -p tcp --dport 222 -j DNAT --to-destination 10.1.0.9:22
(I would reboot the machine to see how libvirt and firewalld really play together.)
However, that is a "hack". IMHO, one should do this via the libvirt. That is our culprit that puts its rules where we don't expect them. No we have two paths:
A. Change where the libvirt-rules go. I think there is an upstream (bugzilla) request about that.
B. Make the libvirt add our rule into FORWARD when it adds rules. Libvirt does have documentation for doing that (filtering access to/from guest).
Check the option B.