"ip rule add fwmark ..." not working in latest Kong build

3 posts / 0 new
Last post
david.woodward
david.woodward's picture
"ip rule add fwmark ..." not working in latest Kong build

I've loaded the latest Kong release (06-30-2015) on my router and have succeffully configured the router to create a VPN connection and setup a policy to route a specific IP LAN address through that VPN by using the seemingly common setup of configuring a new route table with a default route for the VPN gateway and a simple "ip rule add from ... via ..." policy.  So, I know my VPN and additional route table are setup properly.

Here's where I'm having trouble.  I want to exclude certain traffic from this IP address from passing through the VPN.  From what I've seen this is best accomplished by using iptables mangle table to mark traffic with a few bits and then setup a rule to route that traffic through the VPN.  But I'm having no luck getting that to work.

Here are the key bits I've got in place attempting to accomplish a basic proof of concept for this right now (from memory - I'm not at home to see my exact setup at the moment).  And not that this is just trying to get ALL traffic from an IP to flwo through the new table using marks/mangle rather than the "from <ip>" rule:


ip route add default via <vpn_ip> dev <vpn_if> table 200

ip rule add fwmark 0x7 table 200

iptables -I PREROUTING -t mangle -s 192.168.1.5 -j MARK --set-mark 0x7

iptables -A POSTROUTING -t nat -o <vpn_if> -j MASQUERADE


As far as I can tell, the mangle/marking is happening because iptables output shows packets being affected by the mangle/mark entry.  But, the ip rule to route marked traffic to the new VPN table doesn't seem to work at all because traffic still flows over the standard ISP gateway. And I know my table setup works because if I use the "ip rule add from <ip> ..." method without marking anything then traffic from that LAN ip is routed over the VPN gateway as expected)

My first thought is that some other ip rule catches that traffic first, but with "ip rule show" and "ip rule list" returning nothing (by design in ddwrt I believe), I really don't know where to go from here.

Can anyone point me in the right direction?

david.woodward
david.woodward's picture
So, I figured it out.

So, I figured it out.

The key point I was missing is that the mangle table doesn't work quite like the filter/nat tables I've used in the past.

From my (limited) experience with the filter/nat tables, those tend to stop processing as soon as a matching rule is found.  In otherwords if the packet being processed is for 192.168.1.2 and the top of the chain has a rule that matches all packets from 192.168.1.2, then that rule is processed (DROP, ACCEPT, logaccept, etc.) and nothing below it has any effect.

In contrast, from what I've seen on the mangle table the rules jumping to the "MARK" target do not stop the remaing rules in the chain from processing.  So, if a packet for 192.168.1.2 is being processed and the first rule in the PREROUTING chaing says mark all 192.168.1.2 traffic with mark 0x7, but then the second rule says clear all the marks and set mark 0x8000, then the both rules are run, but the second rule essentially wiped out the marks set by the first rule.

So, my problem was that I was using:


iptables -I PREROUTING -t mangle -s 192.168.1.5 -j MARK --set-mark 0x7


Which inserted (-I) my VPN mark (0x7) rule at the top of the PREROUTING chain on the mangle table allowing all the other rules below it in the chain to further manipulate the marks before being routed.

Once I changed "-I" to "-A" to append my rule the bottom of the list, everything started working as expected and VPN traffic started getting routed to table 200.

 

On a side note... Unfortunately, OpenVPN seems to be a little heavy for my use case.  Twice now I've had the router slow down to a snails pace (it was literally taking 10 seconds or more to respond to keystrokes in an SSH session) and it has been remedied by killing the openvpn process that was being used to route traffic over the VPN connection. frown

So, I guess it's back to the drawing board for me.  But at least I learned a little more about iptables.

david.woodward
david.woodward's picture
Figured out the openvpn

Figured out the openvpn slowness issue.  I had forgotten to turn off logging for EVERYTHING going over VPN (useful while I was troubleshooting).  So, the router was flipping out trying to log all the traffic going over the VPN.

Once I disabled the logging, all is well and my setup is working perfectly.  All traffic from a specific Ip address goes over the VPN with the exception of certain traffic I have flagged to go over the straight ISP connection.