Linux can use multiple routing tables, which is convenient for providing different routes for specific networks based on many different metrics, such as the source address. For example, if we want to route traffic from 192.168.99.0/24 out the 172.17.22.1 default gateway, you could create a new table and route it as such:
# ip route add default via 172.17.22.1 dev eth7 table 100 # ip rule add from 192.168.99.0/24 lookup 100
Now imagine another scenario, where you wish to route traffic from 192.168.99.0/24 to an external network (the Internet), but 220.127.116.11/24 is (for some reason) link-local on your host. That is, an address like 18.104.22.168 is directly assigned to an adapter on your host. Linux tracks link-local connections through its ‘local’ routing table, and the ip rule’s show the preference order as:
# ip rule show 0: from all lookup local 32766: from all lookup main 32767: from all lookup default
You might think deleting and adding the ‘local’ rule above with a higher preference and placing your new rule above it would fix the problem, but I’ve tried it—and it doesn’t. Searching around shows that others have had the same problem.
So what to do? Use fwmark.
First, change local’s preference from 0 to 100:
ip rule del from all pref 0 lookup local ip rule add from all pref 100 lookup local
Next, mark all traffic from 192.168.99.0/24 with some mark, we are using “1”. Note that I am using OUTPUT because 192.168.99.0/24 is my local address. You might want PREROUTING if this is a forwarding host.
iptables -t mangle -s 22.214.171.124/24 -A OUTPUT -j MARK --set-mark 1
And finally add the rule that routes it through table 100:
# ip rule add fwmark 1 pref 10 lookup 100 # ip rule show 10: from all fwmark 0x1 lookup 100 100: from all lookup local 32766: from all lookup main 32767: from all lookup default # ip route flush cache
Now all locally generated traffic to 126.96.36.199/24 from 192.168.99.0/24 will head out 172.17.22.1 on eth7 through table 100, instead of being looked up in the ‘local’ table.