[Click] Outgoing traffic uses one network interface (out of two) only

[Eddie Kohler]

Hi Quentin,

This was indeed a bug.  We fixed it a little while ago; if you were following 
the mailing list you may have seen the fix, but I believe it should work on 
the current Git version of Click.

Eddie


On 8/8/11 9:38 AM, Quentin Monnet wrote:
> Hello,
>
> I have the following network configuration :
> Host1 (H1)<----- 10.0.0.0/24 ----->  Router1 (R1)<----- 192.168.1.0/24
> ----->  Router2 (R2)<----- 10.0.1.0/24 ----->  Host2 (H2)
>
> I would like to run (at userlevel) the canonical IP router from the SOSP
> paper on R1 and R2, so as to ping H2 with H1.
> I use the same configuration file for R1 and R2 (changing only IP and
> MAC addresses).
>
> Routing works well on R1, but I have something wrong with R2. When
> debugging with tcpdump, it seems that the whole outgoing traffic goes
> through the first network interface declared with "FromDevice".
> I mean that if my configuration file contains:
>
> "FromDevice(eth0) ->  [0]c0;
> FromDevice(eth1) ->  [0]c1;"
>
> then all the outgoing packets are sent to eth0, whatever network
> interface I give to "ToDevice".
> If my file contains:
>
> "FromDevice(eth1) ->  [0]c1;
> FromDevice(eth0) ->  [0]c0;"
>
> then everything goes to eth1.
>
> My network seems to work well without Click: I can ping H2 with H1. I
> have tried several machines and Click versions, but I can't figure out
> what is wrong.
>
> R1 is Ubuntu 10.10 with kernel 2.6.35 and Click version 1.8. There is no
> problem.
> R2: I tried Ubuntu 11.04 with kernel version 2.6.38 and Click version
> from git (it was 1.8 then).
> As I don't know where the problem comes from, I've tried with another
> router for R2: Ubuntu 9.04 with kernel 2.6.28 and Click version 2.0. I
> get the same behaviour. I can't see what the difference from R1 is.
> I am working at userlevel only.
>
> I give the Click configuration file I use for R1 below. R2 is the same
> with modified IP and MAC addresses.
> Thanks to anyone who could help.
>
> Regards,
> Quentin
>
> --------------------------------------------------------------------------------------
>
> // Kernel configuration for cone as a router between
> // 192.168.1 (eth0) and 10.0.0 (vboxnet1).
> // Proxy ARPs for 10.0.0 on eth0.
>
> // eth0,        00:00:C0:AE:67:EF, 192.168.1.222
> // vboxnet1,    00:00:C0:4F:71:EF, 10.0.0.1
>
> // 0. ARP queries
> // 1. ARP replies
> // 2. IP
> // 3. Other
> // We need separate classifiers for each interface because
> // we only want proxy ARP on eth0.
> c0 :: Classifier(12/0806 20/0001,
>                   12/0806 20/0002,
>                   12/0800,
>                   -);
>
> c1 :: Classifier(12/0806 20/0001,
>                   12/0806 20/0002,
>                   12/0800,
>                   -);
>
>
> FromDevice(eth0) ->  [0]c0;
> FromDevice(vboxnet1) ->  [0]c1;
> FromHost(virtual) ->  [0]c0;
>
> out0 :: Queue(200) ->  ToDevice(eth0) ->  Print(eth0) ->  Discard;
> out1 :: Queue(200) ->  ToDevice(vboxnet1) ->  Print(vboxnet1) ->  Discard;
> tol :: ToHost(virtual);
>
> // An "ARP querier" for each interface.
> arpq0 :: ARPQuerier(192.168.1.222, 00:00:c0:ae:67:ef);
> arpq1 :: ARPQuerier(10.0.0.1, 00:00:c0:4f:71:ef);
>
> // Deliver ARP responses to ARP queriers as well as Linux.
> t :: Tee(3);
> c0[1] ->  t;
> c1[1] ->  t;
> t[0] ->  tol;
> t[1] ->[1]arpq0
> t[2] ->[1]arpq1
>
> // Connect ARP outputs to the interface queues.
> arpq0 ->  out0;
> arpq1 ->  out1;
>
> // Proxy ARP on eth0 for 18.26.7, as well as cone's IP address.
> ar0 :: ARPResponder(192.168.1.222        00:00:c0:ae:67:ef,
>                      10.0.0.1/24        00:00:c0:4f:71:ef);
> c0[0] ->  ar0 ->  out0;
>
> // Ordinary ARP on eth1.
> ar1 :: ARPResponder(10.0.0.1        00:00:c0:4f:71:ef);
> c1[0] ->  ar1 ->  out1;
>
> // IP routing table. Outputs:
> // 0: packets for this machine.
> // 1: packets for 18.26.4.
> // 2: packets for 18.26.7.
> // All other packets are sent to output 1, with 18.26.4.1 as the gateway.
> rt :: RadixIPLookup(    192.168.1.222/32        0,
>                          10.0.1.1/24        1,
>                          10.0.0.1/32        0,
>                          192.168.1.222/24        1,
>                          10.0.0.1/24        2);
>
> // Hand incoming IP packets to the routing table.
> // CheckIPHeader checks all the lengths and length fields
> // for sanity.
> ip ::   Strip(14)
>       ->  ciph :: CheckIPHeader(INTERFACES 192.168.1.222/24 10.0.0.1/24)
>       ->  [0]rt;
> c0[2] ->  Paint(1) ->  ip;
> c1[2] ->  Paint(2) ->  ip;
>
> // IP packets for this machine.
> // ToHost expects ethernet packets, so cook up a fake header.
> rt[0] ->  EtherEncap(0x0800, 1:1:1:1:1:1, 2:2:2:2:2:2) ->  tol;
>
> // These are the main output paths; we've committed to a
> // particular output device.
> // Check paint to see if a redirect is required.
> // Process record route and timestamp IP options.
> // Fill in missing ip_src fields.
> // Discard packets that arrived over link-level broadcast or multicast.
> // Decrement and check the TTL after deciding to forward.
> // Fragment.
> // Send outgoing packets through ARP to the interfaces.
> rt[1] ->  DropBroadcasts
>        ->  cp1 :: PaintTee(1)
>        ->  gio1 :: IPGWOptions(192.168.1.222)
>        ->  FixIPSrc(192.168.1.222)
>        ->  dt1 :: DecIPTTL
>        ->  fr1 :: IPFragmenter(1500)
>        ->  [0]arpq0;
>
> rt[2] ->  DropBroadcasts
>        ->  cp2 :: PaintTee(2)
>        ->  gio2 :: IPGWOptions(10.0.0.1)
>        ->  FixIPSrc(10.0.0.1)
>        ->  dt2 :: DecIPTTL
>        ->  fr2 :: IPFragmenter(1500)
>        ->  [0]arpq1;
>
> // DecIPTTL[1] emits packets with expired TTLs.
> // Reply with ICMPs. Rate-limit them?
> dt1[1] ->  ICMPError(192.168.1.222, timeexceeded) ->  [0]rt;
> dt2[1] ->  ICMPError(192.168.1.222, timeexceeded) ->  [0]rt;
>
> // Send back ICMP UNREACH/NEEDFRAG messages on big packets with DF set.
> // This makes path mtu discovery work.
> fr1[1] ->  ICMPError(10.0.0.1, unreachable, needfrag) ->  [0]rt;
> fr2[1] ->  ICMPError(10.0.0.1, unreachable, needfrag) ->  [0]rt;
>
> // Send back ICMP Parameter Problem messages for badly formed
> // IP options. Should set the code to point to the
> // bad byte, but that's too hard.
> gio1[1] ->  ICMPError(192.168.1.222, parameterproblem) ->  [0]rt;
> gio2[1] ->  ICMPError(192.168.1.222, parameterproblem) ->  [0]rt;
>
> // Send back an ICMP redirect if required.
> cp1[1] ->  ICMPError(192.168.1.222, redirect, host) ->  [0]rt;
> cp2[1] ->  ICMPError(10.0.0.1, redirect, host) ->  [0]rt;
>
> // Unknown ethernet type numbers.
> c0[3] ->  Discard;
> c1[3] ->  Discard;
>
>
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click