[Click] SMP Click - Question Task scheduling

[Beyers Cronje]

Hi Ashok,

As far as I can tell when you run PollDevice then ToDevice always gets
scheduled. Once scheduled PollDevice will push at most BURST number of
packets where BURST is 8 by default, and ToDevice will try and transmit
BURST number of packets where BURST is 16 by default.

It could happen that ToDevice can not enqueue the packet to the adapter TX
ring because it is full, which could lead to PollDevice pushing packets
faster than what ToDevice can transmit them. Check ToDevice's 'drops'
handler for this. In which case it might help increasing you adapter's TX
ringsize, the default with e1000 is 256, I typically increase this to the
maximum of 4096. You can also experiment with different settings for BURST
in PollDevice and ToDevice. Also play with Queue's capacity value.

Click also provides the ScheduleInfo element for you to manually tune
scheduling paramaters of each element. I am NOT sure if ScheduleInfo is
compatible with SMP Click, I have never tried it before.
See http://read.cs.ucla.edu/click/elements/scheduleinfo
So you could configure ToDevice to get scheduled more often than PollDevice.
This can obviously lead to the receiving adapater's RX ring getting full if
PollDevice is not scheduled fast/often enough to enqueue the packets to
Click. At least dropping at the RX adapter is a lot less expensive than
dropping at Click's Queue or the TX adapter, so this might be a "good"
thing.

Hope this helps a bit.

Beyers

On Feb 7, 2008 5:44 PM, Ashok Ambati <ashok.ambati at gmail.com> wrote:

> Hello,
>
> I am a new user. I am configuring the kernel module to route between 2
> interfaces. On a 2-CPU server, I am creating 2 threads - one polling eth0
> and transmitting to eth1 and the other polling eth1 and trasmitting to
> eth0.
> I do not understand how each thread switches between the 2 tasks so as to
> maintain a balance between the incoming and outgoing packets. For the eth1
> -> eth0 path where there is heavy traffic, the queue fronting the
> ToDevice(eth0) is overflowing. Am I doing something wrong?
>
> Here is my config -
>
>
> // Generated by make-ip-conf.pl
>
> // eth0 192.168.200.136 00:04:23:C5:5D:CA
>
> // eth1 9.47.83.136 00:04:23:C5:5D:CB
>
> // Shared IP input path and routing table
>
> ip :: Strip(14)
>
> -> CheckIPHeader(INTERFACES 192.168.200.136/255.255.255.0
> 9.47.83.136/255.255.255.0)
>
> -> rt :: StaticIPLookup(
>
> 192.168.200.136/32 0,
>
> 192.168.200.255/32 0,
>
> 192.168.200.0/32 0,
>
> 9.47.83.136/32 0,
>
> 9.47.83.255/32 0,
>
> 9.47.83.0/32 0,
>
> 192.168.200.0/255.255.255.0 1,
>
> 9.47.83.0/255.255.255.0 2,
>
> 255.255.255.255/32 0.0.0.0 0,
>
> 0.0.0.0/32 0,
>
> 0.0.0.0/0.0.0.0 9.47.83.1 2);
>
> // ARP responses are copied to each ARPQuerier and the host.
>
> arpt :: Tee(3);
>
> // Input and output paths for eth0
>
> c0 :: Classifier(12/0806 20/0001, 12/0806 20/0002, 12/0800, -);
>
> pd0 :: PollDevice(eth0);
>
> todevice0 :: ToDevice(eth0, 8);
>
> pd0 -> c0;
>
> out0 :: Queue(2000) -> todevice0;
>
> c0[0] -> ar0 :: ARPResponder(192.168.200.136/24 00:04:23:C5:5D:CA) ->
> out0;
>
> arpq0 :: ARPQuerier(192.168.200.136, 00:04:23:C5:5D:CA) -> out0;
>
> c0[1] -> arpt;
>
> arpt[0] -> [1]arpq0;
>
> c0[2] -> Paint(1) -> ip;
>
> c0[3] -> Print("eth0 non-IP") -> Discard;
>
> // Input and output paths for eth1
>
> c1 :: Classifier(12/0806 20/0001, 12/0806 20/0002, 12/0800, -);
>
> pd1 :: PollDevice(eth1);
>
> pd1 -> c1;
>
> todevice1 :: ToDevice(eth1, 8);
>
> out1 :: Queue(2000) -> todevice1;
>
> c1[0] -> ar1 :: ARPResponder(9.47.83.136/24 00:04:23:C5:5D:CB) -> out1;
>
> arpq1 :: ARPQuerier(9.47.83.136, 00:04:23:C5:5D:CB) -> out1;
>
> c1[1] -> arpt;
>
> arpt[1] -> [1]arpq1;
>
> c1[2] -> Paint(2) -> ip;
>
> c1[3] -> Print("eth1 non-IP") -> Discard;
>
> // Local delivery
>
> toh :: ToHost;
>
> arpt[2] -> toh;
>
> rt[0] -> EtherEncap(0x0800, 1:1:1:1:1:1, 2:2:2:2:2:2) -> toh;
>
> // Forwarding path for eth0
>
> rt[1] -> DropBroadcasts
>
> -> cp0 :: PaintTee(1)
>
> -> gio0 :: IPGWOptions(192.168.200.136)
>
> -> FixIPSrc(192.168.200.136)
>
> -> dt0 :: DecIPTTL
>
> -> fr0 :: IPFragmenter(1500)
>
> -> [0]arpq0;
>
> dt0[1] -> ICMPError(192.168.200.136, timeexceeded) -> rt;
>
> fr0[1] -> ICMPError(192.168.200.136, unreachable, needfrag) -> rt;
>
> gio0[1] -> ICMPError(192.168.200.136, parameterproblem) -> rt;
>
> cp0[1] -> ICMPError(192.168.200.136, redirect, host) -> rt;
>
> // Forwarding path for eth1
>
> rt[2] -> DropBroadcasts
>
> -> cp1 :: PaintTee(2)
>
> -> gio1 :: IPGWOptions(9.47.83.136)
>
> -> FixIPSrc(9.47.83.136)
>
> -> dt1 :: DecIPTTL
>
> -> fr1 :: IPFragmenter(1500)
>
> -> [0]arpq1;
>
> dt1[1] -> ICMPError(9.47.83.136, timeexceeded) -> rt;
>
> fr1[1] -> ICMPError(9.47.83.136, unreachable, needfrag) -> rt;
>
> gio1[1] -> ICMPError(9.47.83.136, parameterproblem) -> rt;
>
> cp1[1] -> ICMPError(9.47.83.136, redirect, host) -> rt;
>
> //CPU assignment for Polling and Target devices
>
> StaticThreadSched(pd0 0);
>
> StaticThreadSched(todevice0 1);
>
> StaticThreadSched(pd1 1);
>
> StaticThreadSched(todevice1 0);
>
> // ThreadMonitor();
>
> Thanks.
>
>
>
> Ashok.
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>