[Click] Click on Multi-core problem

[Adam Greenhalgh]

sorry for the shameless self plug but a number of these papers might
be of use to you, they explain some of the issues you will be seeing
with multi cpu issues and click.

http://www.comp.lancs.ac.uk/~laurent/papers/egi_npc.pdf

http://www.comp.lancs.ac.uk/~laurent/papers/high_perf_vrouters-CoNEXT08.pdf

http://www.comp.lancs.ac.uk/~laurent/papers/fairness_vrouters-PRESTO08.pdf

Adam

On 12 July 2011 15:53, ahmed A. <amego83 at gmail.com> wrote:
> Hi,
>
> I am examining the forwarding performance of CLICK in our four core CPU
> machine. I am assigning two different simple forwarding path to
> two different core each time and watch the forwarding rate using a CLICK
> counter. My CLICK configuration file is as follow :
>
> d1::PollDevice(eth24,PROMISC true,BURST 16) ->  queue1::Queue(10000) ->
> c1::Counter -> td1::ToDevice(eth22);
> pd2::PollDevice(eth25,PROMISC true,BURST 16) -> queue2::Queue(10000) ->
> c2::Counter -> td2::ToDevice(eth23);
>
>        Idle -> ToDevice(eth24);
>        Idle -> ToDevice(eth25);
>
>
>        StaticThreadSched(pd1 0,td1 0,pd2 1,td2 1);
>
>        CpuPin(0 0,1 1);
>
> I was expecting to have almost the same forwarding rate (counter rate) for
> both paths whatever was the assigned cores, but actually I got different
> results
> depending on the cores that I use, for example when I use core 0 and 1, I
> got *1.0 Million Packets Per Second MPPS* for core 0 and *1.42 MPPS* for
> core 1. for core 1 and 2
> I got *1.39 MPPS* for both cores. and for core 2 and 3 I got *1.42 MPPS* for
> core 2 and core *1 MPPS* for core 2. In summary, there  alway are two cores
> with bad performance comparing to the
> other cores.
>
> By checking the *monitored_empty_polls_rate* of the cores, I found out that
> the cores with bad performance has 0.781 monitored_empty_polls_rate whereas
> the good
> cores has 207111 *monitored_empty_polls_rate*. The number of dropped packets
> in the NIC port assigned to the bad cores are much bigger than number of
> dropped packets assigned to the good cores. My explanation is the Polldevice
> is not
> getting enough CPU cycles (i.e not scheduled enough) to poll packets and
> refill the DMA ring with skb buffers, but I have no idea why ????
>
> Does the Linux scheduler interfere with  click ? I checked the load at each
> core using *top* but I could not see any other processes running on the bad
> cores, they are idle all the time.
> I would appreciate any tips or help.
>
> thank you in advance
>
> Ahmed
>
> PS: I have 2.6.18 kernel running in fedora filesystem with CLICK 1.6 and
> e1000 batched driver
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>