[Click] Click on Multi-core problem

[Eddie Kohler]

Hey Adam,

Have you added these papers to the wiki page for such things??

Eddie


On 7/13/11 2:50 AM, Adam Greenhalgh wrote:
> sorry for the shameless self plug but a number of these papers might
> be of use to you, they explain some of the issues you will be seeing
> with multi cpu issues and click.
>
> http://www.comp.lancs.ac.uk/~laurent/papers/egi_npc.pdf
>
> http://www.comp.lancs.ac.uk/~laurent/papers/high_perf_vrouters-CoNEXT08.pdf
>
> http://www.comp.lancs.ac.uk/~laurent/papers/fairness_vrouters-PRESTO08.pdf
>
> Adam
>
> On 12 July 2011 15:53, ahmed A.<amego83 at gmail.com>  wrote:
>> Hi,
>>
>> I am examining the forwarding performance of CLICK in our four core CPU
>> machine. I am assigning two different simple forwarding path to
>> two different core each time and watch the forwarding rate using a CLICK
>> counter. My CLICK configuration file is as follow :
>>
>> d1::PollDevice(eth24,PROMISC true,BURST 16) ->    queue1::Queue(10000) ->
>> c1::Counter ->  td1::ToDevice(eth22);
>> pd2::PollDevice(eth25,PROMISC true,BURST 16) ->  queue2::Queue(10000) ->
>> c2::Counter ->  td2::ToDevice(eth23);
>>
>>         Idle ->  ToDevice(eth24);
>>         Idle ->  ToDevice(eth25);
>>
>>
>>         StaticThreadSched(pd1 0,td1 0,pd2 1,td2 1);
>>
>>         CpuPin(0 0,1 1);
>>
>> I was expecting to have almost the same forwarding rate (counter rate) for
>> both paths whatever was the assigned cores, but actually I got different
>> results
>> depending on the cores that I use, for example when I use core 0 and 1, I
>> got *1.0 Million Packets Per Second MPPS* for core 0 and *1.42 MPPS* for
>> core 1. for core 1 and 2
>> I got *1.39 MPPS* for both cores. and for core 2 and 3 I got *1.42 MPPS* for
>> core 2 and core *1 MPPS* for core 2. In summary, there  alway are two cores
>> with bad performance comparing to the
>> other cores.
>>
>> By checking the *monitored_empty_polls_rate* of the cores, I found out that
>> the cores with bad performance has 0.781 monitored_empty_polls_rate whereas
>> the good
>> cores has 207111 *monitored_empty_polls_rate*. The number of dropped packets
>> in the NIC port assigned to the bad cores are much bigger than number of
>> dropped packets assigned to the good cores. My explanation is the Polldevice
>> is not
>> getting enough CPU cycles (i.e not scheduled enough) to poll packets and
>> refill the DMA ring with skb buffers, but I have no idea why ????
>>
>> Does the Linux scheduler interfere with  click ? I checked the load at each
>> core using *top* but I could not see any other processes running on the bad
>> cores, they are idle all the time.
>> I would appreciate any tips or help.
>>
>> thank you in advance
>>
>> Ahmed
>>
>> PS: I have 2.6.18 kernel running in fedora filesystem with CLICK 1.6 and
>> e1000 batched driver
>> _______________________________________________
>> click mailing list
>> click at amsterdam.lcs.mit.edu
>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>>
>
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click