[Click] userlevel performance

[Eddie Kohler]

I've checked in this change -- why not.

Eddie


On 11/02/2010 09:54 AM, Roman Chertov wrote:
> Beyers,
>
> I just tested with the CAPTURE PCAP and it works great.  I also applied the
> patch that you posted.  I do agree that CAPTURE PCAP method should be a default
> option.  I wonder if anybody has any objections to changing the default CAPTURE
> method to PCAP.
>
> Roman
>
>
> On Sat, 30 Oct 2010 02:02:55 +0200 Beyers Cronje<bcronje at gmail.com>  wrote
>
>> Hi Roman,
>>
>> I've done a couple of tests. Here are my findings:
>>
>> 1) Using FromDevice with default Linux capture method:
>>
>> [root at slacker click]# userlevel/click conf/r.click -h ctr1.count h
>> ctr2.count
>> ctr1.count: 100000
>> ctr2.count: 12980
>>
>> 2) Increasing /proc/sys/net/core/rmem_max
>> and  /proc/sys/net/core/rmem_default to 1048568 yields better results, but
>> still very poor:
>>
>> [root at slacker click]# userlevel/click conf/r.click -h ctr1.count -h
>> ctr2.count
>> ctr1.count: 100000
>> ctr2.count: 58263
>>
>> 3) And finally, using PCAP capture method, i.e. FromDevice(eth1, SNIFFER
>> false, PROMISC true, CAPTURE PCAP)
>>
>> [root at slacker click]# userlevel/click conf/r.click -h ctr1.count -h
>> ctr2.count
>> ctr1.count: 100000
>> ctr2.count: 100000
>>
>>
>> My conclusion is that the default FromDevice capture method utilizing Linux
>> packet socket RX processing is pretty crap. I'm sure this is not because of
>> Click but rather how the kernel handles it. On the other hand the PCAP
>> capture method seems to perform really well. Maybe we should look at setting
>> the default method to PCAP for Linux as well?
>>
>> One last thing, the patch I just mailed to the list fixes an issue where
>> "SNIFFER false" was not honored and you ended up in sniffer mode, which
>> might affect performance.
>>
>> Beyers
>>
>>
>>
>> On Fri, Oct 29, 2010 at 1:37 AM, Roman Chertov<rchertov at cs.ucsb.edu>  wrote:
>>
>>> On Thu, 28 Oct 2010 13:58:50 -0700 Cliff Frey<cliff at meraki.com>  wrote
>>>
>>>> I don't have a setup that I can easily/quickly use to test.  All the
>>> same,
>>>> I'm curious what the interface statistics do during this time on both of
>>> the
>>>> interfaces (i.e. any errors?  how many packets transmitted/received)
>>>
>>> [local]# click test2.click -h ctr1.count -h ctr2.count -h fd.count -h
>>> fd.kernel_drops
>>> ctr1.count:
>>> 100000
>>>
>>> ctr2.count:
>>> 4503
>>>
>>> fd.count:
>>> 0
>>>
>>> fd.kernel_drops:
>>> ??
>>>
>>> ?? means that the number of kernel drops is unknown.
>>>
>>> [local]# ethtool -S eth2
>>> NIC statistics:
>>>      rx_packets: 9
>>>      tx_packets: 100011
>>>      rx_bytes: 2749
>>>      tx_bytes: 150401691
>>>
>>> [local]# ethtool -S eth3
>>> NIC statistics:
>>>      rx_packets: 100004
>>>      tx_packets: 17
>>>      rx_bytes: 150400641
>>>      tx_bytes: 3429
>>>
>>> According to ethtool, all of the 100000 packets were sent and received w/o
>>> issues (the extra packets are due to IPv6 when the interfaces came on).
>>>
>>> So it looks like that problem is somewhere between Linux and Click getting
>>> the
>>> packets to ToDevice.  I am curious what results Bejers will get.
>>>
>>> Roman
>>>
>>>>
>>>> Also, I'm curious what was going on inside of ToDevice and FromDevice.
>>>   The
>>>> linuxmodule version of these elements have a bunch of counters (visible
>>>> through the "calls" handler), it would be cool if the userlevel versions
>>> had
>>>> these things too (i.e. run_select_count, packets for both, write_errors,
>>>> empty_pulls for ToDevice,  recvfrom_errors, recvfrom_eagains for
>>> fromdevice)
>>>>
>>>> I doubt it, but you also might get different behavior if you add a Queue
>>>> between the ratedsource and the todevice.
>>>>
>>>> Cliff
>>>>
>>>> On Thu, Oct 28, 2010 at 10:55 AM, Roman Chertov<rchertov at cs.ucsb.edu
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I was curious if anybody could try this test for me using userlevel
>>> click.
>>>>>   I am
>>>>> running CentOS 5.5 2.6.18-194.17.1.el5, and I have two network cards
>>>>> connected
>>>>> with a single cable.  If I run the script below using userlevel click,
>>>>> FromDevice(eth3) will receive very few packets and the majority would
>>> end
>>>>> up
>>>>> dropped on input.
>>>>>
>>>>> src :: RatedSource(\<00>, LENGTH 1458, RATE 8000, LIMIT 100000)
>>>>>     ->  UDPIPEncap(10.0.1.1, 6667, 20.0.0.2, 6667)
>>>>>     ->  EtherEncap(0x0800, 00:30:48:F9:EA:7B, 00:17:cb:0d:f8:db)
>>>>>     ->  ctr1 :: AverageCounter
>>>>>     ->  ToDevice(eth2);
>>>>>
>>>>>
>>>>> fd :: FromDevice(eth3, SNIFFER false, PROMISC true)
>>>>>    ->  ctr2 :: AverageCounter
>>>>>    ->  Discard;
>>>>>
>>>>> This script just requires ethX names to be changed; otherwise, it can
>>> be
>>>>> run as
>>>>> is.
>>>>>
>>>>> [root]# click test2.click -h ctr1.count -h ctr2.count
>>>>> ctr1.count:
>>>>> 100000
>>>>>
>>>>> ctr2.count:
>>>>> 4482
>>>>>
>>>>> Roman
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> click mailing list
>>>>> click at amsterdam.lcs.mit.edu
>>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>>>>>
>>>
>>>
>>> _______________________________________________
>>> click mailing list
>>> click at amsterdam.lcs.mit.edu
>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>>>
>
>
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click