[Click] userlevel performance

[Beyers Cronje]

Hi Roman,

I've done a couple of tests. Here are my findings:

1) Using FromDevice with default Linux capture method:

[root at slacker click]# userlevel/click conf/r.click -h ctr1.count h
ctr2.count
ctr1.count: 100000
ctr2.count: 12980

2) Increasing /proc/sys/net/core/rmem_max
and  /proc/sys/net/core/rmem_default to 1048568 yields better results, but
still very poor:

[root at slacker click]# userlevel/click conf/r.click -h ctr1.count -h
ctr2.count
ctr1.count: 100000
ctr2.count: 58263

3) And finally, using PCAP capture method, i.e. FromDevice(eth1, SNIFFER
false, PROMISC true, CAPTURE PCAP)

[root at slacker click]# userlevel/click conf/r.click -h ctr1.count -h
ctr2.count
ctr1.count: 100000
ctr2.count: 100000


My conclusion is that the default FromDevice capture method utilizing Linux
packet socket RX processing is pretty crap. I'm sure this is not because of
Click but rather how the kernel handles it. On the other hand the PCAP
capture method seems to perform really well. Maybe we should look at setting
the default method to PCAP for Linux as well?

One last thing, the patch I just mailed to the list fixes an issue where
"SNIFFER false" was not honored and you ended up in sniffer mode, which
might affect performance.

Beyers



On Fri, Oct 29, 2010 at 1:37 AM, Roman Chertov <rchertov at cs.ucsb.edu> wrote:

> On Thu, 28 Oct 2010 13:58:50 -0700 Cliff Frey <cliff at meraki.com> wrote
>
> > I don't have a setup that I can easily/quickly use to test.  All the
> same,
> > I'm curious what the interface statistics do during this time on both of
> the
> > interfaces (i.e. any errors?  how many packets transmitted/received)
>
> [local]# click test2.click -h ctr1.count -h ctr2.count -h fd.count -h
> fd.kernel_drops
> ctr1.count:
> 100000
>
> ctr2.count:
> 4503
>
> fd.count:
> 0
>
> fd.kernel_drops:
> ??
>
> ?? means that the number of kernel drops is unknown.
>
> [local]# ethtool -S eth2
> NIC statistics:
>     rx_packets: 9
>     tx_packets: 100011
>     rx_bytes: 2749
>     tx_bytes: 150401691
>
> [local]# ethtool -S eth3
> NIC statistics:
>     rx_packets: 100004
>     tx_packets: 17
>     rx_bytes: 150400641
>     tx_bytes: 3429
>
> According to ethtool, all of the 100000 packets were sent and received w/o
> issues (the extra packets are due to IPv6 when the interfaces came on).
>
> So it looks like that problem is somewhere between Linux and Click getting
> the
> packets to ToDevice.  I am curious what results Bejers will get.
>
> Roman
>
> >
> > Also, I'm curious what was going on inside of ToDevice and FromDevice.
>  The
> > linuxmodule version of these elements have a bunch of counters (visible
> > through the "calls" handler), it would be cool if the userlevel versions
> had
> > these things too (i.e. run_select_count, packets for both, write_errors,
> > empty_pulls for ToDevice,  recvfrom_errors, recvfrom_eagains for
> fromdevice)
> >
> > I doubt it, but you also might get different behavior if you add a Queue
> > between the ratedsource and the todevice.
> >
> > Cliff
> >
> > On Thu, Oct 28, 2010 at 10:55 AM, Roman Chertov <rchertov at cs.ucsb.edu
> >wrote:
> >
> > > Hello,
> > >
> > > I was curious if anybody could try this test for me using userlevel
> click.
> > >  I am
> > > running CentOS 5.5 2.6.18-194.17.1.el5, and I have two network cards
> > > connected
> > > with a single cable.  If I run the script below using userlevel click,
> > > FromDevice(eth3) will receive very few packets and the majority would
> end
> > > up
> > > dropped on input.
> > >
> > > src :: RatedSource(\<00>, LENGTH 1458, RATE 8000, LIMIT 100000)
> > >    -> UDPIPEncap(10.0.1.1, 6667, 20.0.0.2, 6667)
> > >    -> EtherEncap(0x0800, 00:30:48:F9:EA:7B, 00:17:cb:0d:f8:db)
> > >    -> ctr1 :: AverageCounter
> > >    -> ToDevice(eth2);
> > >
> > >
> > > fd :: FromDevice(eth3, SNIFFER false, PROMISC true)
> > >   -> ctr2 :: AverageCounter
> > >   -> Discard;
> > >
> > > This script just requires ethX names to be changed; otherwise, it can
> be
> > > run as
> > > is.
> > >
> > > [root]# click test2.click -h ctr1.count -h ctr2.count
> > > ctr1.count:
> > > 100000
> > >
> > > ctr2.count:
> > > 4482
> > >
> > > Roman
> > >
> > >
> > > _______________________________________________
> > > click mailing list
> > > click at amsterdam.lcs.mit.edu
> > > https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> > >
>
>
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>