[Click] Userlevel performance issues

[Robert Ross]

The only clear item that seems to have a marked difference is the
LinkUnqueue element.  The fact that our ToDevice and FromDevice/Socket
performance appears to be related somehow to the configuration of a
LinkUnqueue element sitting in the middle of our configuration is too
obvious to ignore.  Does LinkUnqueue perform some kind of
upstream/downstream notification to these elements, causing them to
throttle their behavior based on LinkUnqueue?
 
In our tests, with all other elements remaining the same, here is what
we found from two independent read handler counts:
 
LinkUnqueue("512Kbps") = Maximum ~190 packets/second pushed from the
Socket element and pulled by the ToDevice element
LinkUnqueue("1Mbps") = Maxmum ~290 packets/second pushed from the Socket
element and pulled by the ToDevice element
LinkUnqueue("2Mbps") = Maximum ~490 packets/second pushed from the
Socket element and pulled by the ToDevice element
LinkUnqueue("4Mbps") = Maximum ~780 packets/second pushed from the
Socket element and pulled by the ToDevice element
LinkUnqueue("6Mbps") = Maximum ~980 packets/second pushed from the
Socket element and pulled by the ToDevice element
LinkUnqueue("8Mbps") = Maximum ~1360 packets/second pushed from the
Socket element and pulled by the ToDevice element
 
It is also telling that independant handler counters corroborate exactly
the same maximum packets per second in two very different places in the
configuration.  Clearly you can see that the limitation on processing is
completely artificial and not an actual performance problem, since
increasing LinkUnqueue increases the performance in a very controlled
and obvious manner.
 
I have attached a simple configuration that examines specific handlers
and outputs values each second to a CSV file for analysis.  The
configuration is scaled back to complete simplicity, yet has the same
performance as our actual configuration which has a much more
complicated configuration.  Nevertheless, the performance is identical
and seems to point squarely at LinkUnqueue.
 
What is LinkUnqueue doing that could be causing this type of effect on
FromHost, Socket and ToDevice?  


________________________________

From: Robert Ross 
Sent: Friday, January 25, 2008 7:40 PM
To: 'Beyers Cronje'
Cc: click at pdos.csail.mit.edu
Subject: RE: [Click] Userlevel performance issues


Sorry, I wasn't clear that the queues are necessary for our
configuration.  The configuration is somewhat complex.  I was only
attempting to highlight the important parts.
 
 


________________________________

From: Beyers Cronje [mailto:bcronje at gmail.com] 
Sent: Friday, January 25, 2008 7:31 PM
To: Robert Ross
Cc: click at pdos.csail.mit.edu
Subject: Re: [Click] Userlevel performance issues


Hi Robert,


 

	*       We first found that when UserLevel Click started pulling
from a
	PCAP file, the performance of the ToDevice() appeared to drop
sharply.
	What I mean by this is that the ToDevice() pull handler reported
values
	in the range of 200 packets/second once the PCAP file started
reading.
	This resulted in the outbound queue just prior to the ToDevice()
filling
	up and eventually overflowing because the packet rate in the
PCAP file
	is far more than 200 packets/second.


You dont have to use a queue between FromDump and ToDevice as FromDump
is an agnostic element. In other words you can connect Todevice directly
to FromDump which should ensure that at least no packets are dropped and
you should see best ToDevice performance.

Also there are a few tuning parameters. Try tuning your NIC TX Ring
size. On the e1000 driver the default TX ring size is 256, experiment
with different value to see if it makes a difference.ToDevice uses a
packet socket from transmit, so it might be worth experimenting with
/proc/sys/net/core/wmem_default
/proc/sys/net/core/wmem_max


Beyers

-------------- next part --------------
A non-text attachment was scrubbed...
Name: simple test.click
Type: application/octet-stream
Size: 3478 bytes
Desc: simple test.click
Url : https://pdos.csail.mit.edu/pipermail/click/attachments/20080126/736b59a0/attachment.obj