Questions on ToLinux and ToLinuxSniffers Performance

[Henry Owen]

Question 1: Is it an expected result that using the element ToLinux and then
in another approach using ToLinuxSniffers in order to pass packets to a user
level sockets program (that basically does nothing but count packets)
results in the same (poor) performance?

Question 2: Is it against the philosophy of click to attempt to do any
processing in the user space (for example counting) since I see such a
performance penalty when attempting to pass packets to a simple sockets
program?

Question 3 (unrelated to the remainder of this email) How do I use the click
counter element's reset handler which is write only? I do not understand how
to clear the counter's count value without having to do another cat to
/proc/config which does a reset every time.

Thanks to anyone that can give me some advice.......

Measurements details:
Click 1.2.1  in Linux 2.2.19 with Intel Pro/1000 gigabit NIC using the
driver found in the CVS archive.

I have used PollDevice(eth1) -> Counter -> Discard to see that click allows
counting packets at wire speed while using an Intel e1000 gigabit NIC.  I
have been able to repeat the reported Intel Pro/1000 gigabit driver polling
configuration receiving 1.3 million 64 byte packets per second :>)

Next I attempted to count packets using a sockets program. I did this using
four different element configurations to get the packets to the sockets
program: 1) using interrupts passing to the ordinary Linux networking stack,
2) using polling passing to the ordinary Linux networking stack, 3) using
interrupts NOT passing to the ordinary Linux networking stack, 4) using
polling NOT passing to the ordinary Linux networking stack

The results below seem to indicate that there is a very heavy penalty for
passing packets to a socket based program that counts packets. In both test
cases when the  click counter element configurations were used counting
performance was excellent.

Test Case 1500:
1,000,000 packets at 1,500 bytes each at 300 mbits/sec from Smartbits
Elements Used                                              Number of packets
counted (using sockets)
1) FromDevice(eth1) -> ToLinux(eth1)               238,566
2) PollDevice(eth1) -> ToLinux(eth1)                 238,061
3) FromDevice(eth1) ->ToLinuxSniffers(eth1)   238,530
4) PollDevice(eth1) -> ToLinuxSniffers(eth1)    243,891

Using click counter element (no sockets user level program running). This
shows click has excellent performance when not passing anything to the user
space.
FromDevice(eth1) -> Counter -> Discard        1,000,000
PollDevice(eth1) -> Counter -> Discard          1,000,000


Test Case 64:
1,000,000 packets at 64 each at 300 mbits/sec from Smartbits
Elements Used                                              Number of packets
counted (using sockets)
1) FromDevice(eth1) -> ToLinux(eth1)               62
2) PollDevice(eth1) -> ToLinux(eth1)                 180
3) FromDevice(eth1) ->ToLinuxSniffers(eth1)   100
4) PollDevice(eth1) -> ToLinuxSniffers(eth1)    2882

Using click counter element (no sockets user level program running):
FromDevice(eth1) -> Counter -> Discard        953,065
PollDevice(eth1) -> Counter -> Discard          1,000,000


I see that almost the only difference between ToLinux and ToLinuxSniffers is
ptype_dispatch(skb, skb->protocol) and ptype_dispatch(skb, 0xFFFF) which
appears to allow packets to bypass the "remainder" of the ordinary Linux
networking stack. I would have thought there would be a performance gain by
using ToLinuxSniffers which bypasses the "remainder" of the ordinary Linux
networking stack as opposed to ToLinux which uses the ordinary Linux
networking stack.

Aside:
The main part of the sockets program I used to count the packets when not
using click's counter element (lots of stuff cut just to show the
fundamentals):

   s = socket (PF_INET, SOCK_PACKET, htons (ETH_P_ALL));
   r = sizeof (struct sockaddr);
   while (1)
   {
      r = recvfrom (s, buffer, sizeof (buffer), 0, &socket_info, &r);
      tot_pkts++;
   }