[Click] performance experiment problems - low forwarding rates

Wed May 19 01:23:53 EDT 2004

Thanks for the suggestions, we've tried FastUDPSource and now we can
generate up to 400,000 pkts/sec, which seems reasonable.
Yet, that does not explain that we can only get ~240,000 pkts/sec for
the Simple
configuration; you say yourself that with similar hardware you were
able to do
~360,000 pps for the IP router configuration which is substantially
more
complex than ours.

The Simple configuration we are using just polls the device for
incoming
packets, stores them in a queue, and transmits them to the destination
without any processing whatsoever. What we are seeing is that the
queue overflows very quickly and a lot of packets get dropped.

I'll try and figure out something useful using the /proc/net
handlers...

nikitas

----- Original Message ----- 
From: "Giorgio Calarco" <gcalarco at deis.unibo.it>
To: "Nikitas Liogkas" <nikitas at CS.UCLA.EDU>
Cc: <click at amsterdam.lcs.mit.edu>
Sent: Tuesday, May 18, 2004 12:07 am
Subject: Re: [Click] performance experiment problems - low forwarding
rates

> Nikitas,
>
> it should be quite difficult to experience PCI bus performance
limitations
> if you use 64-byte packets, this should happen only when
> you increase the packet lenght.
> This is particularly true with your sources PCs, keep in mind that
> you have around 4 Gb/s of bus bandwidth
> (which is lower than the the nic bandwidth...1Gb/s).
>
> However, some of your values seem quite resonable:
> with a 800MHz pentium + PCI1.0 bus as a source we could
> generate around 170-180kpps...thus, if you can
> reach 280kpps with your Athlon you should consider that
> quite normal (if you want to speed up, actually we are reaching
> more than 700kpps with a 3GHz CPU + PCIX bus ... +
> FastUDPSource ).
>
> The proc filesystem also exports some interesting handlers
> for monitoring the PRO1000 activity (take a look inside the
> /proc/net/PRO_LAN_Adapters directory), like FIFO errors, this could
help
> you understanding if you are losing packets inside the NIC
> FIFO. If so, the NIC is not receiving DMA descriptors
> fast enough (this means that you have CPU-memory
> limitations)
>
> The router:
> with similar hardware (1.6GHz pentium IV+ PCI 1.0 bus) we could
> forward around 350-370Kpps (with a RFC1812 router config,
> thus having more packet processing than you... but we
> were using a bit faster CPU).
> I would also try using the PRO1000XT or MT Server
> Adapters instead of the dual port to see if you can go beyond
> (even if they have a PCI-X interface, you can also plug them
> on a standard PCI bus).
>
>
> ciao
> giorgio
>
>
> PS: if you have full text access to the LNCS on-line library,
> our past testbed + results was better described here:
>
> Volume 2720 / 2003
> Title:  High-Speed Networks and Multimedia Communications:
> 6th IEEE International Conference, HSNMC 2003, Estoril,
> Portugal, July 23-25, 2003. Proceedings
>
>
> ----- Original Message ----- 
> From: "Nikitas Liogkas" <nikitas at CS.UCLA.EDU>
> To: <click at amsterdam.lcs.mit.edu>
> Sent: Tuesday, May 18, 2004 6:48 AM
> Subject: [Click] performance experiment problems - low forwarding
rates
>
>
> > Hi to all!
> >
> > We've been trying to set up a testbed here at UCLA, in order to
run
> > Click and measure some optimizations we have implemented. So,
> > following the example set forth by the TOCS and ASPLOS paper, we
> > have two machines serving as sources, two serving as sinks, and
one
> > router that is receiving packets from the sources and is supposed
> > to deliver them to the sinks (machine configurations are at the
> > end of this email). The sources and sinks are connected with
gigabit
> > ethernet point-to-point links to the router, and we are running
the
> > latest CVS version of Click. Here are the machine configurations:
> >
> > - sources and sinks: AMD Athlon 1,666 MHz with Intel PRO/1000
> > Desktop Adapter gigabit ethernet cards (with Click's polling
> > extensions) on a PCI 64bit/66MHz bus running Linux 2.4.21
> > (patched with Click's patch)
> >
> > - router: Pentium III 1,266 MHz with two Intel PRO/1000 Dual
> > Port Server Adapter gigabit ethernet cards (with Click's
> > polling extensions) on a single PCI 32bit/33MHz bus
> > running Linux 2.4.21 (patched with Click's patch)
> >
> > The problem is that we are experiencing very low forwarding rates
> > (when I say packets below, I mean 64-byte packets).
> > For example, running the Simple configuration, which just forwards
> > packets without any processing, we can get up to 240,000 pkts/sec,
> > while the TOCS paper reports 452,000 pkts/sec for a similar
machine
> > to ours. The polling extensions for the gigabit cards seem to be
working
> (we
> > get a e1000_poll_on chatter when the router starts up).
> >
> > Another (maybe related) problem is that the sources sending rate
> > saturates around 280,000 pkts/sec. If we go beyond that, we start
> > experiencing drops on the ToDevice which is sending out the
packets
> > (PCI bus limitation?) Surprisingly, in the ASPLOS paper
> > it is mentioned that the sources in their experiments, which were
arguably
> > weaker than our sources, can generate 1 million pkts/sec each!
> >
> > Has anyone else experienced something similar?
> > Any ideas/suggestions will be greatly appreciated :-)
> >
> > Nikitas Liogkas
> > University of California - Los Angeles
> > _______________________________________________
> > click mailing list
> > click at amsterdam.lcs.mit.edu
> > https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>