[Click] Queue Memory Allocation: vmalloc or kmalloc

[Cliff Frey]

if you look at click/lib/glue.cc at the implementation of click_lalloc, you
can see that it will use vmalloc for allocations that are larger than 128kb,
so that would be queue size of 32k for 32 bit machines, or 16k for 64 bit
machines.

I don't believe that ThreadSafeQueue is necessary as long as you have only
one producer and one consumer.

Also, this probably isn't necessary to say, but I really hope that you have
compiled a 64 bit kernel.  All packet memory has to be mapped into the
kernel's virtual address space, so if you want more than ~1-2GB of packet
memory allocated allocated at the same time, you *must* be using a 64 bit
architecture (not just using PAE).

Cliff

On Thu, Jul 1, 2010 at 1:42 PM, Latency Buster <latencybuster at gmail.com>wrote:

> Thanks.. The 'problem' is that I am not able to find out whether the
> queue  element is storing the packets via vmalloc or kmalloc. I
> observed that after 700,000 packets (for my system), y the drain rate
> of the queue is decreasing rapidly under constant service rate...
>
> Also, do I need to use ThreadSafeQueue for a multicore system? There
> is only one producer pushing into the queue and only one consumer..
> But both the producer and consumer elements have been mapped to
> different cores.
>
> On Thu, Jul 1, 2010 at 4:00 PM,  <rchertov at cs.ucsb.edu> wrote:
> > Not sure on the largest size array that you can allocate in Click, but if
> > the Queue element complains about the size, you should be able to just
> link
> > the packets back to back using "next" defined in the Packet class.  That
> > will require making a custom element, but it should do the trick.
> >
> > Roman
> >
> > On 12:44 pm 07/01/10 Latency Buster <latencybuster at gmail.com> wrote:
> >> I have a machine with 32GB Ram... I am testing a scenario to find out
> >> how long a burst can click handle based on varying service rate. If I
> >> make the incoming queue length = 1.5 million packets, will click
> >> allocate the memory using kmalloc or vmalloc? Since I have a machine
> >> with large ram (32GB), I was thinking of allocating 2GB for packet
> >> queues. Is that feasible with the current version of Click?
> >>
> >> Thanks,
> >> _______________________________________________
> >> click mailing list
> >> click at amsterdam.lcs.mit.edu
> >> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> >
> >
>
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>