[Click] bighashmap ain't that big?

[Eddie Kohler]

Hi Marco,

The reason for that limit is a Linux kernel limit on the size of objects that 
can be dynamically allocated.  32767 buckets * 4 bytes = (131072 - 4) bytes, 
and 131072 is the biggest allocation Linux's slab allocator allows.

You can, however, allocate larger objects with CLICK_LALLOC.  So I've just 
checked in a patch that does this, allowing up to 4194303 buckets.

Eddie


Marco Canini wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi,
> I'm trying to understand why is there a limit of 32767 buckets in the
> implementation of HashMap (bighashmap.hh:76).
> I don't see a strong reason for that. In my case it actually becomes a
> very bad issue as I'm using a modified AggregateIPFlows to analyze a
> link with 1M+ of active TCP flows at any point in time.
> Clearly what happens there is that all the buckets become full pretty
> soon and the performance deteriorate to the point that the system drops
> packets.
> I'm going to try to set that limit to 1073741823 = 2^30 - 1 but I'm not
> sure about the implications.
> Does anyone know what was the initial reason to have such limit?
> Cheers
> 
> - --
> Marco Canini
> Ph.D. Candidate
> Department of Communication, Computer and System Sciences
> University of Genoa
> Via all'Opera Pia 13
> 16145 Genoa Italy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (MingW32)
> 
> iD8DBQFHhkhYK52QDm/mFCkRAs/0AKDZTz1sCfV9ni3+nWEjEecGnRtn2wCdGKZx
> Tp7z6J4eetfWVsh+pBjGlE4=
> =gGqH
> -----END PGP SIGNATURE-----
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click