[Click] [Help!!!] SMP click problem

Mon Oct 24 12:09:30 EDT 2005

Beyers Cronje wrote:
> Hi Kim,
> 
> If memory serves me correctly someone picked up a SMP bug in click's sk_buff

There was such a patch in July/August I believe.

> recycle routines for Click 1.4.3. So I suggest you check out latest CVS
> click. Unfortunately I havent used latest cvs version, so I cant comment on
> how stable it is yet.

I have a strong suspicion that a good part of the problem occurs in the 
driver code itself.  In my case the driver crash would manifest itself 
when I had a 150K PPS UDP packet flow.  I did apply the synchronization 
patch but I still had the same problems.  I got click stable when I used 
version 1.4.3 and the e1000 5.x from July's CVS tree.

Roman

> 
> Regards
> 
> Beyers
> 
> On 10/19/05, 김병철 <protobird at chollian.net> wrote:
> 
>>Hi,
>>I am a newbie of Click and appreciate you for good router software.
>>
>>While non-SMP click does not occur any problem,
>>it seems SMP click can not guaranttee mutual exclusion during sk_buff
>>operation.
>>
>>Which versoins of kernel and click can you make sure SMP click runs
>>correctly?
>>
>>I am working on vanilla Linux Kernel 2.4.20, Dual Zeon-HT 2.0 GHz,
>>2 Intel Pro/1000 gigabit ethernet,
>>Click-1.4.3 configured with options '--enable-multithread=4'
>>launched with module parameter '--thread=4'
>>
>>Click configuration is as below;
>>
>>
>>==================================================================================
>>// Generated by make-ip-conf.pl
>>// eth0 192.168.3.94 <http://192.168.3.94> 00:02:B3:F3:18:50
>>// eth1 10.10.1.94 <http://10.10.1.94> 00:02:B3:F3:18:51
>>// eth1 192.168.2.94 <http://192.168.2.94> 00:02:B3:F3:18:51
>>
>>// Shared IP input path and routing table
>>ip :: Strip(14)
>>-> CheckIPHeader(INTERFACES 192.168.3.94/255.255.255.0<http://192.168.3.94/255.255.255.0>
>>10.10.1.94/255.255.255.0 <http://10.10.1.94/255.255.255.0>
>>192.168.2.94/255.255.255.0 <http://192.168.2.94/255.255.255.0>)
>>-> rt :: StaticIPLookup(
>>192.168.3.94/32 <http://192.168.3.94/32> 0,
>>192.168.3.255/32 <http://192.168.3.255/32> 0,
>>192.168.3.0/32 <http://192.168.3.0/32> 0,
>>10.10.1.94/32 <http://10.10.1.94/32> 0,
>>10.10.1.255/32 <http://10.10.1.255/32> 0,
>>10.10.1.0/32 <http://10.10.1.0/32> 0,
>>192.168.2.94/32 <http://192.168.2.94/32> 0,
>>192.168.2.255/32 <http://192.168.2.255/32> 0,
>>192.168.2.0/32 <http://192.168.2.0/32> 0,
>>192.168.3.0/255.255.255.0 <http://192.168.3.0/255.255.255.0> 1,
>>10.10.1.0/255.255.255.0 <http://10.10.1.0/255.255.255.0> 2,
>>192.168.2.0/255.255.255.0 <http://192.168.2.0/255.255.255.0> 3,
>>255.255.255.255/32 <http://255.255.255.255/32> 0.0.0.0 <http://0.0.0.0> 0,
>>0.0.0.0/32 <http://0.0.0.0/32> 0,
>>0.0.0.0/0 <http://0.0.0.0/0> 4);
>>
>>// ARP responses are copied to each ARPQuerier and the host.
>>arpt :: Tee(4);
>>
>>// Input and output paths for eth0
>>c0 :: Classifier(12/0806 20/0001, 12/0806 20/0002, 12/0800, -);
>>//in0 :: PollDevice(eth0) -> c0;
>>in0 :: FromDevice(eth0) -> c0;
>>out0 :: Queue(10240) -> todevice0 :: ToDevice(eth0);
>>c0[0] -> ar0 :: ARPResponder(192.168.3.94 <http://192.168.3.94>00:02:B3:F3:18:50) -> out0;
>>arpq0 :: ARPQuerier(192.168.3.94 <http://192.168.3.94>, 00:02:B3:F3:18:50)
>>-> out0;
>>c0[1] -> arpt;
>>arpt[0] -> [1]arpq0;
>>//c0[2] -> Paint(1) -> ip;
>>//c0[2] -> Paint(1) -> srchash :: HashSwitch (6, 6);
>>c0[2] -> Paint(1) -> srchash :: RoundRobinSwitch;
>>c0[3] -> Discard
>>// load balancing with Ethernet source address
>>srchash [0] -> Queue (10240) -> thread0 :: Unqueue -> ip;
>>srchash [1] -> Queue (10240) -> thread1 :: Unqueue -> ip;
>>
>>
>>// Input and output paths for eth1
>>c1 :: Classifier(12/0806 20/0001, 12/0806 20/0002, 12/0800, -);
>>//in1 :: PollDevice(eth1) -> c1;
>>in1 :: FromDevice(eth1) -> c1;
>>out1 :: Queue(10240) -> todevice1 :: ToDevice(eth1);
>>c1[0] -> ar1 :: ARPResponder(10.10.1.94 <http://10.10.1.94> 192.168.2.94<http://192.168.2.94>00:02:B3:F3:18:51) -> out1;
>>arpq1 :: ARPQuerier(10.10.1.94 <http://10.10.1.94>, 00:02:B3:F3:18:51) ->
>>out1;
>>arpq2 :: ARPQuerier(192.168.2.94 <http://192.168.2.94>, 00:02:B3:F3:18:51)
>>-> out1;
>>c1[1] -> arpt;
>>arpt[1] -> [1]arpq1;
>>arpt[2] -> [1]arpq2;
>>c1[2] -> Paint(2) -> ip;
>>c1[3] -> Discard
>>
>>// Local delivery
>>toh :: ToHost;
>>arpt[3] -> toh;
>>rt[0] -> EtherEncap(0x0800, 1:1:1:1:1:1, 2:2:2:2:2:2) -> toh;
>>
>>// Forwarding path for eth0
>>rt[1] -> DropBroadcasts
>>-> cp0 :: PaintTee(1)
>>-> gio0 :: IPGWOptions(192.168.3.94 <http://192.168.3.94>)
>>-> FixIPSrc(192.168.3.94 <http://192.168.3.94>)
>>-> dt0 :: DecIPTTL
>>-> fr0 :: IPFragmenter(1500)
>>-> [0]arpq0;
>>dt0[1] -> ICMPError(192.168.3.94 <http://192.168.3.94>, timeexceeded) ->
>>rt;
>>fr0[1] -> ICMPError(192.168.3.94 <http://192.168.3.94>, unreachable,
>>needfrag) -> rt;
>>gio0[1] -> ICMPError(192.168.3.94 <http://192.168.3.94>, parameterproblem)
>>-> rt;
>>cp0[1] -> ICMPError(192.168.3.94 <http://192.168.3.94>, redirect, host) ->
>>rt;
>>
>>// Forwarding path for eth1
>>rt[2] -> DropBroadcasts
>>-> cp1 :: PaintTee(2)
>>-> gio1 :: IPGWOptions(10.10.1.94 <http://10.10.1.94>)
>>-> FixIPSrc(10.10.1.94 <http://10.10.1.94>)
>>-> dt1 :: DecIPTTL
>>-> fr1 :: IPFragmenter(1500)
>>-> [0]arpq1;
>>dt1[1] -> ICMPError(10.10.1.94 <http://10.10.1.94>, timeexceeded) -> rt;
>>fr1[1] -> ICMPError(10.10.1.94 <http://10.10.1.94>, unreachable, needfrag)
>>-> rt;
>>gio1[1] -> ICMPError(10.10.1.94 <http://10.10.1.94>, parameterproblem) ->
>>rt;
>>cp1[1] -> ICMPError(10.10.1.94 <http://10.10.1.94>, redirect, host) -> rt;
>>
>>// Forwarding path for eth1
>>rt[3] -> DropBroadcasts
>>-> cp2 :: PaintTee(3)
>>-> gio2 :: IPGWOptions(192.168.2.94 <http://192.168.2.94>)
>>-> FixIPSrc(192.168.2.94 <http://192.168.2.94>)
>>-> dt2 :: DecIPTTL
>>-> fr2 :: IPFragmenter(1500)
>>-> [0]arpq2;
>>dt2[1] -> ICMPError(192.168.2.94 <http://192.168.2.94>, timeexceeded) ->
>>rt;
>>fr2[1] -> ICMPError(192.168.2.94 <http://192.168.2.94>, unreachable,
>>needfrag) -> rt;
>>gio2[1] -> ICMPError(192.168.2.94 <http://192.168.2.94>, parameterproblem)
>>-> rt;
>>cp2[1] -> ICMPError(192.168.2.94 <http://192.168.2.94>, redirect, host) ->
>>rt;
>>
>>rt[4] -> Discard;
>>
>>//StaticThreadSched (in0 0);
>>//StaticThreadSched (out1 0);
>>//StaticThreadSched (in1 1);
>>//StaticThreadSched (out0 1);
>>//StaticThreadSched (thread0 2);
>>//StaticThreadSched (thread1 3);
>>
>>//ThreadMonitor ();
>>
>>==================================================================================
>>
>>Thansks in advance
>>
>>Kim, Byungchul
>>
>>
>>
>>
>>_______________________________________________
>>click mailing list
>>click at amsterdam.lcs.mit.edu
>>https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>>
>>
>>
>>------------------------------------------------------------------------
>>
>>_______________________________________________
>>click mailing list
>>click at amsterdam.lcs.mit.edu
>>https://amsterdam.lcs.mit.edu/mailman/listinfo/click