[Click] SMP SpinLock and Deadlocks

[Eddie Kohler]

Thomas,

Long-delayed reply: I wonder if you compiled your kernel with preemption.  I 
bet you did.  Have you tried without it?

Eddie


Paine, Thomas Asa wrote:
> Beyers,
> 
> The last configuration used was...
> 
> ./configure --prefix=/backups/staging/click --disable-userlevel
> --enable-multithread=2 --enable-intel-cpu --disable-dynamic-linking
> 
> Adding the additional output in sync.hh of course generated what I would
> suspect...
> .
> .
> releasing someone else's lock - owner 0 current cpu 1
> releasing someone else's lock - owner 1 current cpu 0
> releasing someone else's lock - owner 0 current cpu 1
> releasing someone else's lock - owner 1 current cpu 0
> .
> .
>  
> 	As of now, I am not using any additional click-install
> parameters when the module is installed.  Adding threads would just
> aggravate the situation.  Likewise, since there is only one thread, I am
> not using the StaticThreadSched option to bind elements to threads.
> 	Right now I am using the sched_setaffinity() system call from
> another process to set the affinity for the click PID, gathered from
> /proc/click/threads after click is running, to keep the thread on one
> virtual CPU, and the system stable.
> 
> Thanks, 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
>    Thomas Paine (paineta at uwec.edu) 
>    University of Wisconsin - Eau Claire 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
>  
>  
> 
> ________________________________
> 
> From: Beyers Cronje [mailto:bcronje at gmail.com] 
> Sent: Saturday, September 16, 2006 6:27 AM
> To: Paine, Thomas Asa
> Cc: click at pdos.csail.mit.edu
> Subject: Re: [Click] SMP SpinLock and Deadlocks
> 
> 
> HI Thomas,
> 
> What is your click './configure' line used and also what click-install
> parameters are you specifying? Are you using StaticThreadSched in your
> click configuration? 
> 
> If you are using StaticThreadSched change the Spinlock::Release line to:
> 
> 
> click_chatter("releasing someone else's lock - owner %i current cpu %i",
> _owner, my_cpu);
> 
> This should help narrow down if the problem is in the push/pull path or
> timer/handler related.
> 
> Beyers 
> 
> 
> On 9/12/06, Paine, Thomas Asa <PAINETA at uwec.edu> wrote: 
> 
> 
> 	        I have a click package, which contains my custom
> elements.
> 	Those elements basically maintain maps of IP and data.  To
> protect the
> 	maps during read/write handler operations, I have wrapped the
> critical
> 	sections using the SpinLock found in the sync.hh.
> 	
> 	My typical usage...
> 	
> 	   ...
> 	   _lock.acquire();
> 	   bleMap->clear();
> 	   _lock.release();
> 	   ...
> 	
> 	
> 	   ...
> 	   _lock.acquire();
> 	   bleMap->insert(n, type); 
> 	   _lock.release();
> 	   ...
> 	
> 	
> 	
> 	However, I'm experiencing periodic deadlocks and am seeing these
> types
> 	of messages very frequently...
> 	
> 	        "chatter: releasing someone else's lock" ...which is
> coming from 
> 	the SpinLock
> 	
> 	I'm seeing these messages even when handlers are not being
> called and I
> 	nice it a bit...  "echo -19 > /click/priority", so I'm not sure
> of the
> 	scope of the problem.  Clearly calling _depth-- on the wrong
> lock would 
> 	lead to accounting problems with the semaphore and deadlock.
> 	        The configuration is currently running on an appliance
> using the
> 	latest CVS code and the 2.6.16.3 kernel.  The appliance is using
> an 
> 	Intel(R) Pentium(R) 4 CPU 3.20GHz with hyperthreading, and the
> kernel is
> 	running with SMP support.  This configuration has been run on
> SMP
> 	enabled machines before, but with 2 physical processors, and was
> still
> 	using the 2.4 kernel, so I don't want to make too many
> comparisons
> 	there.  I can say I have not experienced any behavior like this
> in the
> 	past with the my elements or with click.
> 	
> 	The click and custom packages were last configured using the
> following, 
> 	and I've tried several variations of config options to see if I
> could at
> 	least cause a change, but I haven't seemed to find what's
> broken.
> 	
> 	"./configure --prefix=/backups/staging/click
> --disable-userlevel" 
> 	
> 	At this point, I'm not quite sure where the problem may lie.  Is
> this an
> 	affinity problem with the click kernel module, SpinLock,
> hyperthreading,
> 	kernel compilation option, or am I just not configuring/using
> something 
> 	correctly?  Has anyone else seen these messages unexpectedly?
> 	
> 	
> 	
> 	Thanks,
> 	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 	   Thomas Paine (paineta at uwec.edu)
> 	   University of Wisconsin - Eau Claire 
> 	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 	
> 	
> 	_______________________________________________
> 	click mailing list
> 	click at amsterdam.lcs.mit.edu
> 	https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> 	
> 
> 
> 
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click