[Click] Repeatable way to kernel panic via Click?
dmoore7@nd.edu
dmoore7 at nd.edu
Mon Aug 27 12:01:28 EDT 2007
Eddie,
The major re-occuring issue was a wrong-cpu error that then cascaded into other
things. Using the latest CVS click I have not had repeatable errors since.
I believe I crashed it once since installing the new source, but that error did
not repeat, and may have been due to an overly large configuration file. I am
not in a position to push boundary conditions like that currently: I don't have
physical access to my machines generally so I can't afford to crash them for
testing at the moment.
Thanks,
- David
Quoting Eddie Kohler <kohler at cs.ucla.edu>:
> David,
>
> Can you reproduce the problem with current CVS Click?
>
> There is a possibility that one of the following patches, which are not in
> 1.5.0 (but is in the current CVS), fixed this problem.
>
>
> 2007-02-05 Monday 15:11 Eddie Kohler <kohler at cs.ucla.edu>
>
> * elements/linuxmodule/: anydevice.cc (1.35), anydevice.hh (1.43),
> fromdevice.cc (1.94), fromdevice.hh (1.45), fromhost.cc (1.55),
> polldevice.cc (1.144), todevice.cc (1.151), todevice.hh (1.75),
> tohost.cc (1.50), tulipstats.cc (1.24): add safer locking to
> AnyDeviceMap manipulations: we need to lock on BOTH reading and
> writing. Hope this causes no deadlocks. John Bicket reported a
> problem that sent me on this trail in 6/2006
>
> 2007-02-03 Saturday 18:06 Eddie Kohler <kohler at cs.ucla.edu>
>
> * linuxmodule/: clickfs.cc (1.53), config.cc (1.25): avoid "wrong
> cpu" problems in click-install by not holding any spinlock_t
> objects for long periods
>
>
> Eddie
>
>
> dmoore7 at nd.edu wrote:
> > Did some testing with a second machine, same hardware as first. I'm
> getting
> > crashes/panics in the same way. Both machines are recompiled vanilla
> kernels
> > with preemption turned off. However on another machine (a mac-mini) the
> same
> > click config loads without errors. I'm going to fiddle with them
> physically
> > and see if I can get somewhere.
> >
> > If anyone cares, here is the dump left when click crashed. Every line was
> > proceeded by the "Message from..." line, I took those out for readability.
> The
> > config file in question is posted below.
> > For a graphical view of what this config does, check out:
> > cse.nd.edu/~dmoore7/myrouter.jpg
> > ------------------------------------
> > [root at firenze ~]# click-install ./Firenze-base.click
> > Segmentation fault
> > [root at firenze ~]#
> > Message from syslogd at firenze at Thu Aug 23 16:49:31 2007 ...
> > firenze kernel: Oops: 0002 [#1]
> > firenze kernel: SMP
> > firenze kernel: CPU: 0
> > firenze kernel: EIP is at _ZN4Task17strong_rescheduleEv+0x12/0x1a0 [click]
> > firenze kernel: eax: f543d8d4 ebx: 00000001 ecx: 00000000 edx:
> f78b3000
> > firenze kernel: esi: f543d8d4 edi: f692e000 ebp: 00000000 esp:
> f692ecb8
> > firenze kernel: ds: 007b es: 007b ss: 0068
> > firenze kernel: Process click-install (pid: 3962, threadinfo=f692e000
> > task=f78b3000)
> > firenze kernel: Stack: <0>f8affbaa 00000001 f7015000 f692ecd4 f8b005c7
> f692ecd4
> > 00ffffff f57a55e0
> > firenze kernel: 00000001 00000004 f7015000 f8c79ac8 f7015000
> 00000001
> > c02e28f6 f7015000
> > firenze kernel: 00000000 00001002 c028a6b9 f7015000 00001043
> c028b916
> > f692ed2c 00000001
> > firenze kernel: Call Trace:
> > firenze kernel: [<f8affbaa>]
> > _ZN8ToDevice13change_deviceEP10net_device+0x2a/0x60 [click]
> > firenze kernel: [<f8b005c7>] device_notifier_hook+0x77/0x90 [click]
> > firenze kernel: [<c02e28f6>] notifier_call_chain+0x17/0x2e
> > firenze kernel: [<c028a6b9>] dev_open+0x66/0x6d
> > firenze kernel: [<c028b916>] dev_change_flags+0x48/0xed
> > firenze kernel: [<c028c14b>] dev_ioctl+0x309/0x3e2
> > firenze kernel: [<f8afc8a2>]
> > _Z10dev_updownP10net_deviceiP12ErrorHandler+0x72/0x110 [click]
> > firenze kernel: [<f8afc78c>]
> > _ZN8FromHost20set_device_addressesEP12ErrorHandler+0xac/0x150 [click]
> > firenze kernel: [<f8afca11>]
> _ZN8FromHost10initializeEP12ErrorHandler+0xd1/0xf0
> > [click]
> > firenze kernel: [<f8aaa5e6>] _Znaj+0x16/0x20 [click]
> > firenze kernel: [<f8aca942>]
> _ZN6Router10initializeEP12ErrorHandler+0x682/0x780
> > [click]
> > firenze kernel: [<c013ec1d>] __alloc_pages+0x59/0x273
> > firenze kernel: [<f8b20111>]
> > _Z12write_configRK6StringP7ElementPvP12ErrorHandler+0x121/0x1e0 [clic k]
> > firenze kernel: [<f8ac456f>]
> > _ZNK7Handler10call_writeERK6StringP7ElementbP12ErrorHandler+0x13f/0x2 10
> > [click]
> > firenze kernel: [<c01533b2>] cache_grow+0x128/0x14a
> > firenze kernel: [<f8b23789>] handler_flush+0x499/0x590 [click]
> > firenze kernel: [<f8aaa5e6>] _Znaj+0x16/0x20 [click]
> > firenze kernel: [<c0155720>] filp_close+0x31/0x52
> > firenze kernel: [<c01031ab>] sysenter_past_esp+0x54/0x75
> > firenze kernel: Code: ff eb d7 c7 04 24 b5 26 b3 f8 e8 0a f4 ff ff e9 60 ff
> ff
> > ff 90 8d 74 26 00 57 bf 00 f0 ff ff 56 89 c6 53 83 ec 04 21 e7 8b 4e 1c
> <f0>
> > ff 41 4c 8d 59 44 8b 47 10 ba 01 00 00 00 39 43 04 74 2a 8d
> > --------------------------------
> > //Routing Configuration for Host Firenze
> > // Routing for Device eth2
> > // All from's and to's this real device and its spoof children
> > // Real Device Connection
> > RealFromDevice_eth2 :: FromDevice(eth2);
> > RealToDevice_eth2 :: Queue -> ToDevice(eth2);
> > // Real Host Spoof
> > RealFromHost_eth2 :: FromHost(fake_eth2, 192.168.10.6/24, ETHER
> > 00:04:23:C3:6A:6A);
> > RealToHost_eth2 :: ToHost(fake_eth2);
> >
> > // Spoof Host spoof_eth2_0
> > FromHost(spoof_eth2_0, 192.168.10.11/24, ETHER 00:04:23:C3:6A:6A) -> Queue
> ->
> > BandwidthRatedUnqueue(1000000 bps) -> Queue -> SpoofFromHost_spoof_eth2_0
> ::
> > DelayUnqueue(5 ms)
> > SpoofToHost_spoof_eth2_0 :: Queue -> BandwidthRatedUnqueue(8000000 bps) ->
> Queue
> > -> DelayUnqueue(5 ms) -> ToHost(spoof_eth2_0);
> >
> > // Spoof Host spoof_eth2_1
> > FromHost(spoof_eth2_1, 192.168.10.12/24, ETHER 00:04:23:C3:6A:6A) -> Queue
> ->
> > BandwidthRatedUnqueue(1000000 bps) -> Queue -> SpoofFromHost_spoof_eth2_1
> ::
> > DelayUnqueue(5 ms)
> > SpoofToHost_spoof_eth2_1 :: Queue -> BandwidthRatedUnqueue(8000000 bps) ->
> Queue
> > -> DelayUnqueue(5 ms) -> ToHost(spoof_eth2_1);
> >
> > // Spoof Host spoof_eth2_2
> > FromHost(spoof_eth2_2, 192.168.10.13/24, ETHER 00:04:23:C3:6A:6A) -> Queue
> ->
> > BandwidthRatedUnqueue(1000000 bps) -> Queue -> SpoofFromHost_spoof_eth2_2
> ::
> > DelayUnqueue(5 ms)
> > SpoofToHost_spoof_eth2_2 :: Queue -> BandwidthRatedUnqueue(8000000 bps) ->
> Queue
> > -> DelayUnqueue(5 ms) -> ToHost(spoof_eth2_2);
> >
> >
> > // Now we define all globally used Pipe's and Switches
> > // Entrance for all inc on real devices traffic, splits IP and 'others'
> > YePipeInc_eth2 :: Classifier( 12/0800,// IP packets
> > -); // Any others
> >
> > // Entrance for all inc on host traffic, splits IP and 'others'
> > YePipeOut_eth2 :: Classifier( 12/0800,// IP packets
> > -); // Any others
> >
> > UnknownOutTee_eth2 :: Tee; // For non-IP traffic to pass unhindered
> > UnknownIncTee_eth2 :: Tee; // For non-IP traffic to pass unhindered
> >
> > PipeOutRealToDevice_eth2_0 :: Print(OutPipe) -> Strip(14) ->
> EtherEncap(0x0800,
> > 00:04:23:C3:6A:6A, 00:04:23:C2:8C:1E) ->
> > Queue -> BandwidthRatedUnqueue(1000000000 bps) -> Queue -> DelayUnqueue(50
> ms)
> > -> RealToDevice_eth2;
> >
> > //This Adds pipe IP and click_name to the routing table for external routes
> >
> > // Splitter to grab any packet destined for a local real or spoof device
> > LocalDstSplitter_eth2 :: IPClassifier(dst 192.168.10.6, // dst for
> > RealToHost_eth2
> > dst 192.168.10.11, // dst for
> > SpoofToHost_spoof_eth2_0
> > dst 192.168.10.12, // dst for
> > SpoofToHost_spoof_eth2_1
> > dst 192.168.10.13, // dst for
> > SpoofToHost_spoof_eth2_2
> > -); // All other IP packets
> > // Splitter to grab any packet destined for a connected real device
> > RemoteDstSplitter_eth2 :: IPClassifier(dst 192.168.10.4, // dst for
> > PipeOutRealToDevice_eth2_0
> > dst 192.168.10.3, // dst for
> > PipeOutRealToDevice_eth2_0
> > -); // All other IP packets
> >
> > // Routing Setup
> > // All packets coming in on Real Devices must be passed to YePipeInc
> > // All packets going out on From all real and spoof interfaces must be
> passed to
> > YePipeOut
> > // Now you see why we call them Ye Pipes!
> > RealFromDevice_eth2 -> Print(FromDevice) -> YePipeInc_eth2
> > RealFromHost_eth2 -> Print(FromRealHost) -> YePipeOut_eth2
> > SpoofFromHost_spoof_eth2_0 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > SpoofFromHost_spoof_eth2_1 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > SpoofFromHost_spoof_eth2_2 -> Print(FromSpoofHost) -> YePipeOut_eth2
> >
> > YePipeInc_eth2[1] -> UnknownIncTee_eth2; // Non-IP traffic heads off
> unhindered
> > YePipeOut_eth2[1] -> UnknownOutTee_eth2; // Non-IP traffic heads off
> unhindered
> >
> > YePipeInc_eth2[0] -> Print(PipeIncToLclSplitter) -> LocalDstSplitter_eth2;
> //
> > All IP traffic hits the first splitter
> > YePipeOut_eth2[0] -> Print(PipeOutToLclSplitter) -> LocalDstSplitter_eth2;
> //
> > All IP traffic hits the first splitter
> >
> > // Here all non-IP traffic is passed on: outgoing traffic on all outgoing
> > devices,
> > // and incoming traffic to all registered hosts.
> > UnknownIncTee_eth2[0] -> RealToHost_eth2
> > UnknownOutTee_eth2[0] -> RealToDevice_eth2
> > UnknownIncTee_eth2[1] -> SpoofToHost_spoof_eth2_0
> > UnknownIncTee_eth2[2] -> SpoofToHost_spoof_eth2_1
> > UnknownIncTee_eth2[3] -> SpoofToHost_spoof_eth2_2
> > UnknownOutTee_eth2[1] -> UnknownIncTee_eth2; // For the benefit of Spoof
> ARP's,
> > we send non-IP stuff back around
> >
> > // Time to process the IP traffic: which was sent through the filter for
> local
> > destinations
> > LocalDstSplitter_eth2[0] -> Print(ToHost) -> RealToHost_eth2;
> > LocalDstSplitter_eth2[1] -> Print(ToHost) -> SpoofToHost_spoof_eth2_0;
> > LocalDstSplitter_eth2[2] -> Print(ToHost) -> SpoofToHost_spoof_eth2_1;
> > LocalDstSplitter_eth2[3] -> Print(ToHost) -> SpoofToHost_spoof_eth2_2;
> > // This final output is sent on to the remote splitter
> > LocalDstSplitter_eth2[4] -> Print(ToRemoteSplitter) ->
> RemoteDstSplitter_eth2;
> > // Not local destination, on to routing fun
> >
> > // Now we process all IP traffic that is NOT going to a local destination
> > RemoteDstSplitter_eth2[0] -> PipeOutRealToDevice_eth2_0;
> > RemoteDstSplitter_eth2[1] -> PipeOutRealToDevice_eth2_0;
> > RemoteDstSplitter_eth2[2] -> Print(Discarded) -> Discard; // No idea who
> its
> > going to, drop it
> >
> >
> >
> >
> > _______________________________________________
> > click mailing list
> > click at amsterdam.lcs.mit.edu
> > https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>
More information about the click
mailing list