[Click] Repeatable way to kernel panic via Click?
dmoore7@nd.edu
dmoore7 at nd.edu
Wed Aug 22 15:49:07 EDT 2007
Two things of note:
First off... I was wrong about this issue being easily avoidable... I'm still
getting the lockup/panic when I try to load the file given earlier. I would
like to do more testing to narrow down the source of my lockup but...
Secondly, the system is not rebooting itself after the fault, despite mods to
the panic and panic_oops files. Because rebooting this means getting someone
to drive across town for me, its not likely I will progress quickly in figuring
out what is doing this. I'm dropping all the spoofed IP's (which I have not had
trouble with in the past) in the interest of seeing if this fixes the issue, but
won't get another "test run" until probably tomorrow...
Can anyone offer input on what these error messages might mean?
Message from syslogd at filius at Wed Aug 22 10:03:54 2007 ...
filius kernel: BUG: spinlock wrong CPU on CPU#2, click-install/12444
Message from syslogd at filius at Wed Aug 22 10:03:54 2007 ...
filius kernel: lock: f8c79c44, .magic: dead4ead, .owner: click-install/12444,
.owner_cpu: 1
Any input is appreciated. Thanks,
- David
Quoting Beyers Cronje <bcronje at gmail.com>:
> Doh, as David pointed out use echo instead of cat :)
>
> On 8/22/07, dmoore7 at nd.edu <dmoore7 at nd.edu> wrote:
> >
> > I did not know this... the IT guys will be happy though.
> > Although, the command is:
> > echo 60 > ....
> > Because of course I have no file named '60' kicking around for cat to
> > print out
> > of.
> > Thanks so much for the tip, you are my hero.
> > - David
> >
> > Quoting Beyers Cronje <bcronje at gmail.com>:
> >
> > > Hi David,
> > >
> > > I'm sure you know this already, what I always do on remote servers is to
> > set
> > > panic and panic_on_oops proc setting which will reboot the server on an
> > > oops:
> > >
> > > cat 60 > /proc/sys/kernel/panic
> > > cat 1 > /proc/sys/kernel/panic_on_oops
> > >
> > > So if the kernel oopses at least the server reboots, saved me a lot of
> > > headaches.
> > >
> > > Beyers
> > >
> > > On 8/22/07, dmoore7 at nd.edu <dmoore7 at nd.edu> wrote:
> > > >
> > > > Hello all,
> > > > I've been working with click most of the summer and seem to have found
> > a
> > > > repeatable way to make a kernel panic (something not-so-fun as my
> > servers
> > > > are
> > > > not physically accessible).
> > > > It is avoidable with a little care, but I presume folks will want to
> > know
> > > > about
> > > > it.
> > > >
> > > > The machine in question has 2 NIC's, and if I ssh in on NIC1, and then
> > > > attempt
> > > > to run this click file, without bringing up the second NIC via
> > ifconfig, I
> > > > get
> > > > the following error:
> > > >
> > > > [root at filius ~]# click-install ./Filius.click
> > > >
> > > > Message from syslogd at filius at Wed Aug 22 10:03:54 2007 ...
> > > > filius kernel: BUG: spinlock wrong CPU on CPU#2, click-install/12444
> > > >
> > > > Message from syslogd at filius at Wed Aug 22 10:03:54 2007 ...
> > > > filius kernel: lock: f8c79c44, .magic: dead4ead, .owner:
> > > > click-install/12444,
> > > > .owner_cpu: 1
> > > >
> > > > At this stage the ssh window stops responding, the machine no longer
> > > > responds to
> > > > pings, and hard reset is needed. I can avoid this if I take care to
> > > > "ifconfig
> > > > up" my eth's ahead of time.
> > > > For those interested, the click file in question is included
> > > > below. Basically
> > > > it performs To and From Device and Host on an eth that was not up at
> > the
> > > > time,
> > > > in addition to a bunch of other stuff.
> > > > It also spoofs a bunch of IP's for fun and games.
> > > >
> > > > To recap: I can work around this issue, but I thought the community
> > might
> > > > benefit from me showing this.
> > > > Thanks,
> > > > - David Moore
> > > >
> > > > The File:
> > > > ---------------------------------------------------
> > > > //Routing Configuration for Host Filius
> > > > // Routing for Device eth2
> > > > // All from's and to's this real device and its spoof children
> > > > // Real Device Connection
> > > > RealFromDevice_eth2 :: FromDevice(eth2);
> > > > RealToDevice_eth2 :: Queue -> ToDevice(eth2);
> > > > // Real Host Spoof
> > > > RealFromHost_eth2 :: FromHost(fake_eth2, 192.168.10.5/24, ETHER
> > > > 00:04:23:C2:8A:EC);
> > > > RealToHost_eth2 :: ToHost(fake_eth2);
> > > >
> > > > // Spoof Host spoof_eth2_0
> > > > FromHost(spoof_eth2_0, 192.168.10.11/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_0 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_0 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_0);
> > > >
> > > > // Spoof Host spoof_eth2_1
> > > > FromHost(spoof_eth2_1, 192.168.10.12/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_1 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_1 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_1);
> > > >
> > > > // Spoof Host spoof_eth2_2
> > > > FromHost(spoof_eth2_2, 192.168.10.13/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_2 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_2 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_2);
> > > >
> > > > // Spoof Host spoof_eth2_3
> > > > FromHost(spoof_eth2_3, 192.168.10.14/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_3 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_3 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_3);
> > > >
> > > > // Spoof Host spoof_eth2_4
> > > > FromHost(spoof_eth2_4, 192.168.10.15/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_4 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_4 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_4);
> > > >
> > > > // Spoof Host spoof_eth2_5
> > > > FromHost(spoof_eth2_5, 192.168.10.16/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_5 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_5 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_5);
> > > >
> > > > // Spoof Host spoof_eth2_6
> > > > FromHost(spoof_eth2_6, 192.168.10.17/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_6 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_6 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_6);
> > > >
> > > > // Spoof Host spoof_eth2_7
> > > > FromHost(spoof_eth2_7, 192.168.10.18/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_7 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_7 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_7);
> > > >
> > > > // Spoof Host spoof_eth2_8
> > > > FromHost(spoof_eth2_8, 192.168.10.19/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_8 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_8 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_8);
> > > >
> > > > // Spoof Host spoof_eth2_9
> > > > FromHost(spoof_eth2_9, 192.168.10.20/24, ETHER 00:04:23:C2:8A:EC) ->
> > Queue
> > > > ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_9 ::
> > > > DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_9 :: Queue -> BandwidthRatedUnqueue(1000000000
> > bps)
> > > > ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_9);
> > > >
> > > > // Spoof Host spoof_eth2_10
> > > > FromHost(spoof_eth2_10, 192.168.10.21/24, ETHER 00:04:23:C2:8A:EC) ->
> > > > Queue ->
> > > > BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > > > SpoofFromHost_spoof_eth2_10
> > > > :: DelayUnqueue(100 ms)
> > > > SpoofToHost_spoof_eth2_10 :: Queue -> BandwidthRatedUnqueue(1000000000
> > > > bps) ->
> > > > Queue -> DelayUnqueue(100 ms) -> ToHost(spoof_eth2_10);
> > > >
> > > >
> > > > // Now we define all globally used Pipe's and Switches
> > > > // Entrance for all inc on real devices traffic, splits IP and
> > 'others'
> > > > YePipeInc_eth2 :: Classifier( 12/0800,// IP packets
> > > > -); // Any others
> > > >
> > > > // Entrance for all inc on host traffic, splits IP and 'others'
> > > > YePipeOut_eth2 :: Classifier( 12/0800,// IP packets
> > > > -); // Any others
> > > >
> > > > UnknownOutTee_eth2 :: Tee; // For non-IP traffic to pass unhindered
> > > > UnknownIncTee_eth2 :: Tee; // For non-IP traffic to pass unhindered
> > > >
> > > > PipeOutRealToDevice_eth2_0 :: Print(OutPipe) -> Strip(14) ->
> > > > EtherEncap(0x0800,
> > > > 00:04:23:C2:8A:EC, 00:04:23:C2:8C:1E) ->
> > > > Queue -> BandwidthRatedUnqueue(1000000000 bps) -> Queue ->
> > DelayUnqueue(0
> > > > ms) ->
> > > > RealToDevice_eth2;
> > > >
> > > > //This Adds pipe IP and click_name to the routing table for external
> > > > routes
> > > >
> > > > // Splitter to grab any packet destined for a local real or spoof
> > device
> > > > LocalDstSplitter_eth2 :: IPClassifier(dst 192.168.10.5, // dst for
> > > > RealToHost_eth2
> > > > dst 192.168.10.11, // dst for
> > > > SpoofToHost_spoof_eth2_0
> > > > dst 192.168.10.12, // dst for
> > > > SpoofToHost_spoof_eth2_1
> > > > dst 192.168.10.13, // dst for
> > > > SpoofToHost_spoof_eth2_2
> > > > dst 192.168.10.14, // dst for
> > > > SpoofToHost_spoof_eth2_3
> > > > dst 192.168.10.15, // dst for
> > > > SpoofToHost_spoof_eth2_4
> > > > dst 192.168.10.16, // dst for
> > > > SpoofToHost_spoof_eth2_5
> > > > dst 192.168.10.17, // dst for
> > > > SpoofToHost_spoof_eth2_6
> > > > dst 192.168.10.18, // dst for
> > > > SpoofToHost_spoof_eth2_7
> > > > dst 192.168.10.19, // dst for
> > > > SpoofToHost_spoof_eth2_8
> > > > dst 192.168.10.20, // dst for
> > > > SpoofToHost_spoof_eth2_9
> > > > dst 192.168.10.21, // dst for
> > > > SpoofToHost_spoof_eth2_10
> > > > -); // All other IP packets
> > > > // Splitter to grab any packet destined for a connected real device
> > > > RemoteDstSplitter_eth2 :: IPClassifier(dst 192.168.10.3, // dst for
> > > > PipeOutRealToDevice_eth2_0
> > > > dst 192.168.10.4, // dst for
> > > > PipeOutRealToDevice_eth2_0
> > > > -); // All other IP packets
> > > >
> > > > // Routing Setup
> > > > // All packets coming in on Real Devices must be passed to YePipeInc
> > > > // All packets going out on From all real and spoof interfaces must be
> > > > passed to
> > > > YePipeOut
> > > > // Now you see why we call them Ye Pipes!
> > > > RealFromDevice_eth2 -> Print(FromDevice) -> YePipeInc_eth2
> > > > RealFromHost_eth2 -> Print(FromRealHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_0 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_1 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_2 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_3 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_4 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_5 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_6 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_7 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_8 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_9 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > > SpoofFromHost_spoof_eth2_10 -> Print(FromSpoofHost) -> YePipeOut_eth2
> > > >
> > > > YePipeInc_eth2[1] -> UnknownIncTee_eth2; // Non-IP traffic heads off
> > > > unhindered
> > > > YePipeOut_eth2[1] -> UnknownOutTee_eth2; // Non-IP traffic heads off
> > > > unhindered
> > > >
> > > > YePipeInc_eth2[0] -> Print(PipeIncToLclSplitter) ->
> > LocalDstSplitter_eth2;
> > > > //
> > > > All IP traffic hits the first splitter
> > > > YePipeOut_eth2[0] -> Print(PipeOutToLclSplitter) ->
> > LocalDstSplitter_eth2;
> > > > //
> > > > All IP traffic hits the first splitter
> > > >
> > > > // Here all non-IP traffic is passed on: outgoing traffic on all
> > outgoing
> > > > devices,
> > > > // and incoming traffic to all registered hosts.
> > > > UnknownIncTee_eth2[0] -> RealToHost_eth2
> > > > UnknownOutTee_eth2[0] -> RealToDevice_eth2
> > > > UnknownIncTee_eth2[1] -> SpoofToHost_spoof_eth2_0
> > > > UnknownIncTee_eth2[2] -> SpoofToHost_spoof_eth2_1
> > > > UnknownIncTee_eth2[3] -> SpoofToHost_spoof_eth2_2
> > > > UnknownIncTee_eth2[4] -> SpoofToHost_spoof_eth2_3
> > > > UnknownIncTee_eth2[5] -> SpoofToHost_spoof_eth2_4
> > > > UnknownIncTee_eth2[6] -> SpoofToHost_spoof_eth2_5
> > > > UnknownIncTee_eth2[7] -> SpoofToHost_spoof_eth2_6
> > > > UnknownIncTee_eth2[8] -> SpoofToHost_spoof_eth2_7
> > > > UnknownIncTee_eth2[9] -> SpoofToHost_spoof_eth2_8
> > > > UnknownIncTee_eth2[10] -> SpoofToHost_spoof_eth2_9
> > > > UnknownIncTee_eth2[11] -> SpoofToHost_spoof_eth2_10
> > > > UnknownOutTee_eth2[1] -> UnknownIncTee_eth2; // For the benefit of
> > Spoof
> > > > ARP's,
> > > > we send non-IP stuff back around
> > > >
> > > > // Time to process the IP traffic: which was sent through the filter
> > for
> > > > local
> > > > destinations
> > > > LocalDstSplitter_eth2[0] -> Print(ToHost) -> RealToHost_eth2;
> > > > LocalDstSplitter_eth2[1] -> Print(ToHost) -> SpoofToHost_spoof_eth2_0;
> > > > LocalDstSplitter_eth2[2] -> Print(ToHost) -> SpoofToHost_spoof_eth2_1;
> > > > LocalDstSplitter_eth2[3] -> Print(ToHost) -> SpoofToHost_spoof_eth2_2;
> > > > LocalDstSplitter_eth2[4] -> Print(ToHost) -> SpoofToHost_spoof_eth2_3;
> > > > LocalDstSplitter_eth2[5] -> Print(ToHost) -> SpoofToHost_spoof_eth2_4;
> > > > LocalDstSplitter_eth2[6] -> Print(ToHost) -> SpoofToHost_spoof_eth2_5;
> > > > LocalDstSplitter_eth2[7] -> Print(ToHost) -> SpoofToHost_spoof_eth2_6;
> > > > LocalDstSplitter_eth2[8] -> Print(ToHost) -> SpoofToHost_spoof_eth2_7;
> > > > LocalDstSplitter_eth2[9] -> Print(ToHost) -> SpoofToHost_spoof_eth2_8;
> > > > LocalDstSplitter_eth2[10] -> Print(ToHost) ->
> > SpoofToHost_spoof_eth2_9;
> > > > LocalDstSplitter_eth2[11] -> Print(ToHost) ->
> > SpoofToHost_spoof_eth2_10;
> > > > // This final output is sent on to the remote splitter
> > > > LocalDstSplitter_eth2[12] -> Print(ToRemoteSplitter) ->
> > > > RemoteDstSplitter_eth2;
> > > > // Not local destination, on to routing fun
> > > >
> > > > // Now we process all IP traffic that is NOT going to a local
> > destination
> > > > RemoteDstSplitter_eth2[0] -> PipeOutRealToDevice_eth2_0;
> > > > RemoteDstSplitter_eth2[1] -> PipeOutRealToDevice_eth2_0;
> > > > RemoteDstSplitter_eth2[2] -> Print(Discarded) -> Discard; // No idea
> > who
> > > > its
> > > > going to, drop it
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > click mailing list
> > > > click at amsterdam.lcs.mit.edu
> > > > https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> > > >
> > >
> >
> >
> >
> >
> >
> >
>
More information about the click
mailing list