[Click] Mac OS X assert failed when trying to use KernelTun

Eddie Kohler kohler at cs.ucla.edu
Tue Dec 1 18:11:32 EST 2009


Pekka,

Thanks so much!!  This is excellent.  I've checked it in with a couple 
edits to the documentation.

Eddie


Pekka Nikander wrote:
> Eddie,
> 
> This a patch that implements the route adding on Mac OS X, and explains the small difference in the document.  Minimally tested and seems to work.
> 
> --Pekka
> 
> diff --git a/elements/userlevel/kerneltun.cc b/elements/userlevel/kerneltun.cc
> index ca59225..abe95ac 100644
> --- a/elements/userlevel/kerneltun.cc
> +++ b/elements/userlevel/kerneltun.cc
> @@ -40,6 +40,7 @@
>  # define KERNELTUN_OSX 1
>  // assume tun driver installed from http://chrisp.de/en/projects/tunnel.html
>  // this driver doesn't produce or expect packets with an address family prepended
> +#include <net/route.h>
>  #endif
>  #if defined(HAVE_NET_IF_TAP_H)
>  # define KERNELTAP_NET 1
> @@ -302,6 +303,50 @@ KernelTun::updown(IPAddress addr, IPAddress mask, ErrorHandler *errh)
>  #else
>  # error "Lacking SIOCSIFADDR and/or SIOCSIFNETMASK"
>  #endif
> +#if defined(KERNELTUN_OSX)
> +    // On OSX, we have to explicitly add a route, too
> +    {
> +       static int seq = 0;
> +       struct {
> +           struct rt_msghdr msghdr;
> +           struct sockaddr_in sin[3]; // Destination, gateway, netmask, 
> +       } msg;
> +
> +       memset(&msg, 0, sizeof(msg));
> +       msg.msghdr.rtm_msglen  = sizeof(msg);
> +       msg.msghdr.rtm_version = RTM_VERSION;
> +       msg.msghdr.rtm_type    = RTM_ADD;
> +       msg.msghdr.rtm_index   = 0;
> +       msg.msghdr.rtm_pid     = 0;
> +       msg.msghdr.rtm_addrs   = RTA_DST | RTA_GATEWAY | RTA_NETMASK;
> +       msg.msghdr.rtm_seq     = ++seq;
> +       msg.msghdr.rtm_errno   = 0;
> +       msg.msghdr.rtm_flags   = RTF_UP | RTF_GATEWAY;
> +
> +       for (unsigned int i = 0; i < sizeof(msg.sin) / sizeof(msg.sin[0]); i++) {
> +           msg.sin[i].sin_len    = sizeof(msg.sin[0]);
> +           msg.sin[i].sin_family = AF_INET;
> +       }
> +           
> +       msg.sin[0].sin_addr = addr & mask; // Destination
> +       msg.sin[1].sin_addr = addr;        // Gateway
> +       msg.sin[2].sin_addr = mask;        // Netmask
> +
> +       int s = socket(PF_ROUTE, SOCK_RAW, AF_INET);
> +       if (s < 0) {
> +           errh->warning("Opening a PF_ROUTE socket failed: %s", strerror(errno));
> +           goto out;
> +       }
> +       int r = write(s, (char *)&msg, sizeof(msg));
> +       if (r < 0) {
> +           errh->warning("Writing to the PF_ROUTE socket failed: %s", strerror(errno));
> +       }
> +       r = close(s);
> +       if (r < 0) {
> +           errh->warning("Closing the PF_ROUTE socket failed: %s", strerror(errno));
> +       }
> +    }
> +#endif
>  #if defined(SIOCSIFHWADDR)
>      if (_macaddr) {
>         ifr.ifr.ifr_hwaddr.sa_family = ARPHRD_ETHER;
> diff --git a/elements/userlevel/kerneltun.hh b/elements/userlevel/kerneltun.hh
> index 88cdf87..540dffe 100644
> --- a/elements/userlevel/kerneltun.hh
> +++ b/elements/userlevel/kerneltun.hh
> @@ -79,17 +79,19 @@ directory" may indicate that your kernel isn't set up, or that some
>  required kernel module hasn't been loaded (on Linux, the relevant module is
>  "tun").
>  
> -Packets sent to ADDR will be processed by the host kernel stack; packets sent
> -to any other address in ADDR/MASK will be sent to KernelTun.  Say you run this
> -configuration:
> +On Linux and most BSDs, packets sent to ADDR will be processed by the host 
> +kernel stack; on Mac OS X there is no special handling for ADDR.
> +Packets sent to any (other) address in ADDR/MASK will be sent to KernelTun.  
> +Say you run this configuration:
>  
>      tun :: KernelTun(1.0.0.1/8);
>      tun -> IPClassifier(icmp type echo) -> ICMPPingResponder
>          -> IPPrint -> tun;
>  
> -If you then "C<ping 1.0.0.1>", I<your own kernel> will respond.  Click will
> -never see the packets, so it won't print anything.  But if you "C<ping
> -1.0.0.2>", the pings are sent to Click.  You should see printouts from Click,
> +If you then "C<ping 1.0.0.1>", on Linux and most BSDs I<your own kernel> will respond.  
> +On those systems, click will never see the packets, so it won't print anything;
> +on Mac OS X, Click will see even these packets.
> +If you "C<ping 1.0.0.2>", the pings are sent to Click.  You should see printouts from Click,
>  and C<ping> should print Click's responses.
>  
>  This element differs from KernelTap in that it produces and expects IP
> 
> 
> On 2009-11 -26, at 20:36 , Eddie Kohler wrote:
> 
>> Thanks much for the patch!  I've checked it in.
>>
>> Too bad about the high variability in tun semantics.  I'd love to accept a documentation patch :)
>>
>> Eddie
>>
>>
>> Pekka Nikander wrote:
>>> Eddie,
>>> Thanks, it works!  I needed to do two other small things to get it working:
>>> 1. It didn't compile out of last night's git; the following patch made it to compile:
>>> --- a/lib/timestamp.cc
>>> +++ b/lib/timestamp.cc
>>> @@ -67,7 +67,7 @@ Timestamp::warp(bool from_now)
>>>     if (_warp_class == warp_simulation) {
>>>        *this = _warp_flat_offset;
>>>        if (from_now) {
>>> -# if TIMESTAMP_MATH_FLAT64
>>> +# if TIMESTAMP_REP_FLAT64 || TIMESTAMP_MATH_FLAT64
>>>            ++_warp_flat_offset._t.x;
>>> # else
>>>            ++_warp_flat_offset._t.subsec;
>>> I'm not sure if that is the right patch, though...
>>> 2. Mac OS X tun devices are point-to-point links, requiring both a local and remote IP address.
>>> Right now click configures only the local one, resulting in an interface like the following:
>>> $ ifconfig tun0
>>> tun0: flags=8951<UP,POINTOPOINT,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>>> 	inet 192.168.15.1 --> 0.0.0.0 netmask 0xffffff00 	open (pid 63049)
>>> Once I added a remote address there with ifconfig, it started to work:
>>> # ifconfig tun0 192.168.15.1 192.168.16.1
>>> # click -e "tun :: KernelTun(192.168.15.1/24); tun -> IPPrint -> Discard" &
>>> # ping 192.168.16.1 > /dev/null 2>&1
>>> 1259231868.596160: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 0)
>>> 1259231869.596198: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 256)
>>> 1259231870.596240: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 512)
>>> 1259231871.596328: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 768)
>>> 1259231872.596394: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 1024)
>>> 1259231873.596553: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 1280)
>>> Alternatively, I was also able to do the same by manipulating the routing table:
>>> # route add -net 192.168.16.0 192.168.15.1
>>> However, pinging anything on the local subnet 192.168.15.0/24 doesn't go to the click process.
>>> I guess the documentation should be updated...
>>> Anyway, this should be enough slay some GMPLS Dragons locally on Mac OS X. :-)
>>> --Pekka
>>> On 2009-11 -25, at 12:21 , Eddie Kohler wrote:
>>>> Pekka,
>>>>
>>>> Thans very much for this note!  There was indeed a bug in the select handling -- we essentially assumed that every file descriptor was present in some array for both reading and writing.  This should be fixed now.  Let us know if you have any further issues.
>>>>
>>>> Eddie
>>>>
>>>>
>>>> Pekka Nikander wrote:
>>>>> I'm a relative newbie to Click, and trying to use KernelTun on Mac OS X, with the latest GIT version.  Unfortunately it looks like that there is a bug, apparently related to the interactions between kevents and select/poll.  The OS X tun/tap devices don't currently support kevents, and therefore click tries to back off to use select/poll.  However, once it gets to the actual poll in master.cc, something has gone wrong and I get a an assertion failure on line 850 in master.cc:
>>>>> 	Element *read_elt = (p->revents & ~POLLOUT ? _read_elements[fd] : 0);
>>>>> This results in an assertion failure, with this trivial script:
>>>>> click -e "KernelTun(192.168.15.1/24) -> Discard"
>>>>> Assertion failed: (i>=0 && i<_n), function operator[], file ../include/click/vector.hh, line 184.
>>>>> Abort trap
>>>>> My gut feeling is that the bug may line somewhere in master.cc Master:add_select, in the code that tries to make sure that one can fall back to select/poll in the case of kqueue error.  But I may be wrong.
>>>>> In any case, when tracing the execution in opening the tun/tap device, the kevent system call at line 602 of master.cc fails, causing the _kqueue socket to be closed and made unused.   However, much before that I can see with ifconfig that the tun/tap interface is indeed open and correctly ifconfig'ed.
>>>>> Anyone an idea where to continue debugging?  Or would it be easier to add KEVENT support to the Mac tun/tap kexts?
>>>>> --Pekka Nikander
>>>>> _______________________________________________
>>>>> click mailing list
>>>>> click at amsterdam.lcs.mit.edu
>>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> 


More information about the click mailing list