[Click] Mac OS X assert failed when trying to use KernelTun
Eddie Kohler
kohler at cs.ucla.edu
Tue Dec 1 18:11:32 EST 2009
Pekka,
Thanks so much!! This is excellent. I've checked it in with a couple
edits to the documentation.
Eddie
Pekka Nikander wrote:
> Eddie,
>
> This a patch that implements the route adding on Mac OS X, and explains the small difference in the document. Minimally tested and seems to work.
>
> --Pekka
>
> diff --git a/elements/userlevel/kerneltun.cc b/elements/userlevel/kerneltun.cc
> index ca59225..abe95ac 100644
> --- a/elements/userlevel/kerneltun.cc
> +++ b/elements/userlevel/kerneltun.cc
> @@ -40,6 +40,7 @@
> # define KERNELTUN_OSX 1
> // assume tun driver installed from http://chrisp.de/en/projects/tunnel.html
> // this driver doesn't produce or expect packets with an address family prepended
> +#include <net/route.h>
> #endif
> #if defined(HAVE_NET_IF_TAP_H)
> # define KERNELTAP_NET 1
> @@ -302,6 +303,50 @@ KernelTun::updown(IPAddress addr, IPAddress mask, ErrorHandler *errh)
> #else
> # error "Lacking SIOCSIFADDR and/or SIOCSIFNETMASK"
> #endif
> +#if defined(KERNELTUN_OSX)
> + // On OSX, we have to explicitly add a route, too
> + {
> + static int seq = 0;
> + struct {
> + struct rt_msghdr msghdr;
> + struct sockaddr_in sin[3]; // Destination, gateway, netmask,
> + } msg;
> +
> + memset(&msg, 0, sizeof(msg));
> + msg.msghdr.rtm_msglen = sizeof(msg);
> + msg.msghdr.rtm_version = RTM_VERSION;
> + msg.msghdr.rtm_type = RTM_ADD;
> + msg.msghdr.rtm_index = 0;
> + msg.msghdr.rtm_pid = 0;
> + msg.msghdr.rtm_addrs = RTA_DST | RTA_GATEWAY | RTA_NETMASK;
> + msg.msghdr.rtm_seq = ++seq;
> + msg.msghdr.rtm_errno = 0;
> + msg.msghdr.rtm_flags = RTF_UP | RTF_GATEWAY;
> +
> + for (unsigned int i = 0; i < sizeof(msg.sin) / sizeof(msg.sin[0]); i++) {
> + msg.sin[i].sin_len = sizeof(msg.sin[0]);
> + msg.sin[i].sin_family = AF_INET;
> + }
> +
> + msg.sin[0].sin_addr = addr & mask; // Destination
> + msg.sin[1].sin_addr = addr; // Gateway
> + msg.sin[2].sin_addr = mask; // Netmask
> +
> + int s = socket(PF_ROUTE, SOCK_RAW, AF_INET);
> + if (s < 0) {
> + errh->warning("Opening a PF_ROUTE socket failed: %s", strerror(errno));
> + goto out;
> + }
> + int r = write(s, (char *)&msg, sizeof(msg));
> + if (r < 0) {
> + errh->warning("Writing to the PF_ROUTE socket failed: %s", strerror(errno));
> + }
> + r = close(s);
> + if (r < 0) {
> + errh->warning("Closing the PF_ROUTE socket failed: %s", strerror(errno));
> + }
> + }
> +#endif
> #if defined(SIOCSIFHWADDR)
> if (_macaddr) {
> ifr.ifr.ifr_hwaddr.sa_family = ARPHRD_ETHER;
> diff --git a/elements/userlevel/kerneltun.hh b/elements/userlevel/kerneltun.hh
> index 88cdf87..540dffe 100644
> --- a/elements/userlevel/kerneltun.hh
> +++ b/elements/userlevel/kerneltun.hh
> @@ -79,17 +79,19 @@ directory" may indicate that your kernel isn't set up, or that some
> required kernel module hasn't been loaded (on Linux, the relevant module is
> "tun").
>
> -Packets sent to ADDR will be processed by the host kernel stack; packets sent
> -to any other address in ADDR/MASK will be sent to KernelTun. Say you run this
> -configuration:
> +On Linux and most BSDs, packets sent to ADDR will be processed by the host
> +kernel stack; on Mac OS X there is no special handling for ADDR.
> +Packets sent to any (other) address in ADDR/MASK will be sent to KernelTun.
> +Say you run this configuration:
>
> tun :: KernelTun(1.0.0.1/8);
> tun -> IPClassifier(icmp type echo) -> ICMPPingResponder
> -> IPPrint -> tun;
>
> -If you then "C<ping 1.0.0.1>", I<your own kernel> will respond. Click will
> -never see the packets, so it won't print anything. But if you "C<ping
> -1.0.0.2>", the pings are sent to Click. You should see printouts from Click,
> +If you then "C<ping 1.0.0.1>", on Linux and most BSDs I<your own kernel> will respond.
> +On those systems, click will never see the packets, so it won't print anything;
> +on Mac OS X, Click will see even these packets.
> +If you "C<ping 1.0.0.2>", the pings are sent to Click. You should see printouts from Click,
> and C<ping> should print Click's responses.
>
> This element differs from KernelTap in that it produces and expects IP
>
>
> On 2009-11 -26, at 20:36 , Eddie Kohler wrote:
>
>> Thanks much for the patch! I've checked it in.
>>
>> Too bad about the high variability in tun semantics. I'd love to accept a documentation patch :)
>>
>> Eddie
>>
>>
>> Pekka Nikander wrote:
>>> Eddie,
>>> Thanks, it works! I needed to do two other small things to get it working:
>>> 1. It didn't compile out of last night's git; the following patch made it to compile:
>>> --- a/lib/timestamp.cc
>>> +++ b/lib/timestamp.cc
>>> @@ -67,7 +67,7 @@ Timestamp::warp(bool from_now)
>>> if (_warp_class == warp_simulation) {
>>> *this = _warp_flat_offset;
>>> if (from_now) {
>>> -# if TIMESTAMP_MATH_FLAT64
>>> +# if TIMESTAMP_REP_FLAT64 || TIMESTAMP_MATH_FLAT64
>>> ++_warp_flat_offset._t.x;
>>> # else
>>> ++_warp_flat_offset._t.subsec;
>>> I'm not sure if that is the right patch, though...
>>> 2. Mac OS X tun devices are point-to-point links, requiring both a local and remote IP address.
>>> Right now click configures only the local one, resulting in an interface like the following:
>>> $ ifconfig tun0
>>> tun0: flags=8951<UP,POINTOPOINT,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>>> inet 192.168.15.1 --> 0.0.0.0 netmask 0xffffff00 open (pid 63049)
>>> Once I added a remote address there with ifconfig, it started to work:
>>> # ifconfig tun0 192.168.15.1 192.168.16.1
>>> # click -e "tun :: KernelTun(192.168.15.1/24); tun -> IPPrint -> Discard" &
>>> # ping 192.168.16.1 > /dev/null 2>&1
>>> 1259231868.596160: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 0)
>>> 1259231869.596198: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 256)
>>> 1259231870.596240: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 512)
>>> 1259231871.596328: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 768)
>>> 1259231872.596394: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 1024)
>>> 1259231873.596553: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 1280)
>>> Alternatively, I was also able to do the same by manipulating the routing table:
>>> # route add -net 192.168.16.0 192.168.15.1
>>> However, pinging anything on the local subnet 192.168.15.0/24 doesn't go to the click process.
>>> I guess the documentation should be updated...
>>> Anyway, this should be enough slay some GMPLS Dragons locally on Mac OS X. :-)
>>> --Pekka
>>> On 2009-11 -25, at 12:21 , Eddie Kohler wrote:
>>>> Pekka,
>>>>
>>>> Thans very much for this note! There was indeed a bug in the select handling -- we essentially assumed that every file descriptor was present in some array for both reading and writing. This should be fixed now. Let us know if you have any further issues.
>>>>
>>>> Eddie
>>>>
>>>>
>>>> Pekka Nikander wrote:
>>>>> I'm a relative newbie to Click, and trying to use KernelTun on Mac OS X, with the latest GIT version. Unfortunately it looks like that there is a bug, apparently related to the interactions between kevents and select/poll. The OS X tun/tap devices don't currently support kevents, and therefore click tries to back off to use select/poll. However, once it gets to the actual poll in master.cc, something has gone wrong and I get a an assertion failure on line 850 in master.cc:
>>>>> Element *read_elt = (p->revents & ~POLLOUT ? _read_elements[fd] : 0);
>>>>> This results in an assertion failure, with this trivial script:
>>>>> click -e "KernelTun(192.168.15.1/24) -> Discard"
>>>>> Assertion failed: (i>=0 && i<_n), function operator[], file ../include/click/vector.hh, line 184.
>>>>> Abort trap
>>>>> My gut feeling is that the bug may line somewhere in master.cc Master:add_select, in the code that tries to make sure that one can fall back to select/poll in the case of kqueue error. But I may be wrong.
>>>>> In any case, when tracing the execution in opening the tun/tap device, the kevent system call at line 602 of master.cc fails, causing the _kqueue socket to be closed and made unused. However, much before that I can see with ifconfig that the tun/tap interface is indeed open and correctly ifconfig'ed.
>>>>> Anyone an idea where to continue debugging? Or would it be easier to add KEVENT support to the Mac tun/tap kexts?
>>>>> --Pekka Nikander
>>>>> _______________________________________________
>>>>> click mailing list
>>>>> click at amsterdam.lcs.mit.edu
>>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>
More information about the click
mailing list