[Click] Mac OS X assert failed when trying to use KernelTun
Pekka Nikander
pekka.nikander at nomadiclab.com
Fri Nov 27 08:51:19 EST 2009
Eddie,
This a patch that implements the route adding on Mac OS X, and explains the small difference in the document. Minimally tested and seems to work.
--Pekka
diff --git a/elements/userlevel/kerneltun.cc b/elements/userlevel/kerneltun.cc
index ca59225..abe95ac 100644
--- a/elements/userlevel/kerneltun.cc
+++ b/elements/userlevel/kerneltun.cc
@@ -40,6 +40,7 @@
# define KERNELTUN_OSX 1
// assume tun driver installed from http://chrisp.de/en/projects/tunnel.html
// this driver doesn't produce or expect packets with an address family prepended
+#include <net/route.h>
#endif
#if defined(HAVE_NET_IF_TAP_H)
# define KERNELTAP_NET 1
@@ -302,6 +303,50 @@ KernelTun::updown(IPAddress addr, IPAddress mask, ErrorHandler *errh)
#else
# error "Lacking SIOCSIFADDR and/or SIOCSIFNETMASK"
#endif
+#if defined(KERNELTUN_OSX)
+ // On OSX, we have to explicitly add a route, too
+ {
+ static int seq = 0;
+ struct {
+ struct rt_msghdr msghdr;
+ struct sockaddr_in sin[3]; // Destination, gateway, netmask,
+ } msg;
+
+ memset(&msg, 0, sizeof(msg));
+ msg.msghdr.rtm_msglen = sizeof(msg);
+ msg.msghdr.rtm_version = RTM_VERSION;
+ msg.msghdr.rtm_type = RTM_ADD;
+ msg.msghdr.rtm_index = 0;
+ msg.msghdr.rtm_pid = 0;
+ msg.msghdr.rtm_addrs = RTA_DST | RTA_GATEWAY | RTA_NETMASK;
+ msg.msghdr.rtm_seq = ++seq;
+ msg.msghdr.rtm_errno = 0;
+ msg.msghdr.rtm_flags = RTF_UP | RTF_GATEWAY;
+
+ for (unsigned int i = 0; i < sizeof(msg.sin) / sizeof(msg.sin[0]); i++) {
+ msg.sin[i].sin_len = sizeof(msg.sin[0]);
+ msg.sin[i].sin_family = AF_INET;
+ }
+
+ msg.sin[0].sin_addr = addr & mask; // Destination
+ msg.sin[1].sin_addr = addr; // Gateway
+ msg.sin[2].sin_addr = mask; // Netmask
+
+ int s = socket(PF_ROUTE, SOCK_RAW, AF_INET);
+ if (s < 0) {
+ errh->warning("Opening a PF_ROUTE socket failed: %s", strerror(errno));
+ goto out;
+ }
+ int r = write(s, (char *)&msg, sizeof(msg));
+ if (r < 0) {
+ errh->warning("Writing to the PF_ROUTE socket failed: %s", strerror(errno));
+ }
+ r = close(s);
+ if (r < 0) {
+ errh->warning("Closing the PF_ROUTE socket failed: %s", strerror(errno));
+ }
+ }
+#endif
#if defined(SIOCSIFHWADDR)
if (_macaddr) {
ifr.ifr.ifr_hwaddr.sa_family = ARPHRD_ETHER;
diff --git a/elements/userlevel/kerneltun.hh b/elements/userlevel/kerneltun.hh
index 88cdf87..540dffe 100644
--- a/elements/userlevel/kerneltun.hh
+++ b/elements/userlevel/kerneltun.hh
@@ -79,17 +79,19 @@ directory" may indicate that your kernel isn't set up, or that some
required kernel module hasn't been loaded (on Linux, the relevant module is
"tun").
-Packets sent to ADDR will be processed by the host kernel stack; packets sent
-to any other address in ADDR/MASK will be sent to KernelTun. Say you run this
-configuration:
+On Linux and most BSDs, packets sent to ADDR will be processed by the host
+kernel stack; on Mac OS X there is no special handling for ADDR.
+Packets sent to any (other) address in ADDR/MASK will be sent to KernelTun.
+Say you run this configuration:
tun :: KernelTun(1.0.0.1/8);
tun -> IPClassifier(icmp type echo) -> ICMPPingResponder
-> IPPrint -> tun;
-If you then "C<ping 1.0.0.1>", I<your own kernel> will respond. Click will
-never see the packets, so it won't print anything. But if you "C<ping
-1.0.0.2>", the pings are sent to Click. You should see printouts from Click,
+If you then "C<ping 1.0.0.1>", on Linux and most BSDs I<your own kernel> will respond.
+On those systems, click will never see the packets, so it won't print anything;
+on Mac OS X, Click will see even these packets.
+If you "C<ping 1.0.0.2>", the pings are sent to Click. You should see printouts from Click,
and C<ping> should print Click's responses.
This element differs from KernelTap in that it produces and expects IP
On 2009-11 -26, at 20:36 , Eddie Kohler wrote:
> Thanks much for the patch! I've checked it in.
>
> Too bad about the high variability in tun semantics. I'd love to accept a documentation patch :)
>
> Eddie
>
>
> Pekka Nikander wrote:
>> Eddie,
>> Thanks, it works! I needed to do two other small things to get it working:
>> 1. It didn't compile out of last night's git; the following patch made it to compile:
>> --- a/lib/timestamp.cc
>> +++ b/lib/timestamp.cc
>> @@ -67,7 +67,7 @@ Timestamp::warp(bool from_now)
>> if (_warp_class == warp_simulation) {
>> *this = _warp_flat_offset;
>> if (from_now) {
>> -# if TIMESTAMP_MATH_FLAT64
>> +# if TIMESTAMP_REP_FLAT64 || TIMESTAMP_MATH_FLAT64
>> ++_warp_flat_offset._t.x;
>> # else
>> ++_warp_flat_offset._t.subsec;
>> I'm not sure if that is the right patch, though...
>> 2. Mac OS X tun devices are point-to-point links, requiring both a local and remote IP address.
>> Right now click configures only the local one, resulting in an interface like the following:
>> $ ifconfig tun0
>> tun0: flags=8951<UP,POINTOPOINT,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>> inet 192.168.15.1 --> 0.0.0.0 netmask 0xffffff00 open (pid 63049)
>> Once I added a remote address there with ifconfig, it started to work:
>> # ifconfig tun0 192.168.15.1 192.168.16.1
>> # click -e "tun :: KernelTun(192.168.15.1/24); tun -> IPPrint -> Discard" &
>> # ping 192.168.16.1 > /dev/null 2>&1
>> 1259231868.596160: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 0)
>> 1259231869.596198: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 256)
>> 1259231870.596240: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 512)
>> 1259231871.596328: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 768)
>> 1259231872.596394: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 1024)
>> 1259231873.596553: 192.168.15.1 > 192.168.16.1: icmp echo (63066, 1280)
>> Alternatively, I was also able to do the same by manipulating the routing table:
>> # route add -net 192.168.16.0 192.168.15.1
>> However, pinging anything on the local subnet 192.168.15.0/24 doesn't go to the click process.
>> I guess the documentation should be updated...
>> Anyway, this should be enough slay some GMPLS Dragons locally on Mac OS X. :-)
>> --Pekka
>> On 2009-11 -25, at 12:21 , Eddie Kohler wrote:
>>> Pekka,
>>>
>>> Thans very much for this note! There was indeed a bug in the select handling -- we essentially assumed that every file descriptor was present in some array for both reading and writing. This should be fixed now. Let us know if you have any further issues.
>>>
>>> Eddie
>>>
>>>
>>> Pekka Nikander wrote:
>>>> I'm a relative newbie to Click, and trying to use KernelTun on Mac OS X, with the latest GIT version. Unfortunately it looks like that there is a bug, apparently related to the interactions between kevents and select/poll. The OS X tun/tap devices don't currently support kevents, and therefore click tries to back off to use select/poll. However, once it gets to the actual poll in master.cc, something has gone wrong and I get a an assertion failure on line 850 in master.cc:
>>>> Element *read_elt = (p->revents & ~POLLOUT ? _read_elements[fd] : 0);
>>>> This results in an assertion failure, with this trivial script:
>>>> click -e "KernelTun(192.168.15.1/24) -> Discard"
>>>> Assertion failed: (i>=0 && i<_n), function operator[], file ../include/click/vector.hh, line 184.
>>>> Abort trap
>>>> My gut feeling is that the bug may line somewhere in master.cc Master:add_select, in the code that tries to make sure that one can fall back to select/poll in the case of kqueue error. But I may be wrong.
>>>> In any case, when tracing the execution in opening the tun/tap device, the kevent system call at line 602 of master.cc fails, causing the _kqueue socket to be closed and made unused. However, much before that I can see with ifconfig that the tun/tap interface is indeed open and correctly ifconfig'ed.
>>>> Anyone an idea where to continue debugging? Or would it be easier to add KEVENT support to the Mac tun/tap kexts?
>>>> --Pekka Nikander
>>>> _______________________________________________
>>>> click mailing list
>>>> click at amsterdam.lcs.mit.edu
>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
More information about the click
mailing list