From ekohler at gmail.com Fri Mar 1 23:47:25 2013 From: ekohler at gmail.com (Eddie Kohler) Date: Fri, 1 Mar 2013 23:47:25 -0500 Subject: [Click] Compile issues on RHEL 6.4 (2.6.32) In-Reply-To: References: Message-ID: Hi Keith, I got a look at this. There were a couple problems with that kernel that our rewriting, etc. tools could not handle. I were able to fix them and get a pretty clean automatic compilation. These patches are committed. However, an installation attempt fails with "out of memory". I haven't debugged further. Have you gotten further? BTW, I didn't realize you were downloading the current tarball from github. You did get the latest commit, but I'd strongly advise you to install git, which will make it easier to update. Eddie On Sat, Feb 23, 2013 at 2:14 PM, Keith Schoenefeld wrote: > I should have said "I grabbed the latest tarball from github". More > specifically, I downloaded the tarball provided at the URI > "https://github.com/kohler/click/tarball/master". Since the directory > name created when I extracted the tarball (kohler-click-6aa1787) > matches the latest commit, I assumed this matched what I could obtain > using git clone. Should I install git and run a git clone just to be > sure? > > -- KS > > On Sat, Feb 23, 2013 at 12:25 PM, Eddie Kohler wrote: >> Just to double check for a misunderstanding: You don't grab a tarball >> from git. Are you using the click-2.0.1 release, or the current git >> sources, which you can obtain by `git clone` from github.com? >> >> >> On Sat, Feb 23, 2013 at 1:11 PM, Keith Schoenefeld >> wrote: >>> - My understanding is that CentOS is built using the source RPMS for >>> RHEL, so CentOS 6.x would be as close to identical as possible with >>> RHEL 6.x. >>> - I used the stock compiler that comes with RHEL 6.4 >>> - I grabbed the latest tarball from git and used that, yes. >>> >>> I will point out that in the last 30 minutes I realized that when I >>> tried to insmod click.ko (with stock click 2.0.1, which I did finally >>> get to compile) without first running insmod proclikefs.ks -- once I >>> did things in the right order it seems to be working (I'm finalizing a >>> config right now to test). That said, the compile issues for the >>> current git versions still exist, and I'd much rather be running up to >>> date code than stock 2.0.1. Thanks for the quick response. >>> >>> -- KS >>> >>> On Sat, Feb 23, 2013 at 11:34 AM, Eddie Kohler wrote: >>>> Hi Keith, sorry for the errors. >>>> >>>> Some questions, since I don't have RHEL to test. >>>> ?Is there a Fedora/Centos version that corresponds? >>>> ?Specific compiler version? >>>> ?Most importantly, have you tried with current git? >>>> >>>> Eddie >>>> >>>> >>>> On Fri, Feb 22, 2013 at 11:41 PM, Keith Schoenefeld >>>> wrote: >>>>> I'm running RHEL 6.4 with a RedHat kernel. I've worked through >>>>> various compile issues working on the 2.0.1 release code, including >>>>> having to recompile the kernel from source with NR_CPUS set to 8 >>>>> instead of 4096, adding the NETREG line to fixincludes.pl, etc. I >>>>> have 2.0.1 compiling now with no issues, but once I install and run >>>>> insmod click.ko I get the error "insmod: error inserting 'click.ko': >>>>> -1 Cannot allocate memory". I tried running strip -g as suggested in >>>>> another post, but then I get "insmod: error inserting 'click.ko': -1 >>>>> Unknown symbol in module". I've also tried using the latest git >>>>> repository. Using the following configure command: >>>>> >>>>> ./configure --prefix=/opt/click-kohler-6aa1787 --disable-userlevel >>>>> --enable-multithread >>>>> >>>>> I end up with the following errors (I ran make once to compile >>>>> everything that would compile, then again to just get the list of >>>>> errors): >>>>> >>>>> >>>>> # make linuxmodule >>>>> make[1]: Entering directory `/root/kohler-click-6aa1787/linuxmodule' >>>>> make -C /lib/modules/2.6.32-358.el6.click.x86_64/build >>>>> M=/root/kohler-click-6aa1787/linuxmodule modules >>>>> make[2]: Entering directory `/usr/src/kernels/2.6.32-358.el6.click.x86_64' >>>>> CXX [M] tohost.o >>>>> In file included from >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/linux/tcp.h:186, >>>>> from >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/linux/ipv6.h:220, >>>>> from >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/ip.h:344, >>>>> from >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/xfrm.h:23, >>>>> from >>>>> /root/kohler-click-6aa1787/linuxmodule/../elements/linuxmodule/tohost.cc:28: >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h: >>>>> In function ?void inet_csk_clear_xmit_timer(sock*, int)?: >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>> error: too many initializers for ?_ddebug? >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h: >>>>> In function ?void inet_csk_reset_xmit_timer(sock*, int, long unsigned >>>>> int, long unsigned int)?: >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>> error: too many initializers for ?_ddebug? >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>> error: too many initializers for ?_ddebug? >>>>> CREATE /root/kohler-click-6aa1787/linuxmodule/ksyms.c >>>>> nm: 'tohost.o': No such file >>>>> CC [M] ksyms.o >>>>> LD [M] /root/kohler-click-6aa1787/linuxmodule/click.o >>>>> ld: /root/kohler-click-6aa1787/linuxmodule/tohost.o: No such file: No >>>>> such file or directory >>>>> make[3]: *** [/root/kohler-click-6aa1787/linuxmodule/click.o] Error 1 >>>>> make[2]: *** [_module_/root/kohler-click-6aa1787/linuxmodule] Error 2 >>>>> make[2]: Leaving directory `/usr/src/kernels/2.6.32-358.el6.click.x86_64' >>>>> make[1]: *** [all] Error 2 >>>>> make[1]: Leaving directory `/root/kohler-click-6aa1787/linuxmodule' >>>>> make: *** [linuxmodule] Error 2 >>>>> >>>>> I'm not sure where to go with the DEBUG_HASH, DEBUG_HASH2, etc. errors >>>>> and google hasn't helped me at all. Does anyone have any >>>>> recommendations for getting this working? >>>>> >>>>> -- KS >>>>> >>>>> _______________________________________________ >>>>> click mailing list >>>>> click at amsterdam.lcs.mit.edu >>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click From larsbro at gmail.com Sun Mar 10 18:28:19 2013 From: larsbro at gmail.com (Lars Bro) Date: Sun, 10 Mar 2013 23:28:19 +0100 Subject: [Click] I thought this would be illegal Message-ID: Hi, I thought that Click would deny the following construct m::MyElement() m[0] -> [0]m; in the case where flow_code() would be "x/x" but allow it if flow_code() were "x/y" But it seems to me that flow_code is only used for Router::downstream and upstream methods. Is that correctly understood? From larsbro at gmail.com Sun Mar 10 18:39:37 2013 From: larsbro at gmail.com (Lars Bro) Date: Sun, 10 Mar 2013 23:39:37 +0100 Subject: [Click] When does hotconfig apply Message-ID: Hi, I tried hotconfig like below: # click-install << ! RatedSource(RATE 110) -> lars::Queue() -> Shaper(100) -> Discard(); ! Let it run for a while, use Clicky to see that the queue fills up. # click-install --hotconfig << ! RatedSource(RATE 110) -> lars::Queue() -> Shaper(100) -> Discard(); RatedSource() -> Discard(); ! use Clicky to look at the Queue. Now it is empty, and I would have expected it to still contain the piled up packets. The documentation says that Queue should support hotswap, so I guess I am missing something yours Lars Bro From larsbro at gmail.com Sun Mar 10 18:44:38 2013 From: larsbro at gmail.com (Lars Bro) Date: Sun, 10 Mar 2013 23:44:38 +0100 Subject: [Click] Click! on S/390 Message-ID: Hi, I am going to try to run Click! on Debian Wheezy S/390 I just want to know if anybody on the list has experience to share. I plan to use Click to implement the backbone in a system with many virtualized guests. yours, Lars Bro From cliff at meraki.com Sun Mar 10 20:22:03 2013 From: cliff at meraki.com (Cliff Frey) Date: Sun, 10 Mar 2013 17:22:03 -0700 Subject: [Click] I thought this would be illegal In-Reply-To: References: Message-ID: Yeah, I believe that you understand this correctly. Even if the flow code is "x/x", this could theoretically make some sense on an element that had a chance of dropping the packet (i.e. a DecIPTTL or RandomSample element). On Sun, Mar 10, 2013 at 3:28 PM, Lars Bro wrote: > Hi, > > I thought that Click would deny the following construct > > m::MyElement() > m[0] -> [0]m; > > in the case where flow_code() would be "x/x" > > but allow it if flow_code() were "x/y" > > > But it seems to me that flow_code is only used for Router::downstream and > upstream methods. Is that correctly understood? > _______________________________________________ > click mailing list > click at amsterdam.lcs.mit.edu > https://amsterdam.lcs.mit.edu/mailman/listinfo/click > From ms08 at student.aau.dk Wed Mar 13 12:03:29 2013 From: ms08 at student.aau.dk (Martin Dam - NDS) Date: Wed, 13 Mar 2013 17:03:29 +0100 Subject: [Click] FromDevice(lo) -> ... -> ToHost(lo) issue Message-ID: <5140A351.8050707@student.aau.dk> I am trying to build a kernel configuration which intercept packets on the loopback and process certain UDP packets. Other packets should just be send back to host for other applications to process. My configuration is: FromDevice(lo) -> Strip(14) -> CheckIPHeader() -> IPPrint('lo in') -> loClass :: Classifier(udp dst port 3070, -) -> MyElement; loClass[1] -> IPPrint('lo out') -> Unstrip(14) -> ToHost(lo); I am able to observe the packets on loopback, but the other applications with open sockets doesn't get the packets. If I uninstall my configuration it works as intended via loopback. Any help would be much appreciated. Bests, Martin D From keith at schoenefeld.org Thu Mar 14 00:31:03 2013 From: keith at schoenefeld.org (Keith Schoenefeld) Date: Wed, 13 Mar 2013 23:31:03 -0500 Subject: [Click] Compile issues on RHEL 6.4 (2.6.32) In-Reply-To: References: Message-ID: My apologies for the extended delay in responding. I ended up getting Click 1.0.2 to compile with some modifications, but get kernel crashes at install with some regularity. As best I can tell, the crashes are actually caused by the igb driver, not Click. If Click installs and doesn't crash within the first 5 seconds, it stays rock solid. Next time I need to restart Click on this server, I'll see if the latest repository works for me. As for the memory issue, I have always had to run strip -g on the click.ko file in order to get it to install. -- KS On Fri, Mar 1, 2013 at 10:47 PM, Eddie Kohler wrote: > Hi Keith, > > I got a look at this. There were a couple problems with that kernel > that our rewriting, etc. tools could not handle. I were able to fix > them and get a pretty clean automatic compilation. These patches are > committed. However, an installation attempt fails with "out of > memory". I haven't debugged further. Have you gotten further? > > BTW, I didn't realize you were downloading the current tarball from > github. You did get the latest commit, but I'd strongly advise you to > install git, which will make it easier to update. > Eddie > > > On Sat, Feb 23, 2013 at 2:14 PM, Keith Schoenefeld > wrote: >> I should have said "I grabbed the latest tarball from github". More >> specifically, I downloaded the tarball provided at the URI >> "https://github.com/kohler/click/tarball/master". Since the directory >> name created when I extracted the tarball (kohler-click-6aa1787) >> matches the latest commit, I assumed this matched what I could obtain >> using git clone. Should I install git and run a git clone just to be >> sure? >> >> -- KS >> >> On Sat, Feb 23, 2013 at 12:25 PM, Eddie Kohler wrote: >>> Just to double check for a misunderstanding: You don't grab a tarball >>> from git. Are you using the click-2.0.1 release, or the current git >>> sources, which you can obtain by `git clone` from github.com? >>> >>> >>> On Sat, Feb 23, 2013 at 1:11 PM, Keith Schoenefeld >>> wrote: >>>> - My understanding is that CentOS is built using the source RPMS for >>>> RHEL, so CentOS 6.x would be as close to identical as possible with >>>> RHEL 6.x. >>>> - I used the stock compiler that comes with RHEL 6.4 >>>> - I grabbed the latest tarball from git and used that, yes. >>>> >>>> I will point out that in the last 30 minutes I realized that when I >>>> tried to insmod click.ko (with stock click 2.0.1, which I did finally >>>> get to compile) without first running insmod proclikefs.ks -- once I >>>> did things in the right order it seems to be working (I'm finalizing a >>>> config right now to test). That said, the compile issues for the >>>> current git versions still exist, and I'd much rather be running up to >>>> date code than stock 2.0.1. Thanks for the quick response. >>>> >>>> -- KS >>>> >>>> On Sat, Feb 23, 2013 at 11:34 AM, Eddie Kohler wrote: >>>>> Hi Keith, sorry for the errors. >>>>> >>>>> Some questions, since I don't have RHEL to test. >>>>> ?Is there a Fedora/Centos version that corresponds? >>>>> ?Specific compiler version? >>>>> ?Most importantly, have you tried with current git? >>>>> >>>>> Eddie >>>>> >>>>> >>>>> On Fri, Feb 22, 2013 at 11:41 PM, Keith Schoenefeld >>>>> wrote: >>>>>> I'm running RHEL 6.4 with a RedHat kernel. I've worked through >>>>>> various compile issues working on the 2.0.1 release code, including >>>>>> having to recompile the kernel from source with NR_CPUS set to 8 >>>>>> instead of 4096, adding the NETREG line to fixincludes.pl, etc. I >>>>>> have 2.0.1 compiling now with no issues, but once I install and run >>>>>> insmod click.ko I get the error "insmod: error inserting 'click.ko': >>>>>> -1 Cannot allocate memory". I tried running strip -g as suggested in >>>>>> another post, but then I get "insmod: error inserting 'click.ko': -1 >>>>>> Unknown symbol in module". I've also tried using the latest git >>>>>> repository. Using the following configure command: >>>>>> >>>>>> ./configure --prefix=/opt/click-kohler-6aa1787 --disable-userlevel >>>>>> --enable-multithread >>>>>> >>>>>> I end up with the following errors (I ran make once to compile >>>>>> everything that would compile, then again to just get the list of >>>>>> errors): >>>>>> >>>>>> >>>>>> # make linuxmodule >>>>>> make[1]: Entering directory `/root/kohler-click-6aa1787/linuxmodule' >>>>>> make -C /lib/modules/2.6.32-358.el6.click.x86_64/build >>>>>> M=/root/kohler-click-6aa1787/linuxmodule modules >>>>>> make[2]: Entering directory `/usr/src/kernels/2.6.32-358.el6.click.x86_64' >>>>>> CXX [M] tohost.o >>>>>> In file included from >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/linux/tcp.h:186, >>>>>> from >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/linux/ipv6.h:220, >>>>>> from >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/ip.h:344, >>>>>> from >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/xfrm.h:23, >>>>>> from >>>>>> /root/kohler-click-6aa1787/linuxmodule/../elements/linuxmodule/tohost.cc:28: >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h: >>>>>> In function ?void inet_csk_clear_xmit_timer(sock*, int)?: >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>>> error: too many initializers for ?_ddebug? >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h: >>>>>> In function ?void inet_csk_reset_xmit_timer(sock*, int, long unsigned >>>>>> int, long unsigned int)?: >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>>> error: too many initializers for ?_ddebug? >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>>> error: too many initializers for ?_ddebug? >>>>>> CREATE /root/kohler-click-6aa1787/linuxmodule/ksyms.c >>>>>> nm: 'tohost.o': No such file >>>>>> CC [M] ksyms.o >>>>>> LD [M] /root/kohler-click-6aa1787/linuxmodule/click.o >>>>>> ld: /root/kohler-click-6aa1787/linuxmodule/tohost.o: No such file: No >>>>>> such file or directory >>>>>> make[3]: *** [/root/kohler-click-6aa1787/linuxmodule/click.o] Error 1 >>>>>> make[2]: *** [_module_/root/kohler-click-6aa1787/linuxmodule] Error 2 >>>>>> make[2]: Leaving directory `/usr/src/kernels/2.6.32-358.el6.click.x86_64' >>>>>> make[1]: *** [all] Error 2 >>>>>> make[1]: Leaving directory `/root/kohler-click-6aa1787/linuxmodule' >>>>>> make: *** [linuxmodule] Error 2 >>>>>> >>>>>> I'm not sure where to go with the DEBUG_HASH, DEBUG_HASH2, etc. errors >>>>>> and google hasn't helped me at all. Does anyone have any >>>>>> recommendations for getting this working? >>>>>> >>>>>> -- KS >>>>>> >>>>>> _______________________________________________ >>>>>> click mailing list >>>>>> click at amsterdam.lcs.mit.edu >>>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click From ekohler at gmail.com Thu Mar 14 07:47:14 2013 From: ekohler at gmail.com (Eddie Kohler) Date: Thu, 14 Mar 2013 07:47:14 -0400 Subject: [Click] Compile issues on RHEL 6.4 (2.6.32) In-Reply-To: References: Message-ID: I assume you mean 2.0.1, not 1.0.2!! I'll try strip -g. Eddie On Thu, Mar 14, 2013 at 12:31 AM, Keith Schoenefeld wrote: > My apologies for the extended delay in responding. I ended up getting > Click 1.0.2 to compile with some modifications, but get kernel crashes > at install with some regularity. As best I can tell, the crashes are > actually caused by the igb driver, not Click. If Click installs and > doesn't crash within the first 5 seconds, it stays rock solid. > > Next time I need to restart Click on this server, I'll see if the > latest repository works for me. As for the memory issue, I have > always had to run strip -g on the click.ko file in order to get it to > install. > > -- KS > > On Fri, Mar 1, 2013 at 10:47 PM, Eddie Kohler wrote: >> Hi Keith, >> >> I got a look at this. There were a couple problems with that kernel >> that our rewriting, etc. tools could not handle. I were able to fix >> them and get a pretty clean automatic compilation. These patches are >> committed. However, an installation attempt fails with "out of >> memory". I haven't debugged further. Have you gotten further? >> >> BTW, I didn't realize you were downloading the current tarball from >> github. You did get the latest commit, but I'd strongly advise you to >> install git, which will make it easier to update. >> Eddie >> >> >> On Sat, Feb 23, 2013 at 2:14 PM, Keith Schoenefeld >> wrote: >>> I should have said "I grabbed the latest tarball from github". More >>> specifically, I downloaded the tarball provided at the URI >>> "https://github.com/kohler/click/tarball/master". Since the directory >>> name created when I extracted the tarball (kohler-click-6aa1787) >>> matches the latest commit, I assumed this matched what I could obtain >>> using git clone. Should I install git and run a git clone just to be >>> sure? >>> >>> -- KS >>> >>> On Sat, Feb 23, 2013 at 12:25 PM, Eddie Kohler wrote: >>>> Just to double check for a misunderstanding: You don't grab a tarball >>>> from git. Are you using the click-2.0.1 release, or the current git >>>> sources, which you can obtain by `git clone` from github.com? >>>> >>>> >>>> On Sat, Feb 23, 2013 at 1:11 PM, Keith Schoenefeld >>>> wrote: >>>>> - My understanding is that CentOS is built using the source RPMS for >>>>> RHEL, so CentOS 6.x would be as close to identical as possible with >>>>> RHEL 6.x. >>>>> - I used the stock compiler that comes with RHEL 6.4 >>>>> - I grabbed the latest tarball from git and used that, yes. >>>>> >>>>> I will point out that in the last 30 minutes I realized that when I >>>>> tried to insmod click.ko (with stock click 2.0.1, which I did finally >>>>> get to compile) without first running insmod proclikefs.ks -- once I >>>>> did things in the right order it seems to be working (I'm finalizing a >>>>> config right now to test). That said, the compile issues for the >>>>> current git versions still exist, and I'd much rather be running up to >>>>> date code than stock 2.0.1. Thanks for the quick response. >>>>> >>>>> -- KS >>>>> >>>>> On Sat, Feb 23, 2013 at 11:34 AM, Eddie Kohler wrote: >>>>>> Hi Keith, sorry for the errors. >>>>>> >>>>>> Some questions, since I don't have RHEL to test. >>>>>> ?Is there a Fedora/Centos version that corresponds? >>>>>> ?Specific compiler version? >>>>>> ?Most importantly, have you tried with current git? >>>>>> >>>>>> Eddie >>>>>> >>>>>> >>>>>> On Fri, Feb 22, 2013 at 11:41 PM, Keith Schoenefeld >>>>>> wrote: >>>>>>> I'm running RHEL 6.4 with a RedHat kernel. I've worked through >>>>>>> various compile issues working on the 2.0.1 release code, including >>>>>>> having to recompile the kernel from source with NR_CPUS set to 8 >>>>>>> instead of 4096, adding the NETREG line to fixincludes.pl, etc. I >>>>>>> have 2.0.1 compiling now with no issues, but once I install and run >>>>>>> insmod click.ko I get the error "insmod: error inserting 'click.ko': >>>>>>> -1 Cannot allocate memory". I tried running strip -g as suggested in >>>>>>> another post, but then I get "insmod: error inserting 'click.ko': -1 >>>>>>> Unknown symbol in module". I've also tried using the latest git >>>>>>> repository. Using the following configure command: >>>>>>> >>>>>>> ./configure --prefix=/opt/click-kohler-6aa1787 --disable-userlevel >>>>>>> --enable-multithread >>>>>>> >>>>>>> I end up with the following errors (I ran make once to compile >>>>>>> everything that would compile, then again to just get the list of >>>>>>> errors): >>>>>>> >>>>>>> >>>>>>> # make linuxmodule >>>>>>> make[1]: Entering directory `/root/kohler-click-6aa1787/linuxmodule' >>>>>>> make -C /lib/modules/2.6.32-358.el6.click.x86_64/build >>>>>>> M=/root/kohler-click-6aa1787/linuxmodule modules >>>>>>> make[2]: Entering directory `/usr/src/kernels/2.6.32-358.el6.click.x86_64' >>>>>>> CXX [M] tohost.o >>>>>>> In file included from >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/linux/tcp.h:186, >>>>>>> from >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/linux/ipv6.h:220, >>>>>>> from >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/ip.h:344, >>>>>>> from >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/xfrm.h:23, >>>>>>> from >>>>>>> /root/kohler-click-6aa1787/linuxmodule/../elements/linuxmodule/tohost.cc:28: >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h: >>>>>>> In function ?void inet_csk_clear_xmit_timer(sock*, int)?: >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:208: >>>>>>> error: too many initializers for ?_ddebug? >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h: >>>>>>> In function ?void inet_csk_reset_xmit_timer(sock*, int, long unsigned >>>>>>> int, long unsigned int)?: >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:224: >>>>>>> error: too many initializers for ?_ddebug? >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>>>> error: ?DEBUG_HASH? was not declared in this scope >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>>>> error: ?DEBUG_HASH2? was not declared in this scope >>>>>>> /root/kohler-click-6aa1787/include/click-linuxmodule/include0/net/inet_connection_sock.h:241: >>>>>>> error: too many initializers for ?_ddebug? >>>>>>> CREATE /root/kohler-click-6aa1787/linuxmodule/ksyms.c >>>>>>> nm: 'tohost.o': No such file >>>>>>> CC [M] ksyms.o >>>>>>> LD [M] /root/kohler-click-6aa1787/linuxmodule/click.o >>>>>>> ld: /root/kohler-click-6aa1787/linuxmodule/tohost.o: No such file: No >>>>>>> such file or directory >>>>>>> make[3]: *** [/root/kohler-click-6aa1787/linuxmodule/click.o] Error 1 >>>>>>> make[2]: *** [_module_/root/kohler-click-6aa1787/linuxmodule] Error 2 >>>>>>> make[2]: Leaving directory `/usr/src/kernels/2.6.32-358.el6.click.x86_64' >>>>>>> make[1]: *** [all] Error 2 >>>>>>> make[1]: Leaving directory `/root/kohler-click-6aa1787/linuxmodule' >>>>>>> make: *** [linuxmodule] Error 2 >>>>>>> >>>>>>> I'm not sure where to go with the DEBUG_HASH, DEBUG_HASH2, etc. errors >>>>>>> and google hasn't helped me at all. Does anyone have any >>>>>>> recommendations for getting this working? >>>>>>> >>>>>>> -- KS >>>>>>> >>>>>>> _______________________________________________ >>>>>>> click mailing list >>>>>>> click at amsterdam.lcs.mit.edu >>>>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click From dmi216 at nyu.edu Thu Mar 14 18:37:10 2013 From: dmi216 at nyu.edu (David M Iserovich) Date: Thu, 14 Mar 2013 18:37:10 -0400 Subject: [Click] Funny error compiling Click 2.0.1 with kernel module on 3.5.0-22-generic Message-ID: Hi, I'm new to Click and I'm having a lot of fun with it in user-land, but I get a funny error compiling with the linux module on Linux 3.5.0-22-generic #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux Which is Ubuntu 12.10. My compilation command, after copying out the System.map file to the current directory, and putting the current Debian config from /boot/config-`uname -r`. My exact compilation command is ./configure --with-linux=/usr/src/linux-headers-`uname -r` --with-linux-map=./System.map-`uname -r` --enable-linuxmodule The error is, from the configure output configure: making C++-safe versions of Linux kernel headers (may take a while) Usage: fixincludes.pl -o OUTPUTDIR CFLAGS configure: error: ============================================== fixincludes.pl execution failed. ============================================== which seems like fixinclude.pl is getting badly formatted args for some reason, since it outputs that "Usage:" message. Can anyone help? From ndritsos at gmail.com Fri Mar 15 08:49:50 2013 From: ndritsos at gmail.com (ndritsos) Date: Fri, 15 Mar 2013 14:49:50 +0200 Subject: [Click] patch error Message-ID: <514318EE.5080104@gmail.com> hello all , Iam new to Click , and i try to run click on the kernel mode my kernel version is 2.6.24.19-generic iam following these steps : 1) cd ~ 2) sudo bash 3) git clone git://github.com/kohler/click.git 4) git clone git://github.com/kohler/click-packages.git 5) cd /usr/src/linux-headers-2.6.24.7-generic 6) patch -p0 -b < /home/niko/click/etc/linux-2.6.24.7-patch At this step i have the error that is below : can't find file to patch at input line 5 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------------------------- |diff --git a/arch/arm/mach-iop13xx/irq.c b/arch/arm/mach-iop13xx/irq.c |index 69f07b2..e217167 100644 |--- a/arch/arm/mach-iop13xx/irq.c |+++ b/arch/arm/mach-iop13xx/irq.c ------------------------------------------- File to patch: My kernel version is 2.6.24.19-generic and i use linux-2.6.24.7-patch Any idea someone why i have this error ? may the problem be the kernel version ? thank you in advance From ndritsos at gmail.com Fri Mar 15 08:51:08 2013 From: ndritsos at gmail.com (ndritsos) Date: Fri, 15 Mar 2013 14:51:08 +0200 Subject: [Click] patch error In-Reply-To: <514318EE.5080104@gmail.com> References: <514318EE.5080104@gmail.com> Message-ID: <5143193C.4080606@gmail.com> hello all , Iam new to Click , and i try to run click on the kernel mode my kernel version is 2.6.24.19-generic iam following these steps : 1) cd ~ 2) sudo bash 3) git clone git://github.com/kohler/click.git 4) git clone git://github.com/kohler/click-packages.git 5) cd /usr/src/linux-headers-2.6.24.7-generic 6) patch -p0 -b < /home/niko/click/etc/linux-2.6.24.7-patch At this step i have the error that is below : can't find file to patch at input line 5 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------------------------- |diff --git a/arch/arm/mach-iop13xx/irq.c b/arch/arm/mach-iop13xx/irq.c |index 69f07b2..e217167 100644 |--- a/arch/arm/mach-iop13xx/irq.c |+++ b/arch/arm/mach-iop13xx/irq.c ------------------------------------------- File to patch: My kernel version is 2.6.24.19-generic and i use linux-2.6.24.7-patch Any idea someone why i have this error ? may the problem be the kernel version ? thank you in advance From ndritsos at gmail.com Fri Mar 15 09:42:10 2013 From: ndritsos at gmail.com (ndritsos) Date: Fri, 15 Mar 2013 15:42:10 +0200 Subject: [Click] patch error In-Reply-To: <5143193C.4080606@gmail.com> References: <5143193C.4080606@gmail.com> Message-ID: <51432532.10500@gmail.com> hello all , Iam new to Click , and i try to run click on the kernel mode my kernel version is 2.6.24.19-generic iam following these steps : 1) cd ~ 2) sudo bash 3) git clone git://github.com/kohler/click.git 4) git clone git://github.com/kohler/click-packages.git 5) cd /usr/src/linux-headers-2.6.24.7-generic 6) patch -p0 -b < /home/niko/click/etc/linux-2.6.24.7-patch At this step i have the error that is below : can't find file to patch at input line 5 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------------------------- |diff --git a/arch/arm/mach-iop13xx/irq.c b/arch/arm/mach-iop13xx/irq.c |index 69f07b2..e217167 100644 |--- a/arch/arm/mach-iop13xx/irq.c |+++ b/arch/arm/mach-iop13xx/irq.c ------------------------------------------- File to patch: My kernel version is 2.6.24.19-generic and i use linux-2.6.24.7-patch Any idea someone why i have this error ? may the problem be the kernel version ? thank you in advance From avinash.sridharan at gmail.com Fri Mar 15 14:37:01 2013 From: avinash.sridharan at gmail.com (Avinash Sridharan) Date: Fri, 15 Mar 2013 11:37:01 -0700 Subject: [Click] Funny error compiling Click 2.0.1 with kernel module on 3.5.0-22-generic In-Reply-To: References: Message-ID: Hi David, I don't click compiles cleanly with all versions of linux-3.0 kernel. I have tried compiling with 3.2 kernel and it compiles cleanly, however for 3.4 I was running into compile errors. My guess is that the data structures used by click have changed after 3.2 causing click compilation to fail. I would suggest get a plain vanilla 3.2 kernel (from kernel.org) and try compiling click with that. On Thu, Mar 14, 2013 at 3:37 PM, David M Iserovich wrote: > Hi, I'm new to Click and I'm having a lot of fun with it in user-land, but > I get a funny error compiling with the linux module on > > Linux 3.5.0-22-generic #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 x86_64 > x86_64 x86_64 GNU/Linux > > Which is Ubuntu 12.10. My compilation command, after copying out the > System.map file to the current directory, and putting the current Debian > config from /boot/config-`uname -r`. My exact compilation command is > > ./configure --with-linux=/usr/src/linux-headers-`uname -r` > --with-linux-map=./System.map-`uname -r` --enable-linuxmodule > > The error is, from the configure output > > configure: making C++-safe versions of Linux kernel headers (may take a > while) > Usage: fixincludes.pl -o OUTPUTDIR CFLAGS > configure: error: > ============================================== > > fixincludes.pl execution failed. > > ============================================== > > > which seems like fixinclude.pl is getting badly formatted args for some > reason, since it outputs that "Usage:" message. > > Can anyone help? > _______________________________________________ > click mailing list > click at amsterdam.lcs.mit.edu > https://amsterdam.lcs.mit.edu/mailman/listinfo/click > From ekohler at gmail.com Fri Mar 15 18:16:46 2013 From: ekohler at gmail.com (Eddie Kohler) Date: Fri, 15 Mar 2013 18:16:46 -0400 Subject: [Click] Funny error compiling Click 2.0.1 with kernel module on 3.5.0-22-generic In-Reply-To: References: Message-ID: Hi David, We might be able to help. Please attach your config.log. Eddie On Thu, Mar 14, 2013 at 6:37 PM, David M Iserovich wrote: > Hi, I'm new to Click and I'm having a lot of fun with it in user-land, but > I get a funny error compiling with the linux module on > > Linux 3.5.0-22-generic #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 x86_64 > x86_64 x86_64 GNU/Linux > > Which is Ubuntu 12.10. My compilation command, after copying out the > System.map file to the current directory, and putting the current Debian > config from /boot/config-`uname -r`. My exact compilation command is > > ./configure --with-linux=/usr/src/linux-headers-`uname -r` > --with-linux-map=./System.map-`uname -r` --enable-linuxmodule > > The error is, from the configure output > > configure: making C++-safe versions of Linux kernel headers (may take a > while) > Usage: fixincludes.pl -o OUTPUTDIR CFLAGS > configure: error: > ============================================== > > fixincludes.pl execution failed. > > ============================================== > > > which seems like fixinclude.pl is getting badly formatted args for some > reason, since it outputs that "Usage:" message. > > Can anyone help? > _______________________________________________ > click mailing list > click at amsterdam.lcs.mit.edu > https://amsterdam.lcs.mit.edu/mailman/listinfo/click From ekohler at gmail.com Fri Mar 15 20:24:01 2013 From: ekohler at gmail.com (Eddie Kohler) Date: Fri, 15 Mar 2013 20:24:01 -0400 Subject: [Click] Funny error compiling Click 2.0.1 with kernel module on 3.5.0-22-generic In-Reply-To: References: Message-ID: For what it's worth, the current Click head compiles and installs cleanly on Ubuntu 12.10 x86-64. (3.5.) I had to tweak a couple things to get this. I don't recommend using --with-linux=/usr/src/.... Click will default to getting the sources from /lib/modules, which are more complete than the --with-linux line. Eddie On Fri, Mar 15, 2013 at 6:16 PM, Eddie Kohler wrote: > Hi David, > > We might be able to help. Please attach your config.log. > > Eddie > > > On Thu, Mar 14, 2013 at 6:37 PM, David M Iserovich wrote: >> Hi, I'm new to Click and I'm having a lot of fun with it in user-land, but >> I get a funny error compiling with the linux module on >> >> Linux 3.5.0-22-generic #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 x86_64 >> x86_64 x86_64 GNU/Linux >> >> Which is Ubuntu 12.10. My compilation command, after copying out the >> System.map file to the current directory, and putting the current Debian >> config from /boot/config-`uname -r`. My exact compilation command is >> >> ./configure --with-linux=/usr/src/linux-headers-`uname -r` >> --with-linux-map=./System.map-`uname -r` --enable-linuxmodule >> >> The error is, from the configure output >> >> configure: making C++-safe versions of Linux kernel headers (may take a >> while) >> Usage: fixincludes.pl -o OUTPUTDIR CFLAGS >> configure: error: >> ============================================== >> >> fixincludes.pl execution failed. >> >> ============================================== >> >> >> which seems like fixinclude.pl is getting badly formatted args for some >> reason, since it outputs that "Usage:" message. >> >> Can anyone help? >> _______________________________________________ >> click mailing list >> click at amsterdam.lcs.mit.edu >> https://amsterdam.lcs.mit.edu/mailman/listinfo/click From p2pnet10 at gmail.com Sat Mar 16 15:32:10 2013 From: p2pnet10 at gmail.com (pplive) Date: Sat, 16 Mar 2013 15:32:10 -0400 Subject: [Click] ToDevice via a GRE tunnel Message-ID: Hey, I would like to send data via a created GRE tunnel. The .click script is: InfiniteSource(DATA \<00 00 c0 ae 67 ef 00 00 00 00 00 00 08 00 45 00 00 28 00 00 00 00 40 11 77 c3 01 00 00 01 02 00 00 02 13 69 13 69 00 14 d6 41 55 44 50 20 70 61 63 6b 65 74 21 0a>, LIMIT 5, STOP true) // -> Strip(14) // -> Align(4, 0) // in case we're not on x86 // -> CheckIPHeader(BADSRC 18.26.4.255 2.255.255.255 1.255.255.255) // -> Print(ok) ->EtherEncap(0x0800,0:0:0:0:0:1,0:0:0:0:0:2) -> ToDevice(gre1); Also, my ifconfig output is: eth0 Link encap:Ethernet HWaddr 00:0e:0c:66:83:d0 inet addr:10.0.1.1 Bcast:10.0.1.255 Mask:255.255.255.0 inet6 addr: fe80::20e:cff:fe66:83d0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3326 errors:0 dropped:0 overruns:0 frame:0 TX packets:3907 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:275022 (275.0 KB) TX bytes:3419244 (3.4 MB) eth3 Link encap:Ethernet HWaddr 00:11:43:d6:d3:a5 inet addr:192.168.1.119 Bcast:192.168.3.255 Mask:255.255.252.0 inet6 addr: fe80::211:43ff:fed6:d3a5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:12654 errors:0 dropped:0 overruns:0 frame:0 TX packets:6741 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1950463 (1.9 MB) TX bytes:1108595 (1.1 MB) gre1 Link encap:UNSPEC HWaddr 0A-00-01-01-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:192.168.1.2 P-t-P:192.168.1.1 Mask:255.255.255.252 UP POINTOPOINT RUNNING NOARP MTU:1476 Metric:1 RX packets:1113 errors:0 dropped:0 overruns:0 frame:0 TX packets:1417 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:63160 (63.1 KB) TX bytes:2003332 (2.0 MB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:134 errors:0 dropped:0 overruns:0 frame:0 TX packets:134 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:14743 (14.7 KB) TX bytes:14743 (14.7 KB) I have created a point-to-point gre tunnel with some other node (gre1). The eth0 is the data NIC and eth3 is the control NIC in Deterlab. When I try to execute the .click file, the output is : ToDevice(gre1): Invalid argument ToDevice(gre1): Invalid argument ToDevice(gre1): Invalid argument ToDevice(gre1): Invalid argument ToDevice(gre1): Invalid argument It seems that ToDevice cannot send data over the created 'gre1' tunnel. Could you please provide some method to solve it? Thanks! Best regads, Alex From kpele_ntua at yahoo.com Sun Mar 17 19:42:48 2013 From: kpele_ntua at yahoo.com (Kostas) Date: Sun, 17 Mar 2013 16:42:48 -0700 (PDT) Subject: [Click] link Message-ID: <1363563768.84112.YahooMailNeo@web121001.mail.ne1.yahoo.com> http://www.fotoateliertarmstedt.de/ege/mhh From kpele_ntua at yahoo.com Sun Mar 17 19:42:48 2013 From: kpele_ntua at yahoo.com (Kostas) Date: Sun, 17 Mar 2013 16:42:48 -0700 (PDT) Subject: [Click] link Message-ID: <1363563768.81754.YahooMailNeo@web121003.mail.ne1.yahoo.com> http://idescapanama.com/giveoo/bzvbzartuurven From mail at richard-neumann.de Tue Mar 19 12:49:56 2013 From: mail at richard-neumann.de (Richard Neumann) Date: Tue, 19 Mar 2013 17:49:56 +0100 Subject: [Click] Poor forwarding performance with click in a paravirtualized Xen environment Message-ID: <1363711796.5986.13.camel@localhost.localdomain> Hello folks, I have encountered the problem that click is scaling quite badly inside a Xen dom-U paravirtualized environment. The setup is like: [Sender] --10G-Link--> [Router] --10G-Link--> [Receiver] The router is configured as a virtual machine as follows: ### FILE /etc/xen/vm/opensuse12-1 ### name="opensuse12-1" description="None" uuid="6e2e1ffe-2faf-f3f4-bfc2-d12a76c99093" memory=2048 maxmem=2048 vcpus=1 on_poweroff="destroy" on_reboot="restart" on_crash="destroy" localtime=0 keymap="de" builder="linux" bootloader="/usr/bin/pygrub" bootargs="" extra="xencons=tty " disk=[ 'file:/var/lib/xen/images/opensuse12-1/xvda,xvda,w', ] vif=[ 'mac=00:16:3e:4f:b4:7c,bridge=br0', ] pci=['02:00.0', '02:00.1'] nographic=1 ### EOF ### Where the two PCI devices are the 10-G networking cards connected to the sender, respectively to the receiver. Inside the virtual machine, I run click (v. 2.0.1) in user mode with a trivial forwarding configuration: FromDevice(eth2) -> c1::AverageCounter() -> Queue(1000000) -> ToDevice(eth1); where eth2 is the interface connected to the sender and eth1 is the interface connected to the receiver. On the receiver I then run pkt-gen from netmap (http://info.iet.unipi.it/~luigi/netmap/) in receiver mode and on the sender I run pkt-gen in sender mode with the following results: Sender: /usr/src/netmap-linux/net/netmap/pkt-gen -i eth7 -t 500111222 -l 64 -w 5 main [874] map size is 203064 Kb main [906] mmapping 203064 Kbytes Sending on eth7: 6 queues, 1 threads, 1 cpus and affinity -1. 10.0.0.1 -> 10.1.0.1 (00:1b:21:d5:6f:ec -> ff:ff:ff:ff:ff:ff) main [953] Wait 5 secs for phy reset main [955] Ready... main [1074] 14141927 pps [...] main [1074] 11500703 pps main [1074] 3953080 pps Sent 500111222 packets, 64 bytes each, in 41.34 seconds. Speed: 12.10Mpps. Bandwidth: 6.19Gbps. Receiver: pkt-gen -i eth7 main [877] map size is 203064 Kb main [909] mmapping 203064 Kbytes Receiving from eth7: 6 queues, 1 threads, 1 cpus and affinity -1. main [956] Wait 2 secs for phy reset main [958] Ready... main [1077] 0 pps receiver_body [624] waiting for initial packets, poll returns 0 0 main [1077] 0 pps receiver_body [624] waiting for initial packets, poll returns 0 0 main [1077] 0 pps receiver_body [624] waiting for initial packets, poll returns 0 0 receiver_body [624] waiting for initial packets, poll returns 0 0 main [1077] 0 pps main [1077] 112 pps [...] main [1077] 248 pps main [1077] 0 pps Received 4780 packets, in 41.35 seconds. Speed: 115.61pps. If I, instead of click, use a simple linux bridge (using brctl), I get a much higher throughput on the receiver: pkt-gen -i eth7 main [877] map size is 203064 Kb main [909] mmapping 203064 Kbytes Receiving from eth7: 6 queues, 1 threads, 1 cpus and affinity -1. main [956] Wait 2 secs for phy reset main [958] Ready... main [1077] 540069 pps [...] main [1077] 526933 pps main [1077] 0 pps Received 5318241 packets, in 10.00 seconds. Speed: 531.60Kpps. Does anyone know, why click in my case performs so poorly compared to a linux bridge? Best regards, Richard From rizzo at iet.unipi.it Tue Mar 19 14:29:00 2013 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Tue, 19 Mar 2013 19:29:00 +0100 Subject: [Click] Poor forwarding performance with click in a paravirtualized Xen environment In-Reply-To: <1363711796.5986.13.camel@localhost.localdomain> References: <1363711796.5986.13.camel@localhost.localdomain> Message-ID: <20130319182900.GA48640@onelab2.iet.unipi.it> On Tue, Mar 19, 2013 at 05:49:56PM +0100, Richard Neumann wrote: > Hello folks, > > I have encountered the problem that click is scaling quite badly inside > a Xen dom-U paravirtualized environment. > > The setup is like: > > [Sender] --10G-Link--> [Router] --10G-Link--> [Receiver] > > The router is configured as a virtual machine as follows: (summary: only a handful of packets per second when [Router] is a userspace click configuration, and about 500Kpps when you [Router] is a linux bridge. Details at the end) At first sight it seems a case of receive livelock, which does not occur so badly with the linux bridge as it (probably) is helped by NAPI. It is not clear if your userspace click is also using netmap (which may have its own bugs) and/or whether the two interfaces eth1 and eth2 are using pci-passthrough or are emulated/paravirtualized. In any case the 530Kpps you get with linux bridge is probably close to the peak performance you can get, unless (a) Xen gives you access to the real hw (via virtual functions and/or pci-passthrough), and (b) you are using netmap or some very fast os stack bypass to talk to the network interfaces within the virtual machine. I managed to go quite fast within qemu+kvm, see below http://info.iet.unipi.it/~luigi/netmap/talk-google-2013.html http://info.iet.unipi.it/~luigi/papers/20130206-qemu.pdf but that needed a little bit of tweaking here and there (not that i doubt that the Xen folks are smart enough to do something similar). cheers luigi > ### FILE /etc/xen/vm/opensuse12-1 ### > name="opensuse12-1" > description="None" > uuid="6e2e1ffe-2faf-f3f4-bfc2-d12a76c99093" > memory=2048 > maxmem=2048 > vcpus=1 > on_poweroff="destroy" > on_reboot="restart" > on_crash="destroy" > localtime=0 > keymap="de" > builder="linux" > bootloader="/usr/bin/pygrub" > bootargs="" > extra="xencons=tty " > disk=[ 'file:/var/lib/xen/images/opensuse12-1/xvda,xvda,w', ] > vif=[ 'mac=00:16:3e:4f:b4:7c,bridge=br0', ] > pci=['02:00.0', '02:00.1'] > nographic=1 > ### EOF ### > > Where the two PCI devices are the 10-G networking cards connected to the > sender, respectively to the receiver. > > Inside the virtual machine, I run click (v. 2.0.1) in user mode with a > trivial forwarding configuration: > > FromDevice(eth2) -> c1::AverageCounter() -> Queue(1000000) -> > ToDevice(eth1); > > where eth2 is the interface connected to the sender and eth1 is the > interface connected to the receiver. > > On the receiver I then run pkt-gen from netmap > (http://info.iet.unipi.it/~luigi/netmap/) in receiver mode and on the > sender I run pkt-gen in sender mode with the following results: > > Sender: > > /usr/src/netmap-linux/net/netmap/pkt-gen -i eth7 -t 500111222 -l 64 -w 5 > main [874] map size is 203064 Kb > main [906] mmapping 203064 Kbytes > Sending on eth7: 6 queues, 1 threads, 1 cpus and affinity -1. > 10.0.0.1 -> 10.1.0.1 (00:1b:21:d5:6f:ec -> ff:ff:ff:ff:ff:ff) > main [953] Wait 5 secs for phy reset > main [955] Ready... > main [1074] 14141927 pps > [...] > main [1074] 11500703 pps > main [1074] 3953080 pps > Sent 500111222 packets, 64 bytes each, in 41.34 seconds. > Speed: 12.10Mpps. Bandwidth: 6.19Gbps. > > > Receiver: > > pkt-gen -i eth7 > main [877] map size is 203064 Kb > main [909] mmapping 203064 Kbytes > Receiving from eth7: 6 queues, 1 threads, 1 cpus and affinity -1. > main [956] Wait 2 secs for phy reset > main [958] Ready... > main [1077] 0 pps > receiver_body [624] waiting for initial packets, poll returns 0 0 > main [1077] 0 pps > receiver_body [624] waiting for initial packets, poll returns 0 0 > main [1077] 0 pps > receiver_body [624] waiting for initial packets, poll returns 0 0 > receiver_body [624] waiting for initial packets, poll returns 0 0 > main [1077] 0 pps > main [1077] 112 pps > [...] > main [1077] 248 pps > main [1077] 0 pps > Received 4780 packets, in 41.35 seconds. > Speed: 115.61pps. > > If I, instead of click, use a simple linux bridge (using brctl), I get a > much higher throughput on the receiver: > > pkt-gen -i eth7 > main [877] map size is 203064 Kb > main [909] mmapping 203064 Kbytes > Receiving from eth7: 6 queues, 1 threads, 1 cpus and affinity -1. > main [956] Wait 2 secs for phy reset > main [958] Ready... > main [1077] 540069 pps > [...] > main [1077] 526933 pps > main [1077] 0 pps > Received 5318241 packets, in 10.00 seconds. > Speed: 531.60Kpps. > > > Does anyone know, why click in my case performs so poorly compared to a > linux bridge? > > Best regards, > > Richard > > _______________________________________________ > click mailing list > click at amsterdam.lcs.mit.edu > https://amsterdam.lcs.mit.edu/mailman/listinfo/click From dmi216 at nyu.edu Wed Mar 20 13:20:10 2013 From: dmi216 at nyu.edu (David M Iserovich) Date: Wed, 20 Mar 2013 13:20:10 -0400 Subject: [Click] Funny error compiling Click 2.0.1 with kernel module on 3.5.0-22-generic In-Reply-To: References: Message-ID: Could you specify those tweaks? Thanks for the pointer to the /lib/modules directory. Unfortunately my config.log is on another machine that I'm away from this week. I'll attach it when I can. Thank you for your help! On Fri, Mar 15, 2013 at 8:24 PM, Eddie Kohler wrote: > For what it's worth, the current Click head compiles and installs > cleanly on Ubuntu 12.10 x86-64. (3.5.) I had to tweak a couple things > to get this. > > I don't recommend using --with-linux=/usr/src/.... Click will default > to getting the sources from /lib/modules, which are more complete than > the --with-linux line. > > Eddie > > > On Fri, Mar 15, 2013 at 6:16 PM, Eddie Kohler wrote: > > Hi David, > > > > We might be able to help. Please attach your config.log. > > > > Eddie > > > > > > On Thu, Mar 14, 2013 at 6:37 PM, David M Iserovich > wrote: > >> Hi, I'm new to Click and I'm having a lot of fun with it in user-land, > but > >> I get a funny error compiling with the linux module on > >> > >> Linux 3.5.0-22-generic #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 x86_64 > >> x86_64 x86_64 GNU/Linux > >> > >> Which is Ubuntu 12.10. My compilation command, after copying out the > >> System.map file to the current directory, and putting the current Debian > >> config from /boot/config-`uname -r`. My exact compilation command is > >> > >> ./configure --with-linux=/usr/src/linux-headers-`uname -r` > >> --with-linux-map=./System.map-`uname -r` --enable-linuxmodule > >> > >> The error is, from the configure output > >> > >> configure: making C++-safe versions of Linux kernel headers (may take a > >> while) > >> Usage: fixincludes.pl -o OUTPUTDIR CFLAGS > >> configure: error: > >> ============================================== > >> > >> fixincludes.pl execution failed. > >> > >> ============================================== > >> > >> > >> which seems like fixinclude.pl is getting badly formatted args for some > >> reason, since it outputs that "Usage:" message. > >> > >> Can anyone help? > >> _______________________________________________ > >> click mailing list > >> click at amsterdam.lcs.mit.edu > >> https://amsterdam.lcs.mit.edu/mailman/listinfo/click > From dmi216 at nyu.edu Wed Mar 20 13:47:14 2013 From: dmi216 at nyu.edu (David M Iserovich) Date: Wed, 20 Mar 2013 13:47:14 -0400 Subject: [Click] Funny error compiling Click 2.0.1 with kernel module on 3.5.0-22-generic In-Reply-To: References: Message-ID: Oh, actually, I've managed to reproduce the problem on my local machine. Here is exactly what I did, on debian sid this time, with Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.39-2 x86_64 GNU/Linux I symlinked /lib/modules/`uname -r`/.config to /boot/config-`uname -r`, because ./configure complained that there was no .config in the folder I then ran ./configure --prefix=$HOME --enable-analysis --enable-test --enable-linuxmodule -with-linux=/lib/modules/3.2.0-4-amd64/source --with-linux-map=/boot/System.map-`uname -r` configuration ended with the same error configure: making C++-safe versions of Linux kernel headers (may take a while) Usage: fixincludes.pl -o OUTPUTDIR CFLAGS configure: error: ============================================== fixincludes.pl execution failed. ============================================== The full config.log is attached On Wed, Mar 20, 2013 at 1:20 PM, David M Iserovich wrote: > Could you specify those tweaks? > Thanks for the pointer to the /lib/modules directory. > > Unfortunately my config.log is on another machine that I'm away from this > week. I'll attach it when I can. > > Thank you for your help! > > > On Fri, Mar 15, 2013 at 8:24 PM, Eddie Kohler wrote: > >> For what it's worth, the current Click head compiles and installs >> cleanly on Ubuntu 12.10 x86-64. (3.5.) I had to tweak a couple things >> to get this. >> >> I don't recommend using --with-linux=/usr/src/.... Click will default >> to getting the sources from /lib/modules, which are more complete than >> the --with-linux line. >> >> Eddie >> >> >> On Fri, Mar 15, 2013 at 6:16 PM, Eddie Kohler wrote: >> > Hi David, >> > >> > We might be able to help. Please attach your config.log. >> > >> > Eddie >> > >> > >> > On Thu, Mar 14, 2013 at 6:37 PM, David M Iserovich >> wrote: >> >> Hi, I'm new to Click and I'm having a lot of fun with it in user-land, >> but >> >> I get a funny error compiling with the linux module on >> >> >> >> Linux 3.5.0-22-generic #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 >> x86_64 >> >> x86_64 x86_64 GNU/Linux >> >> >> >> Which is Ubuntu 12.10. My compilation command, after copying out the >> >> System.map file to the current directory, and putting the current >> Debian >> >> config from /boot/config-`uname -r`. My exact compilation command is >> >> >> >> ./configure --with-linux=/usr/src/linux-headers-`uname -r` >> >> --with-linux-map=./System.map-`uname -r` --enable-linuxmodule >> >> >> >> The error is, from the configure output >> >> >> >> configure: making C++-safe versions of Linux kernel headers (may take a >> >> while) >> >> Usage: fixincludes.pl -o OUTPUTDIR CFLAGS >> >> configure: error: >> >> ============================================== >> >> >> >> fixincludes.pl execution failed. >> >> >> >> ============================================== >> >> >> >> >> >> which seems like fixinclude.pl is getting badly formatted args for >> some >> >> reason, since it outputs that "Usage:" message. >> >> >> >> Can anyone help? >> >> _______________________________________________ >> >> click mailing list >> >> click at amsterdam.lcs.mit.edu >> >> https://amsterdam.lcs.mit.edu/mailman/listinfo/click >> > > -------------- next part -------------- A non-text attachment was scrubbed... Name: config.log Type: application/octet-stream Size: 128552 bytes Desc: not available Url : http://amsterdam.lcs.mit.edu/pipermail/click/attachments/20130320/db4bce81/attachment-0001.obj From mail at richard-neumann.de Thu Mar 21 07:31:38 2013 From: mail at richard-neumann.de (Richard Neumann) Date: Thu, 21 Mar 2013 12:31:38 +0100 Subject: [Click] Poor forwarding performance with click in a paravirtualized Xen environment In-Reply-To: <20130319182900.GA48640@onelab2.iet.unipi.it> References: <1363711796.5986.13.camel@localhost.localdomain> <20130319182900.GA48640@onelab2.iet.unipi.it> Message-ID: <1363865498.1681.14.camel@localhost.localdomain> Hello again, @Giorgio Calarco: Hi Richard, what happens if you decrease the input rate at lower values? For instance at 40000pps? (maybe you have to change your packet generator to perform this experiment ). And, where is your brctl-based bridge created ? Within dom0? Let me know, I'll try to run a test with a slower packet generator and will let you know how it performs then. But the problem is that we actually do want to do high-speed packet processing here. The brige is created inside the domU environment, connecting the two 10G interfaces which have been passed through from the dom0. @Luigi Rizzo: > At first sight it seems a case of receive livelock, which does not occur > so badly with the linux bridge as it (probably) is helped by NAPI. > > It is not clear if your userspace click is also using netmap (which > may have its own bugs) and/or whether the two interfaces eth1 and > eth2 are using pci-passthrough or are emulated/paravirtualized. in the respective setup, click is is not using netmap and even was compiled without the --with-netmap stuff. The 10G Cards are using pci-passthrough, because I expected better performance from this than from emulation or paravirtualization (I did not actually compare it yet). > In any case the 530Kpps you get with linux bridge is probably close > to the peak performance you can get, unless (a) Xen gives you access > to the real hw (via virtual functions and/or pci-passthrough), and So, for I am actually am using PCI passthrough, the performance should be better even using a linux bridge. > (b) you are using netmap or some very fast os stack bypass to talk > to the network interfaces within the virtual machine. > > I managed to go quite fast within qemu+kvm, see below > > http://info.iet.unipi.it/~luigi/netmap/talk-google-2013.html > http://info.iet.unipi.it/~luigi/papers/20130206-qemu.pdf > > but that needed a little bit of tweaking here and there > (not that i doubt that the Xen folks are smart enough to do > something similar). Thank you both for your help so far. Best regards, Richard From giorgio.calarco at unibo.it Thu Mar 21 08:18:01 2013 From: giorgio.calarco at unibo.it (Giorgio Calarco) Date: Thu, 21 Mar 2013 13:18:01 +0100 Subject: [Click] Poor forwarding performance with click in a paravirtualized Xen environment In-Reply-To: <1363865498.1681.14.camel@localhost.localdomain> References: <1363711796.5986.13.camel@localhost.localdomain> <20130319182900.GA48640@onelab2.iet.unipi.it> <1363865498.1681.14.camel@localhost.localdomain> Message-ID: Hi Richard, I understand your motivations, and that's why I suggested to slow down your generator and see what happens since I had the same suspect introduced by Luigi Rizzo, (receive livelock). Probably if you increase the packet rate progressively, you should be able to clarify this. I would also try (during these tests) to assign a predefined cpu core to the vm and follow its load. This seems similar to my brief experience with Click virtualized within Linux Containers, I couldn't overcome 60.000pps. When I rised up the rate further, the forwarding performance went down very quickly. A curiosity: which hardware platform are you using exactly ? Ciao, Giorgio On Thu, Mar 21, 2013 at 12:31 PM, Richard Neumann wrote: > Hello again, > > @Giorgio Calarco: > Hi Richard, what happens if you decrease the input rate at lower > values? For instance at 40000pps? (maybe you have to change your > packet generator to perform this experiment ). And, where is > your brctl-based bridge created ? Within dom0? > Let me know, > > I'll try to run a test with a slower packet generator and will let you > know how it performs then. But the problem is that we actually do want > to do high-speed packet processing here. > The brige is created inside the domU environment, connecting the two 10G > interfaces which have been passed through from the dom0. > > @Luigi Rizzo: > > At first sight it seems a case of receive livelock, which does not occur > > so badly with the linux bridge as it (probably) is helped by NAPI. > > > > It is not clear if your userspace click is also using netmap (which > > may have its own bugs) and/or whether the two interfaces eth1 and > > eth2 are using pci-passthrough or are emulated/paravirtualized. > > in the respective setup, click is is not using netmap and even was > compiled without the --with-netmap stuff. > The 10G Cards are using pci-passthrough, because I expected better > performance from this than from emulation or paravirtualization (I did > not actually compare it yet). > > > In any case the 530Kpps you get with linux bridge is probably close > > to the peak performance you can get, unless (a) Xen gives you access > > to the real hw (via virtual functions and/or pci-passthrough), and > > So, for I am actually am using PCI passthrough, the performance should > be better even using a linux bridge. > > > (b) you are using netmap or some very fast os stack bypass to talk > > to the network interfaces within the virtual machine. > > > > I managed to go quite fast within qemu+kvm, see below > > > > http://info.iet.unipi.it/~luigi/netmap/talk-google-2013.html > > http://info.iet.unipi.it/~luigi/papers/20130206-qemu.pdf > > > > but that needed a little bit of tweaking here and there > > (not that i doubt that the Xen folks are smart enough to do > > something similar). > > Thank you both for your help so far. > > Best regards, > > Richard > > > _______________________________________________ > click mailing list > click at amsterdam.lcs.mit.edu > https://amsterdam.lcs.mit.edu/mailman/listinfo/click > From giorgio.calarco at unibo.it Thu Mar 21 08:34:54 2013 From: giorgio.calarco at unibo.it (Giorgio Calarco) Date: Thu, 21 Mar 2013 13:34:54 +0100 Subject: [Click] Poor forwarding performance with click in a paravirtualized Xen environment In-Reply-To: <1363865498.1681.14.camel@localhost.localdomain> References: <1363711796.5986.13.camel@localhost.localdomain> <20130319182900.GA48640@onelab2.iet.unipi.it> <1363865498.1681.14.camel@localhost.localdomain> Message-ID: And, sorry, I forgot to say one thing: have you already read this paper by N.Egi ? Evaluating Xen for Router Virtualization http://nrg.cs.ucl.ac.uk/vrouter/publications/vrouter_pmect07.pdf Perhaps you can find some inspiration. Cheers, Giorgio On Thu, Mar 21, 2013 at 12:31 PM, Richard Neumann wrote: > Hello again, > > @Giorgio Calarco: > Hi Richard, what happens if you decrease the input rate at lower > values? For instance at 40000pps? (maybe you have to change your > packet generator to perform this experiment ). And, where is > your brctl-based bridge created ? Within dom0? > Let me know, > > I'll try to run a test with a slower packet generator and will let you > know how it performs then. But the problem is that we actually do want > to do high-speed packet processing here. > The brige is created inside the domU environment, connecting the two 10G > interfaces which have been passed through from the dom0. > > @Luigi Rizzo: > > At first sight it seems a case of receive livelock, which does not occur > > so badly with the linux bridge as it (probably) is helped by NAPI. > > > > It is not clear if your userspace click is also using netmap (which > > may have its own bugs) and/or whether the two interfaces eth1 and > > eth2 are using pci-passthrough or are emulated/paravirtualized. > > in the respective setup, click is is not using netmap and even was > compiled without the --with-netmap stuff. > The 10G Cards are using pci-passthrough, because I expected better > performance from this than from emulation or paravirtualization (I did > not actually compare it yet). > > > In any case the 530Kpps you get with linux bridge is probably close > > to the peak performance you can get, unless (a) Xen gives you access > > to the real hw (via virtual functions and/or pci-passthrough), and > > So, for I am actually am using PCI passthrough, the performance should > be better even using a linux bridge. > > > (b) you are using netmap or some very fast os stack bypass to talk > > to the network interfaces within the virtual machine. > > > > I managed to go quite fast within qemu+kvm, see below > > > > http://info.iet.unipi.it/~luigi/netmap/talk-google-2013.html > > http://info.iet.unipi.it/~luigi/papers/20130206-qemu.pdf > > > > but that needed a little bit of tweaking here and there > > (not that i doubt that the Xen folks are smart enough to do > > something similar). > > Thank you both for your help so far. > > Best regards, > > Richard > > > _______________________________________________ > click mailing list > click at amsterdam.lcs.mit.edu > https://amsterdam.lcs.mit.edu/mailman/listinfo/click > From mail at richard-neumann.de Mon Mar 25 11:54:38 2013 From: mail at richard-neumann.de (Richard Neumann) Date: Mon, 25 Mar 2013 16:54:38 +0100 Subject: [Click] Poor forwarding performance with click in a paravirtualized Xen environment In-Reply-To: <3D36BC4C-2F9A-40B4-A76B-4F07C6CFBBBC@gmail.com> References: <1363711796.5986.13.camel@localhost.localdomain> <20130319182900.GA48640@onelab2.iet.unipi.it> <1363865498.1681.14.camel@localhost.localdomain> <3D36BC4C-2F9A-40B4-A76B-4F07C6CFBBBC@gmail.com> Message-ID: <1364226878.21513.8.camel@localhost.localdomain> Hello again everybody, I ran some further test and also tried to generate packages at lower rates using ping with 10 kpps max. and receiving them using tcpdump. Up until this threshold, click seems to work fine and there seems to be no livelock. As operating system I am using openSUSE 12.1 with the kernel 3.4.33-2.24-xen. Our hardware are Supermicro X8DTG-D Servers with Intel Xeon X5650 CPUs @ 2.67GHz and 3 x 2GB DDR3-1333 RAM. Best regards, Richard Am Donnerstag, den 21.03.2013, 18:39 +0000 schrieb Adam Greenhalgh: > We wrote further work where we had better performance, it's linked from the vrouter page at UCL . > > Adam > > Sent from my iPhone > > On 21 Mar 2013, at 12:34, Giorgio Calarco wrote: > > > And, sorry, I forgot to say one thing: have you already read this paper by > > N.Egi ? > > > > Evaluating Xen for Router Virtualization > > http://nrg.cs.ucl.ac.uk/vrouter/publications/vrouter_pmect07.pdf > > > > Perhaps you can find some inspiration. > > > > Cheers, Giorgio > > > > > > On Thu, Mar 21, 2013 at 12:31 PM, Richard Neumann > > wrote: > > > >> Hello again, > >> > >> @Giorgio Calarco: > >> Hi Richard, what happens if you decrease the input rate at lower > >> values? For instance at 40000pps? (maybe you have to change your > >> packet generator to perform this experiment ). And, where is > >> your brctl-based bridge created ? Within dom0? > >> Let me know, > >> > >> I'll try to run a test with a slower packet generator and will let you > >> know how it performs then. But the problem is that we actually do want > >> to do high-speed packet processing here. > >> The brige is created inside the domU environment, connecting the two 10G > >> interfaces which have been passed through from the dom0. > >> > >> @Luigi Rizzo: > >>> At first sight it seems a case of receive livelock, which does not occur > >>> so badly with the linux bridge as it (probably) is helped by NAPI. > >>> > >>> It is not clear if your userspace click is also using netmap (which > >>> may have its own bugs) and/or whether the two interfaces eth1 and > >>> eth2 are using pci-passthrough or are emulated/paravirtualized. > >> > >> in the respective setup, click is is not using netmap and even was > >> compiled without the --with-netmap stuff. > >> The 10G Cards are using pci-passthrough, because I expected better > >> performance from this than from emulation or paravirtualization (I did > >> not actually compare it yet). > >> > >>> In any case the 530Kpps you get with linux bridge is probably close > >>> to the peak performance you can get, unless (a) Xen gives you access > >>> to the real hw (via virtual functions and/or pci-passthrough), and > >> > >> So, for I am actually am using PCI passthrough, the performance should > >> be better even using a linux bridge. > >> > >>> (b) you are using netmap or some very fast os stack bypass to talk > >>> to the network interfaces within the virtual machine. > >>> > >>> I managed to go quite fast within qemu+kvm, see below > >>> > >>> http://info.iet.unipi.it/~luigi/netmap/talk-google-2013.html > >>> http://info.iet.unipi.it/~luigi/papers/20130206-qemu.pdf > >>> > >>> but that needed a little bit of tweaking here and there > >>> (not that i doubt that the Xen folks are smart enough to do > >>> something similar). > >> > >> Thank you both for your help so far. > >> > >> Best regards, > >> > >> Richard > >> > >> > >> _______________________________________________ > >> click mailing list > >> click at amsterdam.lcs.mit.edu > >> https://amsterdam.lcs.mit.edu/mailman/listinfo/click > > _______________________________________________ > > click mailing list > > click at amsterdam.lcs.mit.edu > > https://amsterdam.lcs.mit.edu/mailman/listinfo/click From giorgio.calarco at unibo.it Mon Mar 25 12:19:05 2013 From: giorgio.calarco at unibo.it (Giorgio Calarco) Date: Mon, 25 Mar 2013 17:19:05 +0100 Subject: [Click] Poor forwarding performance with click in a paravirtualized Xen environment In-Reply-To: <1364226878.21513.8.camel@localhost.localdomain> References: <1363711796.5986.13.camel@localhost.localdomain> <20130319182900.GA48640@onelab2.iet.unipi.it> <1363865498.1681.14.camel@localhost.localdomain> <3D36BC4C-2F9A-40B4-A76B-4F07C6CFBBBC@gmail.com> <1364226878.21513.8.camel@localhost.localdomain> Message-ID: mmm, using ping is probably not the best idea, I think, I suppose your receiver is responding and sending ICMP reply packets to your generator, and this could make things a bit more... let's say, tricky. However, you're not that far from what I was obtaining with containers (I was using a X5300 platform with 2.4 GHz CPUs). Uh, to be paranoid, did you check if your receiver is losing packets on its NIC buffer? It shouldn't at these rates, however, a check is better, just to be sure that tcpdump is collecting everything and showing the true evaluations. ciao, giorgio On Mon, Mar 25, 2013 at 4:54 PM, Richard Neumann wrote: > Hello again everybody, > > I ran some further test and also tried to generate packages at lower > rates using ping with 10 kpps max. and receiving them using tcpdump. > > Up until this threshold, click seems to work fine and there seems to be > no livelock. > > As operating system I am using openSUSE 12.1 with the kernel > 3.4.33-2.24-xen. > > Our hardware are Supermicro X8DTG-D Servers with Intel Xeon X5650 CPUs @ > 2.67GHz and 3 x 2GB DDR3-1333 RAM. > > Best regards, > > Richard > > > Am Donnerstag, den 21.03.2013, 18:39 +0000 schrieb Adam Greenhalgh: > > We wrote further work where we had better performance, it's linked from > the vrouter page at UCL . > > > > Adam > > > > Sent from my iPhone > > > > On 21 Mar 2013, at 12:34, Giorgio Calarco > wrote: > > > > > And, sorry, I forgot to say one thing: have you already read this > paper by > > > N.Egi ? > > > > > > Evaluating Xen for Router Virtualization > > > http://nrg.cs.ucl.ac.uk/vrouter/publications/vrouter_pmect07.pdf > > > > > > Perhaps you can find some inspiration. > > > > > > Cheers, Giorgio > > > > > > > > > On Thu, Mar 21, 2013 at 12:31 PM, Richard Neumann > > > wrote: > > > > > >> Hello again, > > >> > > >> @Giorgio Calarco: > > >> Hi Richard, what happens if you decrease the input rate at > lower > > >> values? For instance at 40000pps? (maybe you have to change > your > > >> packet generator to perform this experiment ). And, where is > > >> your brctl-based bridge created ? Within dom0? > > >> Let me know, > > >> > > >> I'll try to run a test with a slower packet generator and will let you > > >> know how it performs then. But the problem is that we actually do want > > >> to do high-speed packet processing here. > > >> The brige is created inside the domU environment, connecting the two > 10G > > >> interfaces which have been passed through from the dom0. > > >> > > >> @Luigi Rizzo: > > >>> At first sight it seems a case of receive livelock, which does not > occur > > >>> so badly with the linux bridge as it (probably) is helped by NAPI. > > >>> > > >>> It is not clear if your userspace click is also using netmap (which > > >>> may have its own bugs) and/or whether the two interfaces eth1 and > > >>> eth2 are using pci-passthrough or are emulated/paravirtualized. > > >> > > >> in the respective setup, click is is not using netmap and even was > > >> compiled without the --with-netmap stuff. > > >> The 10G Cards are using pci-passthrough, because I expected better > > >> performance from this than from emulation or paravirtualization (I > did > > >> not actually compare it yet). > > >> > > >>> In any case the 530Kpps you get with linux bridge is probably close > > >>> to the peak performance you can get, unless (a) Xen gives you access > > >>> to the real hw (via virtual functions and/or pci-passthrough), and > > >> > > >> So, for I am actually am using PCI passthrough, the performance should > > >> be better even using a linux bridge. > > >> > > >>> (b) you are using netmap or some very fast os stack bypass to talk > > >>> to the network interfaces within the virtual machine. > > >>> > > >>> I managed to go quite fast within qemu+kvm, see below > > >>> > > >>> http://info.iet.unipi.it/~luigi/netmap/talk-google-2013.html > > >>> http://info.iet.unipi.it/~luigi/papers/20130206-qemu.pdf > > >>> > > >>> but that needed a little bit of tweaking here and there > > >>> (not that i doubt that the Xen folks are smart enough to do > > >>> something similar). > > >> > > >> Thank you both for your help so far. > > >> > > >> Best regards, > > >> > > >> Richard > > >> > > >> > > >> _______________________________________________ > > >> click mailing list > > >> click at amsterdam.lcs.mit.edu > > >> https://amsterdam.lcs.mit.edu/mailman/listinfo/click > > > _______________________________________________ > > > click mailing list > > > click at amsterdam.lcs.mit.edu > > > https://amsterdam.lcs.mit.edu/mailman/listinfo/click > > > _______________________________________________ > click mailing list > click at amsterdam.lcs.mit.edu > https://amsterdam.lcs.mit.edu/mailman/listinfo/click > From dmi216 at nyu.edu Fri Mar 29 12:50:41 2013 From: dmi216 at nyu.edu (David M Iserovich) Date: Fri, 29 Mar 2013 12:50:41 -0400 Subject: [Click] Funny error compiling Click 2.0.1 with kernel module on 3.5.0-22-generic In-Reply-To: References: Message-ID: Okay, so I pulled the latest code from git rather than using the Click 2.0.1 release downloaded from the site, and the error seems to have been resolved in this later release. Sorry for the confusion, I was under the impression that the git version was unstable due to careless skimming :) On Wed, Mar 20, 2013 at 1:47 PM, David M Iserovich wrote: > Oh, actually, I've managed to reproduce the problem on my local machine. > > Here is exactly what I did, on debian sid this time, with > > Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.39-2 x86_64 GNU/Linux > > I symlinked /lib/modules/`uname -r`/.config to /boot/config-`uname -r`, > because ./configure complained that there was no .config in the folder > > I then ran > > ./configure --prefix=$HOME --enable-analysis --enable-test > --enable-linuxmodule -with-linux=/lib/modules/3.2.0-4-amd64/source > --with-linux-map=/boot/System.map-`uname -r` > > configuration ended with the same error > > configure: making C++-safe versions of Linux kernel headers (may take a > while) > Usage: fixincludes.pl -o OUTPUTDIR CFLAGS > configure: error: > ============================================== > > fixincludes.pl execution failed. > > ============================================== > > The full config.log is attached > > > > > > > > > On Wed, Mar 20, 2013 at 1:20 PM, David M Iserovich wrote: > >> Could you specify those tweaks? >> Thanks for the pointer to the /lib/modules directory. >> >> Unfortunately my config.log is on another machine that I'm away from this >> week. I'll attach it when I can. >> >> Thank you for your help! >> >> >> On Fri, Mar 15, 2013 at 8:24 PM, Eddie Kohler wrote: >> >>> For what it's worth, the current Click head compiles and installs >>> cleanly on Ubuntu 12.10 x86-64. (3.5.) I had to tweak a couple things >>> to get this. >>> >>> I don't recommend using --with-linux=/usr/src/.... Click will default >>> to getting the sources from /lib/modules, which are more complete than >>> the --with-linux line. >>> >>> Eddie >>> >>> >>> On Fri, Mar 15, 2013 at 6:16 PM, Eddie Kohler wrote: >>> > Hi David, >>> > >>> > We might be able to help. Please attach your config.log. >>> > >>> > Eddie >>> > >>> > >>> > On Thu, Mar 14, 2013 at 6:37 PM, David M Iserovich >>> wrote: >>> >> Hi, I'm new to Click and I'm having a lot of fun with it in >>> user-land, but >>> >> I get a funny error compiling with the linux module on >>> >> >>> >> Linux 3.5.0-22-generic #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 >>> x86_64 >>> >> x86_64 x86_64 GNU/Linux >>> >> >>> >> Which is Ubuntu 12.10. My compilation command, after copying out the >>> >> System.map file to the current directory, and putting the current >>> Debian >>> >> config from /boot/config-`uname -r`. My exact compilation command is >>> >> >>> >> ./configure --with-linux=/usr/src/linux-headers-`uname -r` >>> >> --with-linux-map=./System.map-`uname -r` --enable-linuxmodule >>> >> >>> >> The error is, from the configure output >>> >> >>> >> configure: making C++-safe versions of Linux kernel headers (may take >>> a >>> >> while) >>> >> Usage: fixincludes.pl -o OUTPUTDIR CFLAGS >>> >> configure: error: >>> >> ============================================== >>> >> >>> >> fixincludes.pl execution failed. >>> >> >>> >> ============================================== >>> >> >>> >> >>> >> which seems like fixinclude.pl is getting badly formatted args for >>> some >>> >> reason, since it outputs that "Usage:" message. >>> >> >>> >> Can anyone help? >>> >> _______________________________________________ >>> >> click mailing list >>> >> click at amsterdam.lcs.mit.edu >>> >> https://amsterdam.lcs.mit.edu/mailman/listinfo/click >>> >> >> > From lawrenceqli at gmail.com Sat Mar 30 12:10:03 2013 From: lawrenceqli at gmail.com (Lawrence Lee) Date: Sat, 30 Mar 2013 17:10:03 +0100 Subject: [Click] Question about developing an element of new network layer headers to replace IP headers Message-ID: Hi all, Our group developed a new network layer protocol in which new network layer headers replace IP headers in packets. May I know if we can implement the protocol in Click. I took a look at the source code of Click, e.g., the todevice and fromdevice modules. It seems that we cannot simply develop an element to replace IP headers in Click because IP headers are a must. I am not sure if my understanding is correct. Your comments and suggestions will be greatly appreciated. Best Regards, Lawrence