sigcomm2000@cs.umass.edu: SIGCOMM 2000: result of review for paper 342

[Eddie Kohler]

too bad!
(this is a rejection, but we are "very competent" :)

------- Forwarded Message

Received: from mintaka.lcs.mit.edu (mintaka.lcs.mit.edu [18.26.0.36])
	by amsterdam.lcs.mit.edu (8.8.7/8.8.7) with ESMTP id WAA20895
	for <eddietwo at amsterdam.lcs.mit.edu>; Tue, 18 Apr 2000 22:33:15 -0400 (EDT)
Received: from gaia.cs.umass.edu (gaia.cs.umass.edu [128.119.245.12])
	by mintaka.lcs.mit.edu (8.9.3/8.9.3) with ESMTP id WAA17127
	for <eddietwo at lcs.mit.edu>; Tue, 18 Apr 2000 22:33:15 -0400 (EDT)
	(envelope-from mosaic at gaia.cs.umass.edu)
Received: (from mosaic at localhost)
	by gaia.cs.umass.edu (8.8.7/8.8.8) id WAA23142;
	Tue, 18 Apr 2000 22:33:14 -0400 (EDT)
Date: Tue, 18 Apr 2000 22:33:14 -0400 (EDT)
Message-Id: <200004190233.WAA23142 at gaia.cs.umass.edu>
From: sigcomm2000 at cs.umass.edu
To: eddietwo at lcs.mit.edu, sigcomm2000 at cs.umass.edu
Subject: SIGCOMM 2000: result of review for paper 342

Dear Eddie Kohler,

We are writing you regarding your paper,

ID     : 342
TITLE  : Optimizing a Modular Software Router

that was submitted to the ACM Sigcomm 2000 Conference, to be held in
Stockholm, Sweden August 28 - September 1, 2000. 
See http://www.acm.org/sigcomm/sigcomm2000/ for more details.

This year, 235 papers were submitted to the conference - the largest
number ever submitted to a Sigcomm conference. We were able to accept
only 26 papers into this year's conference, given Sigcomm's tradition as a
single-trak conference. We regret to inform you that it was not possible
to accept your paper into ACM Sigcomm 2000. Referee reports on your paper
are enclosed with this letter; we hope that these will be helpful to you
should you decide to submit your work to another conference or journal.

We thank you for your interest in the ACM Sigcomm 2000 conference and hope
that you will be able to attend the conference this coming August in
Stockholm.


Sincerely,

Christophe Diot and Jim Kurose
ACM Sigcomm 2000 Technical Program Co-chairs.


- ---------------------------------------

Overall score.........: 3.0
Comments to the author: 
This paper details the authors' experiences optimizing a software-based
router.  The authors present their system, describe a number of optimizations,
and carefully quantify the performance gains achieved.  The paper is well-
written, organized, and is of serious interest to the SigComm community.

Unfortunately, I'm very mixed about the paper since I learned absolutely
nothing.  Everything in the paper can be learned from the original Click 
paper in SOSP and from previous work, some of which is cited by the paper,
and some not. It is well known that interrupts are bad for router-based
operating systems. It is well-known that device accesses are expensive
since they cannot be cached.  It is well-known that there is a tradeoff 
between modularity and performance via specialization.

The paper especially does not give enough credit to the x-Kernel work
at Arizona, which has been examining this modularity vs. performance
issue for years. 

For example, the following two papers from Sean O'Malley's thesis work
directly propose and evaluate the idea of micro-protocols (similar to
Click elements):

@article{O'Malley92:Dynamic,
        author =        {Sean O'Malley and Larry L. Peterson},
        title =         {A Dynamic Network Architecture},
        journal =       {ACM Transactions on Computing Systems},
        year =          {1992}
}
@inproceedings{O'Malley91:Highly,
        author =        {Sean O'Malley and Larry L. Peterson},
        year =          {1991},
        booktitle =     {Proceedings of IFIP Workshop on Protocols for
                         High-Speed Networks},
        pages =         {141-156},
        title =         {A Highly Layered Architecture for High-Speed Networks}
}

Another example is Peter Druschel's SigComm 94 paper on high-speed
adaptors, batching interrupts, etc:

@INPROCEEDINGS{Druschel94:Experiences,
  AUTHOR = "Peter Druschel and Larry Peterson and Bruce Davie",
  TITLE = "Experiences with a High-Speed Network Adaptor:
        A Software Perspective",
  BOOKTITLE=sigcomm,
  ADDRESS="London, England",
  MONTH=aug,
  YEAR=1994
}

The main contribution of the paper is that it puts a lot of these issues
in one place, thus it has educational value for those who have not seen
this stuff before.  Unfortunately, readers will then mistake the techniques
as novel and then credit the authors inappropriately.  General concepts
on router-based OS'es are known informally by people who have seen IOS
code or Proteon code, but are perhaps not sufficiently documented.

Again, I'm mixed about it because the authors clearly know what they're doing,
express themselves very well, and do a thorough evaluation.


- ---------------------------------------


Overall score.........: 2.0
Comments to the author: 
This paper describes the optimization of Click, a modular softeware router,
along three dimensions: operating system, device and application. The single
biggest performance gain is achieved through the use of polling which doubles
the throughout. The performance gain of 5 folds  is significant. 

As stated by the authors, the biggest advantage of such a module software
router is flexibility. A router that performs only forwarding task can be
built much faster and cheaper.  

The focuses of the paper is on optimizing the basic packet forwarding path.
While this is important, it is not clear how the system will behave when new
functions like encapsulation, encryption, compression, accounting etc. are
performed. These functions can be fairly computational intensive and can
potentially access a much larger memory space, thereby lessening the impact
of the optimization described.  

While the topic is on optimizing a module software router, most of the
optimization techniques used are operating system and compiler related.
The paper is probably more appropriate for an operating system conference.
  

- ---------------------------------------


Overall score.........: 2.0
Comments to the author: 
This is a very competent paper by very competent authors and the results
are impressive, and so it requires some explanation as to why I gave it
such a low score.  As I understand the context, the authors (or some
others) have already written a paper on a language based software router
implementation called Click which appeared in SOSP which is a highly
visible conference.   The main ideas behin Click appear in that paper.

The authors then show that by simple and well-known performance tricks
they can improve performance.  For example, they use polling as
described by Mogul and Ramakrishnan (but known by router and bridge
vendors for aeons) instead of interrupts; they use a fast classifier
(see earlier work on BPF, BPF optimizations, PathFinder, and DPF),
language based specialization (specialization is an old idea and is
often used in Peterson's work and Scout has similar ideas), and
automatic alignment.  There is not a single new idea or any insight
into networking.  There is, however, a very impressive forwarding
figure for a 700 Mhz Pentium based router.

So while I do like the message (software based routers do fine at the low
end with appropriate optimization, and they are very flexible), I feel this
is stuff that should have been there in the SOSP paper or its final journal
version, and has nothing to do with SIGCOMM.
    

- ---------------------------------------



------- End of Forwarded Message