"Shadow PC" Exercise for NSDI'04

This year, NSDI is running a "Shadow PC" exercise in which students can review NSDI submissions when consent is given by the authors and the reviewing occurs in a supervised setting. The aim of this exercise is to educate students in the art of reviewing by using a more realistic setting than has been previously available -- real submissions and real reviewing software. We believe that doing so will provide a long term benefit to the community as a whole.

The remainder of this page describes the process to be followed by consenting authors and Shadow PC chairs, along with its safeguards. The exercise an experiment, and we will solicit feedback after it is concluded to assess whether it should be run in future years. In the meantime, we welcome your feedback and thank the NSDI PC chairs for their support. We hope that you, as an author, will consent to the use of your paper in this exercise, and you, as faculty or student, will make best use of this opportunity.

Steps for Authors

  1. The submission page includes a checkbox that allows you to "opt in" to the Shadow PC exercise at the time you submit and for each paper that you submit. Only papers for which the authors give explicit consent will be made available to Shadow PCs.
  2. Shadow PC reviewing will occur in parallel and independent of the main review process on a blinded set of papers. The shadow and main reviews will not be mixed, and the shadow process will play no role in determining the outcome for your paper. The shadow review process will be held to the same standards of confidentiality and accountability as the main review process. In particular, the names of the Shadow PC chairs will be made public but the shadow review process itself will be double-blind, with neither Shadow PC members nor Shadow PC chairs knowing the identities of authors.
  3. Shadow PC reviews will be returned to consenting authors as a benefit of participating by December 18, one week after the NSDI notification of acceptance. The exercise is then concluded. We hope that this additional feedback will prove useful to authors.

Steps for Shadow PC Chairs

  1. We anticipate that Shadow PC chairs will make use of the exercise via a class or weekly reading group. In a typical setting, students will be assigned one or two submissions to review before class (using a similar Web setup and review form as the main conference). The class will then meet and function as its own PC by discussing the set of reviews and merits of each paper, including whether the submissions should be accepted. Shadow PC Chairs must accept responsibility for the following:
  2. Any faculty member at a higher educational institution may request that a Shadow PC be set up on their behalf by sending mail to Tom Anderson (tom@cs.washington.edu) before September 22 (one week after the NSDI submission deadline).  Students, please find a faculty sponsor at your institution rather than mail us directly. The mail should describe the intended use of the submissions and indicate that the Shadow PC chair accepts all of the conditions above.
  3. We will set up Shadow PC review sites based on the merit of the request and subject to our resource constraints. The reviewing sites will allow the Shadow PC chair to create their own PC and assign review work to the PC. We expect to undertake to:

Tom Anderson (tom@cs.washington.edu) and David Wetherall (djw@cs.washington.edu
University of Washington