[chord] running dhash/chord on very few nodes (<10)

[Alex Tumanov]

Greetings,

Is it possible to use Chord/Dhash on just two nodes? This may sound absurd, but this is more of a boundary case coverage question. It would be great if the DHT implementation continued working even if the total number of nodes drops under the recommended 16 due to node leaves. Initially I thought it can be accomplished by running multiple virtual nodes on physical machines. I tested the following:

1. started DHT on host1 with 16 vnodes
2. insert file1 on host1
3. join the DHT from host2 with 16 vnodes
4. retrieve file1 on host2 successfully 
5. kill host1
6. retrieve file from host2 fails

How can I increase the replication factor in such a way that the above scenario works? Also, can I work with fewer than 16 virtual nodes? Where is this configurable? 

Thanks a lot for your time. I'm attaching the log of actual commands and their output below.

1. [host1]$ ./start-dhash --bindir /home/atumanov/build/chord/ --root /tmp/atumanov/dht/ -v 16 -j host1:10000 -p 10000
RUNNING: /home/atumanov/build/chord//lsd/adbd adbd -d ./db -S ./adbd-sock -q 0
RUNNING: /home/atumanov/build/chord//maint/maintd maintd -d ./maintdata -m carbonite -C ./maint-sock
RUNNING: /home/atumanov/build/chord//lsd/lsd lsd -d ./adbd-sock -S ./dhash-sock -R ./maint-sock -C ./lsd-sock -v 16 -j host1:10000 -p 10000

2. [host1]$ ./filestore /tmp/atumanov/dht/dhash-sock -s /tmp/atumanov/stuff 1
len 11 inflight 0
len 20
p
len 280
6b53cd00143421fdabfba33f4b9757ab599a2a2e
[host1]$ ./filestore /tmp/atumanov/dht/dhash-sock -f 6b53cd00143421fdabfba33f4b9757ab599a2a2e 1
retrieve 6b53cd00143421fdabfba33f4b9757ab599a2a2e
retrieve 5dc77e99b036481d37f469eb355eb70b711fc97a
retrieve 775dbbdac2d7988c88323fe33c6a2ca53d35393c
<retrieve successful -> insert successful>


3. [host2]$ ./start-dhash --bindir /home/atumanov/build/chord/ --root /tmp/atumanov/dht/ -v 16 -j host1:10000
RUNNING: /home/atumanov/build/chord//lsd/adbd adbd -d ./db -S ./adbd-sock -q 0
RUNNING: /home/atumanov/build/chord//maint/maintd maintd -d ./maintdata -m carbonite -C ./maint-sock
RUNNING: /home/atumanov/build/chord//lsd/lsd lsd -d ./adbd-sock -S ./dhash-sock -R ./maint-sock -C ./lsd-sock -v 16 -j host1:10000

4. [host2]$ ./filestore /tmp/atumanov/dht/dhash-sock -f 6b53cd00143421fdabfba33f4b9757ab599a2a2e 1
retrieve 6b53cd00143421fdabfba33f4b9757ab599a2a2e
retrieve 5dc77e99b036481d37f469eb355eb70b711fc97a
retrieve 775dbbdac2d7988c88323fe33c6a2ca53d35393c

5. [host1] Ctrl^C the foreground process
REAPED adbd (19252): exited normally
REAPED maintd (19253): exited normally
REAPED lsd (19266): exited normally


6. [host2]$ ./filestore /tmp/atumanov/dht/dhash-sock -f 6b53cd00143421fdabfba33f4b9757ab599a2a2e 1
retrieve 6b53cd00143421fdabfba33f4b9757ab599a2a2e
dhashclient::retrieve failed: 6b53cd00143421fdabfba33f4b9757ab599a2a2e: DHASH_NOENT
fatal: lost inode

Misc:
snapshot used 20071115, sfslite: 0.8.16, Centos5

P.S. May I suggest a minor change to start-dhash script? Dying on non-existent directories in the PATH, IMHO, is unnecessary:

--- start-dhash 2007-11-03 02:07:06.000000000 -0400
+++ /home/atumanov/build/chord/lsd/start-dhash  2008-01-24 03:02:53.919694738 -0500
@@ -70,9 +70,9 @@
        use File::Spec;
        $bindir = File::Spec->rel2abs ($bindir);
     }
-    if (! -d $bindir) {
-       die "$bindir: $!\n";
-    }
+#    if (! -d $bindir) {
+#      die "$bindir: $!\n";
+#    }
     if (-x "$bindir/bin/lsd") {
        # e.g. bindir=/usr
        $bindir = "$bindir/bin";
@@ -88,6 +88,7 @@
        $ADBD   = "$bindir/adbd";
        $MAINTD = "$bindir/maintd";
     }
+
     $found = 1;
     for ($LSD, $ADBD, $MAINTD) {
        $found = $found && (-f $_ && -x _);





      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ