[chord] Problem with dhashcli

[Katarzyna Stefanowicz]

Hello,

I'm writing some software on top of dhash and chord. It's integrated
with lsd just like dhash - statically linked library, vector of objects
(one per vnode), object created just after dhash object, etc.

I've got to insert some blocks into dhash. Because my code runs inside
of lsd I'm not using dhashclient class (and unix pipe) but dhashcli
class. Everything works well when I'm inserting block with
ctype==DHASH_CONTENTHASH. But when I changed ctype to DHASH_NOAUTH it
stopped working.

Here is part of code I'm using to insert data (some debugging code
removed):
void incognito_impl::publish(const incognito_store_arg& req)
{
	ref<dhash_block> block =
		New refcounted<dhash_block> (req.data.base(),
			req.data.size(), req.key_type);
	block->ID = req.key_value;

	_dhcli->insert (block, wrap (mkref (this),
		&incognito_impl::publish_insert_cb));
}

And incognito_store_arg defined in .x file:
typedef opaque incognito_value<>;

struct incognito_store_arg
{
         dhash_ctype     key_type;
         bigint          key_value;
         incognito_value data;
};

Here is part of log.lsd on node, where this method is called:
> incognito_impl::publish(c156a0de693cc8197d44ef45fe36f9debe187b4e);
> Publishing....
>  key_type: >>2<< key_value: >>c156a0de693cc8197d44ef45fe36f9debe187b4e<<
> 1201651333.291114 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 3/5) -> ff76e9dc9910579df7a73928048007aa305c7263 in 1ms: DHASH_STALE
>  1201651333:330134 REXMIT fc3c9c4e 344449:1 rexmits 0, timeout 38 ms, destined for 10.14.5.2 out is 138387732
>  1201651333:330272 REXMIT d5a4beb3 344449:1 rexmits 0, timeout 38 ms, destined for 10.14.5.3 out is 138394004
>  1201651333:334176 REXMIT 58c93fc9 344449:1 rexmits 0, timeout 41 ms, destined for 10.14.5.2 out is 138388980
>  1201651333:338137 REXMIT 90e7b467 344449:1 rexmits 0, timeout 45 ms, destined for 10.14.5.1 out is 138392284
> 1201651333.353310 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 5/5) -> 3c3c3c91f8d7c5601249d13c58324f2f8baf1eee in 64ms: DHASH_STALE
>  1201651333:410243 REXMIT fc3c9c4e 344449:1 rexmits 1, timeout 77 ms, destined for 10.14.5.2 out is 138387732
>  1201651333:418156 REXMIT 58c93fc9 344449:1 rexmits 1, timeout 82 ms, destined for 10.14.5.2 out is 138388980
>  1201651333:430147 REXMIT 90e7b467 344449:1 rexmits 1, timeout 91 ms, destined for 10.14.5.1 out is 138392284
> 1201651333.449006 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 2/5) -> fb5fa23d77f54f3679d948430bcda824abd48a60 in 159ms: DHASH_STALE
> 1201651333.531063 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 1/5) -> ea182e8573df8cf54320288a8f70e26bf4cb4464 in 241ms: DHASH_STALE
> 1201651333.568921 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 4/5) -> 24752b6a087ef3456faff165179020c4c9e9b298 in 279ms: DHASH_STALE
> 1201651333.568975 dhashcli: ad91f3d2b200dba01e82a43a4dad65a76a98570f: store (c156a0de693cc8197d44ef45fe36f9debe187b4e): only stored 0 of 5 encoded.
> 1201651333.568995 dhashcli: ad91f3d2b200dba01e82a43a4dad65a76a98570f: store (c156a0de693cc8197d44ef45fe36f9debe187b4e): failed; insufficient frags/blocks stored.


On node 24752b6a087ef3456faff165179020c4c9e9b298 (10.14.5.1) log.lsd
contain only one interesting line:

Failed to unmarshall noauth block


It's very strange for me, because for DHASH_CONTENTHASH it works:

> incognito_impl::publish(c156a0de693cc8197d44ef45fe36f9debe187b4e);
> Publishing....
>  key_type: >>2<< key_value: >>c156a0de693cc8197d44ef45fe36f9debe187b4e<<
> 1201651333.291114 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 3/5) -> ff76e9dc9910579df7a73928048007aa305c7263 in 1ms: DHASH_STALE
>  1201651333:330134 REXMIT fc3c9c4e 344449:1 rexmits 0, timeout 38 ms, destined for 10.14.5.2 out is 138387732
>  1201651333:330272 REXMIT d5a4beb3 344449:1 rexmits 0, timeout 38 ms, destined for 10.14.5.3 out is 138394004
>  1201651333:334176 REXMIT 58c93fc9 344449:1 rexmits 0, timeout 41 ms, destined for 10.14.5.2 out is 138388980
>  1201651333:338137 REXMIT 90e7b467 344449:1 rexmits 0, timeout 45 ms, destined for 10.14.5.1 out is 138392284
> 1201651333.353310 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 5/5) -> 3c3c3c91f8d7c5601249d13c58324f2f8baf1eee in 64ms: DHASH_STALE
>  1201651333:410243 REXMIT fc3c9c4e 344449:1 rexmits 1, timeout 77 ms, destined for 10.14.5.2 out is 138387732
>  1201651333:418156 REXMIT 58c93fc9 344449:1 rexmits 1, timeout 82 ms, destined for 10.14.5.2 out is 138388980
>  1201651333:430147 REXMIT 90e7b467 344449:1 rexmits 1, timeout 91 ms, destined for 10.14.5.1 out is 138392284
> 1201651333.449006 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 2/5) -> fb5fa23d77f54f3679d948430bcda824abd48a60 in 159ms: DHASH_STALE
> 1201651333.531063 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 1/5) -> ea182e8573df8cf54320288a8f70e26bf4cb4464 in 241ms: DHASH_STALE
> 1201651333.568921 dhashcli: store c156a0de693cc8197d44ef45fe36f9debe187b4e (frag 4/5) -> 24752b6a087ef3456faff165179020c4c9e9b298 in 279ms: DHASH_STALE
> 1201651333.568975 dhashcli: ad91f3d2b200dba01e82a43a4dad65a76a98570f: store (c156a0de693cc8197d44ef45fe36f9debe187b4e): only stored 0 of 5 encoded.
> 1201651333.568995 dhashcli: ad91f3d2b200dba01e82a43a4dad65a76a98570f: store (c156a0de693cc8197d44ef45fe36f9debe187b4e): failed; insufficient frags/blocks stored.

I'm using Debian GNU Linux "Etch" operating system, sfslite-0.8.16,
chord-0.1-20071115.
Do you have any ideas why it's not working? Maybe I'm using dhashcli in
a wrong way?

Best regards,
Katsiaryna Stsefanovich