[Click] Packet loss with PollDevice
Yu Chen
yuchen at cs.duke.edu
Fri Jan 20 17:53:34 EST 2012
Hi Roman,
I also tried that combination. Besides, I tried to pin each thread to a
fixed CPU using taskset utility.
At the point, I am kind of wondering whether it is possible that the
queue at the switches might compress the interval between the packets and
make the arrival at the router more bursty. Any comments on that?
Best Regards and Thanks,
Yu
On Thu, Jan 19, 2012 at 02:49:16PM -0500, Yu Chen wrote:
> Hi Roman,
>
> I meant the txqueuelen at the sender side(the users in the experiment).
> My guess is the burstiness of packet sending might affect the performance
> of the receiver(the router in the experiment).
>
> Best Regards,
> Yu
> On Thu, Jan 19, 2012 at 11:46:47AM -0800, rchertov wrote:
> > On 01.19.2012 11:44, Yu Chen wrote:
> > >Hi Roman,
> > >
> > >I tried both. The ring buffer is always 4096 in all the experiments
> > >I mentioned before. However, I tried higher values with
> > >txqueuelen=1000
> >
> > I don't think that txqueuelen is an issue here. You are dropping
> > packets on input (PollDevice doesn't empty the ring buffer fast
> > enough).
> >
> > Roman
> >
> > >only. I doubt higher values with txqueuelen will help. I will try
> > >that
> > >combination anyway.
> > >
> > >Best Regards and Thanks,
> > >Yu
> > >
> > >On Thu, Jan 19, 2012 at 11:09:43AM -0800, rchertov wrote:
> > >>On 01.19.2012 11:06, Yu Chen wrote:
> > >>>1.8.0 on Linux 2.6.24.7. Yes, I played with it(from 1 to 8).
> > >>However,
> > >>
> > >>Have you tried higher values like 16 and 32? Also, since you are
> > >>using Intel Pro NICs, you can use ethtool to increase the RX ring
> > >>buffer. ethtool -G ethX rx <SomeNumber>
> > >>
> > >>Roman
> > >>
> > >>
> > >>>it does not seem to help much. Yes, polldevice::count is
> > >>smaller than
> > >>>the packets I sent to that interface. Also, the packet size 64B. I
> > >>>forgot
> > >>>to mention it in the previous emails.
> > >>>
> > >>>Best Regards and Thanks,
> > >>>Yu
> > >>>
> > >>>On Thu, Jan 19, 2012 at 10:59:14AM -0800, rchertov wrote:
> > >>>>What version of click are you running? Also, you can play
> > >>>>around with
> > >>>>BURST parameter to PollDevice. So are you observing that
> > >>>>polldevice::count is smaller than the number of packets you sent
> > >>>>to that
> > >>>>interface?
> > >>>>
> > >>>>Roman
> > >>>>
> > >>>>On 01.19.2012 10:45, Yu Chen wrote:
> > >>>>> Hi all,
> > >>>>>
> > >>>>> I am having an experiment with the following topology running on
> > >>>>> Deterlab. I
> > >>>>> am having a 4-core Xeon and modified e1000 driver running
> > >>ore Xeon
> > >>>>> and modified
> > >>>>> e1000 driver running. The memory is 4GB.
> > >>>>>
> > >>>>> User1
> > >>>>> |
> > >>>>> User3-----Router------User4
> > >>>>> |
> > >>>>> User2
> > >>>>>
> > >>>>> User 1 is sending packet to User 2; User 3 is sending packets
> > >>>>to User
> > >>>>> 4.
> > >>>>> The router only routes stuff with best effort. However, what I
> > >>>>> observed is
> > >>>>> very few losses, distributed on a small random set of rounds.
> > >>>>What is
> > >>>>> surprising is that there are losses on one particular
> > >>netcard when
> > >>>>> the sending
> > >>>>> rate is only 125,000 packets/second. The loss happens at the
> > >>>>> receiving phase.
> > >>>>>
> > >>>>> Below is what I have tried:
> > >>>>> 1' I am using 4 threads, each corresponding to one pair of
> > >>>>PollDevice
> > >>>>> and
> > >>>>> ToDevice.
> > >>>>> 2' I have changed the queue length at the interface cards. I set
> > >>>>> txqueuelen=1
> > >>>>> for the senders to decrease the possible burstiness. By
> > >>increasing
> > >>>>> the
> > >>>>> receiving queue length from 64 to 4096
> > >>>>> (the maximum queue length), the loss rate decreases.
> > >>>>> 3' I tried to discard the packets on their arrival at the
> > >>>>router. In
> > >>>>> this case,
> > >>>>> there is no observed loss.
> > >>>>>
> > >>>>> I am kind of lost at the point. Any suggestions on what can
> > >>be the
> > >>>>> possible
> > >>>>> reason and what I can try next?
> > >>>>>
> > >>>>> Attached is my code.
> > >>>>>
> > >>>>> Best Regards,
> > >>>>> Yu
> > >>>>>
> > >>>>> Router:
> > >>>>>
> > >>>>> -----------------------------------------------------------------------
> > >>>>> ddressInfo(router1-w1 10.0.1.1 00:15:17:5d:16:4a,
> > >>>>> //eth2
> > >>>>> user1-w1 10.0.1.2 00:15:17:5d:29:28, //eth0
> > >>>>> router1-w2 10.0.2.1 00:15:17:5d:16:48, //eth0
> > >>>>> user2-w2 10.0.2.2 00:15:17:57:ce:14, //eth0
> > >>>>> router1-w3 10.0.3.1 00:15:17:5d:16:4b, //eth3
> > >>>>> user3-w3 10.0.3.2 00:15:17:5d:27:e8, //eth0
> > >>>>> router1-w4 10.0.4.1 00:15:17:5d:16:49, //eth1
> > >>>>> user4-w4 10.0.4.2 00:15:17:5d:2a:3c, //eth0
> > >>>>> );
> > >>>>>
> > >>>>> c1 :: Classifier(12/0806 20/0001,
> > >>>>> 12/0806 20/0002,
> > >>>>> 12/0800,
> > >>>>> -);
> > >>>>> c2 :: Classifier(12/0806 20/0001,
> > >>>>> 12/0806 20/0002,
> > >>>>> 12/0800,
> > >>>>> -);
> > >>>>> c3 :: Classifier(12/0806 20/0001,
> > >>>>> 12/0806 20/0002,
> > >>>>> 12/0800,
> > >>>>> -);
> > >>>>> c4 :: Classifier(12/0806 20/0001,
> > >>>>> 12/0806 20/0002,
> > >>>>> 12/0800,
> > >>>>> -);
> > >>>>>
> > >>>>> q0 :: Discard;
> > >>>>> // Idle -> ToDevice(eth1);
> > >>>>> // Idle -> ToDevice(eth3);
> > >>>>> // Idle -> ToDevice(eth2);
> > >>>>> // Idle -> ToDevice(eth0);
> > >>>>> q1 :: CPUQueue(1000000) -> EtherEncap(0x0800, router1-w1,
> > >>>>user1-w1)
> > >>>>> -> td1 :: ToDevice(eth2);
> > >>>>> q2 :: CPUQueue(1000000) -> EtherEncap(0x0800, router1-w2,
> > >>>>user2-w2)
> > >>>>> -> td3 :: ToDevice(eth0);
> > >>>>> q3 :: CPUQueue(1000000) -> EtherEncap(0x0800, router1-w3,
> > >>>>user3-w3)
> > >>>>> -> td2 :: ToDevice(eth3);
> > >>>>> q4 :: CPUQueue(1000000) -> EtherEncap(0x0800, router1-w4,
> > >>>>user4-w4)
> > >>>>> -> td0 :: ToDevice(eth1);
> > >>>>>
> > >>>>> rt :: LookupIPRouteMP(10.0.1.0/32 0, 10.0.1.1/32 0,
> > >>>>10.0.1.255/32 0,
> > >>>>> 10.0.2.0/32 0, 10.0.2.1/32 0,
> > >>>>10.0.2.255/32 0,
> > >>>>> 10.0.3.0/32 0, 10.0.3.1/32 0,
> > >>>>10.0.3.255/32 0,
> > >>>>> 10.0.4.0/32 0, 10.0.4.1/32 0,
> > >>>>10.0.4.255/32 0,
> > >>>>> 10.0.1.0/24 1, 10.0.2.0/24 2,
> > >>10.0.3.0/24 3,
> > >>>>> 10.0.4.0/24 4, 0.0.0.0/0 0);
> > >>>>> rt[0] -> Discard;
> > >>>>> rt[1] -> q1;
> > >>>>> rt[2] -> q2;
> > >>>>> rt[3] -> q3;
> > >>>>> rt[4] -> q4;
> > >>>>>
> > >>>>> pd1 :: PollDevice(eth2) -> c1;
> > >>>>> c1[0] -> q0;
> > >>>>> c1[1] -> q0;
> > >>>>> c1[2] -> DropBroadcasts -> Strip(14) -> GetIPAddress(16) -> rt;
> > >>>>> c1[3] -> Discard;
> > >>>>>
> > >>>>> pd3 :: PollDevice(eth0) -> c2;
> > >>>>> c2[0] -> q0;
> > >>>>> c2[1] -> q0;
> > >>>>> c2[2] -> DropBroadcasts -> Strip(14) -> GetIPAddress(16) -> rt;
> > >>>>> c2[3] -> Discard;
> > >>>>>
> > >>>>> pd2 :: PollDevice(eth3) -> c3;
> > >>>>> c3[0] -> q0;
> > >>>>> c3[1] -> q0;
> > >>>>> c3[2] -> DropBroadcasts -> Strip(14) -> GetIPAddress(16) -> rt;
> > >>>>> c3[3] -> Discard;
> > >>>>>
> > >>>>> pd0 :: PollDevice(eth1) -> c4;
> > >>>>> c4[0] -> q0;
> > >>>>> c4[1] -> q0;
> > >>>>> c4[2] -> DropBroadcasts -> Strip(14) -> GetIPAddress(16) -> rt;
> > >>>>> c4[3] -> Discard;
> > >>>>>
> > >>>>> StaticThreadSched(pd2 0, td0 0, pd0 1, td2 1, pd3 2, td1 2,
> > >>pd1 3,
> > >>>>> td3 3);
> > >>>>>
> > >>>>> User:
> > >>>>> //
> > >>>>> AddressInfo(router1-w1 10.0.1.1 00:15:17:5d:16:4a,
> > >>>>> //eth2
> > >>>>> user1-w1 10.0.1.2 00:15:17:5d:29:28, //eth0
> > >>>>> router1-w2 10.0.2.1 00:15:17:5d:16:48, //eth0
> > >>>>> user2-w2 10.0.2.2 00:15:17:57:ce:14, //eth0
> > >>>>> router1-w3 10.0.3.1 00:15:17:5d:16:4b, //eth3
> > >>>>> user3-w3 10.0.3.2 00:15:17:5d:27:e8, //eth0
> > >>>>> router1-w4 10.0.4.1 00:15:17:5d:16:49, //eth1
> > >>>>> user4-w4 10.0.4.2 00:15:17:5d:2a:3c, //eth0
> > >>>>> );
> > >>>>> //Send
> > >>>>> udp :: FastUDPSource(RATE 400000, LIMIT 20000000, LENGTH 78,
> > >>>>> SRCETH user1-w1, SRCIP user1-w1, SPORT 1,
> > >>>>> DSTETH router1-w1, DSTIP user2-w2, DPORT 1,
> > >>>>> CHECKSUM false, INTERVAL 0, ACTIVE false)
> > >>>>> -> ToDevice(eth0);
> > >>>>>
> > >>>>> s :: Script(TYPE ACTIVE, write ctr.reset, write udp.active
> > >>false,
> > >>>>> wait 1, wait 9, write udp.active true);
> > >>>>>
> > >>>>>
> > >>>>> _______________________________________________
> > >>>>> click mailing list
> > >>>>> click at amsterdam.lcs.mit.edu
> > >>>>> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> > >>>>
> > >>>>_______________________________________________
> > >>>>click mailing list
> > >>>>click at amsterdam.lcs.mit.edu
> > >>>>https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
More information about the click
mailing list