[Uia-users] UIA data compression

Jacob Strauss jastr at MIT.EDU
Mon Aug 17 10:47:28 EDT 2009


OpenSSL uses its own compression methods internally by default:

  http://www.openssl.org/docs/ssl/SSL_COMP_add_compression_method.html

-Jacob

On Aug 17, 2009, at 10:18 AM, Ye Wang wrote:

> I'm pretty sure that I am noticing compression when I use ifstat to  
> monitor the live interface traffic when running iperf with  
> different types of payload.
>
> When I run iperf over UIA using its default data payload settings  
> (which I believe is highly compressible zero-padded packets) or  
> setting the payload to a highly compressible source file (i.e. /dev/ 
> zero), I noticed that the traffic over the physical interface is  
> less by a factor of 4 or 5 than what is going through the UIA  
> interface and what is reported at both ends by iperf. However, if I  
> switch the data payload (using the -F option in iperf) to something  
> poorly compressible, such as /dev/urandom or a compressed file, I  
> notice that the traffic over the physical interface now matches the  
> UIA interface traffic with some overhead.
>
> I've attached the output from some tests demonstrating the effect.  
> The same thing happens with UDP tests as well.
>
> The router daemons on separate machines communicate via RCP over  
> SSL (which runs over TCP), right? Could the way in which data is  
> serialized by RPC be the cause of the apparent compression?
>
> Ye
>
> Jacob Strauss wrote:
>> It's encrypted via SSL, but not compressed.  You could probably   
>> change it from using the sunrpc_channel_ssl classes to   
>> sunrpc_channel_tcp instead, but I'm not sure if that's a trivial   
>> change or not.
>> -Jacob
>> On Aug 14, 2009, at 3:04 PM, Ye Wang wrote:
>>> It looks like UIA compresses the data payload. Why does it to do  
>>> that?
>>> How could that be disabled?
>>>
>>> Ye
>>> _______________________________________________
>>> Uia-users mailing list
>>> Uia-users at amsterdam.lcs.mit.edu
>>> https://pdos.csail.mit.edu/mailman/listinfo/uia-users
> "ubuntu" is the data source, "nettools" is the data sink with the  
> UIA name "handegg.net1"
>
> ye21867 at ubuntu:~$ iperf -V -c handegg.net1 -i 1 -f K
> ------------------------------------------------------------
> Client connecting to handegg.net1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  4] local fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42497  
> connected with fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  4]  0.0- 1.0 sec  6248 KBytes  6248 KBytes/sec
> [  4]  1.0- 2.0 sec  5248 KBytes  5248 KBytes/sec
> [  4]  2.0- 3.0 sec  5040 KBytes  5040 KBytes/sec
> [  4]  3.0- 4.0 sec  5128 KBytes  5128 KBytes/sec
> [  4]  4.0- 5.0 sec  5976 KBytes  5976 KBytes/sec
> [  4]  5.0- 6.0 sec  5464 KBytes  5464 KBytes/sec
> [  4]  6.0- 7.0 sec  5496 KBytes  5496 KBytes/sec
> [  4]  7.0- 8.0 sec  5424 KBytes  5424 KBytes/sec
> [  4]  8.0- 9.0 sec  5304 KBytes  5304 KBytes/sec
> [  4]  9.0-10.0 sec  6128 KBytes  6128 KBytes/sec
> [  4]  0.0-10.1 sec  55464 KBytes  5479 KBytes/sec
> ye21867 at ubuntu:~$ iperf -V -c handegg.net1 -i 1 -f K -F /dev/zero
> ------------------------------------------------------------
> Client connecting to handegg.net1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  5] local fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42498  
> connected with fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  5]  0.0- 1.0 sec  5096 KBytes  5096 KBytes/sec
> [  5]  1.0- 2.0 sec  4760 KBytes  4760 KBytes/sec
> [  5]  2.0- 3.0 sec  5560 KBytes  5560 KBytes/sec
> [  5]  3.0- 4.0 sec  4216 KBytes  4216 KBytes/sec
> [  5]  4.0- 5.0 sec  5480 KBytes  5480 KBytes/sec
> [  5]  5.0- 6.0 sec  6256 KBytes  6256 KBytes/sec
> [  5]  6.0- 7.0 sec  4896 KBytes  4896 KBytes/sec
> [  5]  7.0- 8.0 sec  5832 KBytes  5832 KBytes/sec
> [  5]  8.0- 9.0 sec  5336 KBytes  5336 KBytes/sec
> [  5]  9.0-10.0 sec  5440 KBytes  5440 KBytes/sec
> [  5]  0.0-10.1 sec  52880 KBytes  5251 KBytes/sec
> ye21867 at ubuntu:~$ iperf -V -c handegg.net1 -i 1 -f K -F /dev/urandom
> ------------------------------------------------------------
> Client connecting to handegg.net1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  5] local fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42499  
> connected with fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  5]  0.0- 1.0 sec  5160 KBytes  5160 KBytes/sec
> [  5]  1.0- 2.0 sec  3832 KBytes  3832 KBytes/sec
> [  5]  2.0- 3.0 sec  4552 KBytes  4552 KBytes/sec
> [  5]  3.0- 4.0 sec  3936 KBytes  3936 KBytes/sec
> [  5]  4.0- 5.0 sec  5472 KBytes  5472 KBytes/sec
> [  5]  5.0- 6.0 sec  4048 KBytes  4048 KBytes/sec
> [  5]  6.0- 7.0 sec  3944 KBytes  3944 KBytes/sec
> [  5]  7.0- 8.0 sec  3536 KBytes  3536 KBytes/sec
> [  5]  8.0- 9.0 sec  4592 KBytes  4592 KBytes/sec
> [  5]  9.0-10.0 sec  4256 KBytes  4256 KBytes/sec
> [  5]  0.0-10.1 sec  43336 KBytes  4288 KBytes/sec
> ye21867 at ubuntu:~$ iperf -V -c handegg.net1 -i 1 -f K -F testgzip.gz
> ------------------------------------------------------------
> Client connecting to handegg.net1, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  5] local fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42500  
> connected with fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  5]  0.0- 1.0 sec  5368 KBytes  5368 KBytes/sec
> [  5]  1.0- 2.0 sec  4352 KBytes  4352 KBytes/sec
> [  5]  2.0- 3.0 sec  3800 KBytes  3800 KBytes/sec
> [  5]  3.0- 4.0 sec  4136 KBytes  4136 KBytes/sec
> [  5]  4.0- 5.0 sec  5176 KBytes  5176 KBytes/sec
> [  5]  5.0- 6.0 sec  3144 KBytes  3144 KBytes/sec
> [  5]  6.0- 7.0 sec  5384 KBytes  5384 KBytes/sec
> [  5]  7.0- 8.0 sec  3416 KBytes  3416 KBytes/sec
> [  5]  8.0- 9.0 sec  3960 KBytes  3960 KBytes/sec
> [  5]  9.0-10.0 sec  4152 KBytes  4152 KBytes/sec
> [  5]  0.0-10.1 sec  42896 KBytes  4236 KBytes/sec
>
>
>
> ye at nettools:~$ iperf -V -s -f K
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 85.3 KByte (default)
> ------------------------------------------------------------
> [  4] local fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001  
> connected with fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42497
> [ ID] Interval       Transfer     Bandwidth
> [  4]  0.0-10.2 sec  55464 KBytes  5429 KBytes/sec
> [  5] local fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001  
> connected with fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42498
> [  5]  0.0-10.4 sec  52880 KBytes  5098 KBytes/sec
> [  4] local fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001  
> connected with fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42499
> [  4]  0.0-10.3 sec  43336 KBytes  4215 KBytes/sec
> [  5] local fc00:e6c:d87d:92ef:79c4:c13f:38b:7df2 port 5001  
> connected with fc00:9d29:7933:dd7a:cd9a:1e92:2f72:7623 port 42500
> [  5]  0.0-10.3 sec  42896 KBytes  4162 KBytes/sec
>
>
>
> ye21867 at ubuntu:~$ ifstat -i eth0,uia
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>     0.33      0.00      0.00      0.00
>     0.66      0.32      0.00      0.00
>     0.27      0.07      0.00      0.00
>     1.06      0.52      0.00      0.00
>     0.43      0.07      0.00      0.00
>     0.61      0.07      0.00      0.00
>   365.65    404.74     64.65   3267.61
>   791.19    876.08    134.83   5568.08
>   724.66    881.19    124.91   5993.02
>   684.42    959.58    105.51   4658.58
>   781.78    974.19    128.51   5753.77
>   709.99    818.84    121.13   6238.82
>   774.81    945.00    120.68   5441.07
>   844.72    905.90    131.12   5838.35
>   735.16    923.67    152.03   6080.30
>   829.19    929.64    161.16   6471.42
>   588.28    712.78    115.63   2888.57
>     0.21      0.07      0.00      0.00
>     2.27      2.02      0.00      0.00
>     0.51      0.13      0.00      0.00
>     0.56      0.34      0.00      0.00
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>     4.17      4.48      0.73      2.79
>   142.10    190.62     37.74   1749.68
>   726.91    842.54    126.73   4732.36
>   755.45    809.56    140.70   5524.58
>   713.11    898.63    127.64   5038.45
>   645.93    881.64    133.15   4956.23
>   837.03    918.45    145.95   6086.14
>   749.19    995.03    142.26   5643.15
>   809.30    864.59    143.00   5697.42
>   704.28    857.71    151.60   6063.62
>   799.34    857.31    130.92   5260.08
>   742.05    942.60    146.11   4687.79
>   138.96    131.11     15.86     40.06
>     0.36      0.07      0.00      0.00
>     0.46      0.07      0.00      0.00
>     0.27      0.07      0.00      0.00
>     4.28      0.07      0.00      0.00
>     0.33      0.07      0.00      0.00
>     0.21      0.07      0.00      0.00
>    58.42    357.75     20.62    916.58
>   791.49   5096.82    105.71   4699.18
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>   841.69   5257.60    107.46   4147.58
>   713.71   5435.90     89.03   5093.01
>   763.03   5799.93     96.30   4471.66
>   674.34   5492.32    102.53   5029.62
>   682.14   5373.94    103.63   4112.27
>   846.24   5106.56    115.20   4319.62
>   768.83   4918.39     92.96   3990.17
>   726.70   5472.67    110.38   4446.72
>   803.75   4702.61    123.59   4249.74
>   231.54   1176.34     24.38      0.07
>     0.65      0.07      0.00      0.00
>     0.82      0.28      0.00      0.00
>     0.27      0.07      0.00      0.00
>     1.69      0.75      0.00      0.00
>     0.43      0.07      0.00      0.00
>     0.30      0.07      0.00      0.00
>     0.49      0.07      0.00      0.00
>     0.51      0.07      0.00      0.00
>    42.45    232.58     23.21   1032.35
>   747.27   5266.79    108.09   4355.17
>   913.02   5503.64    101.70   4500.73
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>   884.03   4997.79    111.42   4833.13
>   868.25   5676.56     87.33   4944.41
>   746.04   5406.48    112.53   4456.75
>   764.34   4716.49     96.72   3844.14
>   806.39   5464.78    106.43   4222.49
>   765.01   5392.27    127.66   5115.10
>   853.76   5104.74     85.92   3435.57
>   907.75   4941.47    116.40   4271.73
>   222.33   1041.86     31.95      0.07
>     0.33      0.07      0.00      0.00
>     1.89      1.59      0.00      0.00
>     0.33      0.07      0.00      0.00
>
>
>
> ye at nettools:~$ ifstat
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>     0.30      0.28      0.00      0.00
>     1.24      0.85      0.00      0.00
>     0.18      0.14      0.00      0.00
>     0.42      0.14      0.00      0.00
>     0.27      0.14      0.00      0.00
>   644.98    479.81   3278.93     76.62
>   763.92    765.01   5579.80    138.33
>   962.22    759.33   5117.65    100.34
>   885.78    692.12   5556.45    118.04
>   782.85    718.61   5768.00    120.60
>  1007.47    645.89   5518.24     99.84
>   865.38    830.85   6194.22    137.98
>   896.89    737.27   5854.05    125.69
>   941.82    790.71   6101.43    146.26
>   846.48    707.30   5738.23    137.49
>   516.80    477.27   3645.84     87.36
>     0.18      0.14      0.00      0.00
>     1.03      0.98      0.00      0.00
>     0.24      0.14      0.00      0.00
>     2.39      1.98      0.47      0.12
>     0.30      0.14      0.00      0.00
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>   315.72    287.52   1786.50     51.90
>   789.33    747.65   4710.56    135.95
>   910.11    652.44   4561.01    109.24
>   811.17    714.09   6031.93    144.52
>   915.12    698.79   4897.74    117.41
>   892.99    793.17   6178.29    148.10
>   933.64    721.33   5657.57    135.60
>   893.12    728.47   5709.00    136.79
>   860.44    720.86   5235.64    125.48
>   877.63    723.22   5004.98    119.94
>   774.44    765.96   5851.94    140.80
>     0.25      0.14      0.00      0.00
>     0.98      0.14      0.00      0.00
>     0.12      0.14      0.00      0.00
>     0.30      0.14      0.00      0.00
>     0.65      0.14      0.00      0.00
>     0.58      0.14      0.00      0.00
>     0.24      0.14      0.00      0.00
>  1553.28    210.32   1141.44     32.17
>  5397.21    713.37   4242.37    106.53
>  4953.36    754.05   4548.00    100.35
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>  5671.04    669.00   4312.31     75.56
>  5376.71    755.95   5133.92    110.66
>  5806.44    557.44   4022.75     76.82
>  4891.59    720.88   4458.73    107.00
>  4951.82    717.91   4338.41    104.00
>  5112.43    768.00   4768.05    114.39
>  5043.71    634.56   4196.89    100.55
>  5090.67    723.38   4423.06    106.03
>    36.31     31.04      0.07      0.00
>     0.30      0.14      0.00      0.00
>     1.48      1.43      0.00      0.00
>     0.24      0.14      0.00      0.00
>     0.12      0.14      0.00      0.00
>     0.24      0.14      0.00      0.00
>     0.18      0.14      0.00      0.00
>     0.18      0.14      0.00      0.00
>     0.24      0.14      0.00      0.00
>  1572.75    220.97   1298.75     30.41
>  5303.01    774.11   4102.28    126.12
>  4990.69    742.16   4512.30     95.64
>  5574.96    689.02   4194.01     78.36
>        eth0                uia
>  KB/s in  KB/s out   KB/s in  KB/s out
>  5781.35    667.44   4639.91     87.07
>  5303.51    724.92   4634.72    111.08
>  4573.13    687.13   4336.15    100.14
>  5021.17    782.20   4551.12    113.09
>  5445.77    746.55   3917.75     93.88
>  4830.97    790.62   4654.49    112.07
>  5041.19    805.03   4279.49    102.67
>     0.30      0.14      0.00      0.00
>     0.24      0.14      0.00      0.00
>     0.87      0.76      0.00      0.00
>     0.24      0.14      0.00      0.00
>     1.26      1.21      0.00      0.00
>



More information about the Uia-users mailing list