MPI-SWS

Glasnost: Results from tests for BitTorrent traffic shaping

* Home * Glasnost Tests * Create your own test * Results * Contact *

 

Over the last three years, a large number of users from locations around the world have used our tool, Glasnost, to test whether their BitTorrent traffic is being shaped by ISPs. On this page, we present results from the tests conducted between January 1st, 2011 and November 5th, 2011.

The source code of our tool has been publicly accessible for more than 3 years now. You are welcome to download and inspect the code. Please contact us if you find any bugs or have questions, comments, or suggestions.

We published a technical paper describing our data collection and analysis methodology in the 7th Usenix Symposium on Networked Systems Design and Implementation (NSDI 2010). The paper explains in detail the potential sources of measurement errors and the checks we apply to minimize their impact on our results. You can download the paper in pdf format here.

Links to older releases of Glasnost results can be found here: February 1st, 2009, November 9th, 2008, and July 25th, 2008.

 

Table of contents
  1. How accurately can users detect BitTorrent traffic shaping by ISPs?
  2. What can we infer about traffic shaping policies of ISPs?
  3. Are ISPs shaping BitTorrent traffic?
  4. Where can we download the raw data and analysis scripts?
  5. Inferring traffic shaping policies using data cited in New York Times article on Glasnost

 

1. How accurately can users detect BitTorrent traffic shaping by ISPs?

We designed Glasnost to enable individual users to detect if their BitTorrent traffic is being shaped by ISPs. We can control the accuracy with which users can detect traffic shaping. But, in the process, we have to make some hard trade-offs between false positives, where the test indicates traffic shaping by ISPs even when they are not, and false negatives, where the test fails to detect traffic shaping even when ISPs are shaping traffic.

For details on how we can choose the trade-offs, please follow this link.

In brief, we can set the controls to detect any traffic shaping that reduces BitTorrent bandwidths by more than 20% with a 4% chance of false positives. Alternately, we could configure the controls to detect any traffic shaping that reduces BitTorrent bandwidths by more than 50% (higher false negatives) with less than 1% chance of false positives (lower false positives).

2. What can we infer about traffic shaping policies of ISPs?

Glasnost has been designed to enable individual users to detect if their traffic is being shaped by ISPs. Glasnost data from individual user tests can be used to infer if an ISP is deploying traffic shaping for at least some of its customers — by checking if the percentage of Glasnost tests for an ISP that indicate traffic shaping far exceeds the chance of false positives. However, Glasnost data cannot be used to accurately infer how widely traffic shaping is deployed within an ISP, i.e., the percentage of the ISP's users who are affected by traffic shaping.

For more details on how we can infer shaping policies of ISPs, please follow this link.

3. Are ISPs shaping BitTorrent traffic?

We present our analysis results for BitTorrent traffic shaping by ISPs world-wide in the table below. The table shows, for each ISP, the number of Glasnost tests ran by the ISP's customers as well the percentage of tests for which we detected traffic shaping of (a) BitTorrent uploads on any random port, (b) any TCP uploads on BitTorrent ports, (c) BitTorrent downloads on any random port, and (d) any TCP downloads on BitTorrent ports.

Note 1: When trying to infer whether an ISP is shaping traffic, it is important to compare the percentage of tests that detected traffic shaping with the chance of false positives. The greater the difference between the percentage of shaping detected tests for an ISP and the false positive rate, the more likely it is that the ISP is deploying shaping for at least some of its customers.

Note 2: To obtain the results, we configured our analysis such that there are few false positives (estimated to be less than 1%), but we fail to detect any traffic shaping that affects BitTorrent bandwidths by less than 50%.

Note 3: Our estimate of false positives, derived in our NSDI 2010 paper, is based on measurements conducted over wired (cable / DSL) access networks. The false positive rate might be higher for wireless access networks like WiMax or Cellular broadband networks. For such networks, it is better to allow for a higher false postive rate when inferring the presence of traffic shaping.

Note 4: We aggregated the results for different autonomous systems (ASs) owned by an ISP. For example, Tiscali UK and OPALTELECOM are two different ASs owned by TalkTalk. So we merged the results for user tests from both ASs under TalkTalk.

CountryISP# testsPercentage of traffic shaped (chance of false positives: 1%)
BitTorrent uploadsUploads on BitTorrent portBitTorrent downloadsDownloads on BitTorrent port
ArgentinaCablevision1250000
ArgentinaTelecentro10215053
ArgentinaTelecom Argentina1560114
ArgentinaTelefonica de Argentina1380100
AustraliaDodo1320012
AustraliaExetel1040310
AustraliaiiNet2720028
AustraliaInternode2301114
AustraliaOptus748112224
AustraliaTelstra7471413
AustraliaTPG Internet3810013
AustraliaWestnet1200057
BelgiumBelgacom1360000
BelgiumTelenet647161711
BrazilCanbras Net223151526
BrazilEmbratel11541358
BrazilGlobal Village Telecom8010114
BrazilHi151011226
BrazilNET Servicos de Comunicao1142161524
BrazilNTT America319383658
BrazilTelecomunicacoes do Brasil1450000
BrazilTelesc10350029
BrazilTelesp8660123
CanadaBell Aliant Regional Communications2610112
CanadaBell Canada103141392021
CanadaCanaca-com10624251213
CanadaCogeco2641010
CanadaDistributel1436646
CanadaEastLink2674321
CanadaMTS Allstream1080000
CanadaRogers271322823
CanadaShaw1740121801
CanadaTekSavvy33324261515
CanadaTelus4621011
CanadaVideotron Telecom2060000
ChileMoviestar1110013
ChileVTR Banda Ancha40101713
CroatiaT-Com Croatia1210010
Czech RepublicTelefonica o21330011
DenmarkTDC Data Networks1550000
EgyptTEData2931810
EstoniaElion Enterprises1030000
EstoniaElisa Eesti17848510
FinlandElisa1750100
FinlandTelia Sonera Finland1231000
FranceFrance Telecom – Orange3060001
FranceFree3860000
FranceNumericable1060000
FranceSociete Francaise du Radiotelephone2216911
GermanyDeutsche Telekom4730000
GermanyKabel Deutschland393118027
GermanyTelefonica o24470010
GermanyVodafone1671201
GreeceForthnet1440000
GreeceHellas OnLine2860001
GreeceOte4230010
Hong KongPCCW16710111622
HungaryDigicable1042402
HungaryMagyar Telekom2710000
HungaryUPC Broadband16171013
IndiaBharti Airtel13110116
IndiaMahanagar Telephone Nigam3170000
IndiaNational Internet Backbone6890011
IndiaReliance Communications1030011
IndiaTata Communications1190015
IndonesiaTelekomunikasi Indonesia1910221
IrelandEircom1031100
Israel013 NetVision16019201822
IsraelBezeqint47418221729
IsraelSmile Communications1850966
ItalyFastweb4830210
ItalyOpitel32611044
ItalyTelecom Italia27860000
ItalyTiscali Italia74500910
ItalyWIND12640011
JapanNTT471226120
JapanSoftbank1361414
JapanTechnology Networks (@NetHome)17247918
JapanVectant1400300
LithuaniaTEO LT1350101
MalaysiaTM Net57912337
MexicoUninet2260001
NetherlandsKPN2170000
NetherlandsT-Mobile1080000
NetherlandsTelfort1401000
NetherlandsZiggo3770011
New ZealandCallPlus3350801
New ZealandTelecom New Zealand1621211
New ZealandTelstraClear1090010
New ZealandVodafone NZ1290041
NorwayTelenor3240100
PhilippinesGlobe Telecoms1360121
PhilippinesSmart Broadband4313223
PolandNetia1770101
PolandTelekomunikacja Polska4470101
PolandToya13137300
PolandUPC Polska315181801
PortugalCabovisao10220151521
PortugalOptimus1200000
PortugalTelepac3031222
PortugalZON TV148201511
RomaniaRCS & RDS3210000
RomaniaRomtelecom1150000
RomaniaUPC Romania1430001
Russian FederationBeeline1560102
SingaporeSingNet416332259
SingaporeStarHub515813610
South AfricaInternet Solutions 2891937
SpainJazz Telecom1980000
SpainONO4560012
SpainOrange Espana3280010
SpainTelefonica6830001
SwedenCom Hem Sweden33700120
SwedenTele21260230
SwedenTelia Sonera1770000
SwitzerlandUPC Cablecom1370001
TaiwanChunghwa Telecom15290225
TaiwanDigital United68282412
TaiwanHoshin Multimedia Center1703226
TaiwanSony Network Taiwan1480202
TaiwanTaiwan Fixed Network38734357
ThailandTOT22524625
ThailandTrue Internet2205101745
Trinidad and TobagoColumbus Communications13823612
TurkeyTurk Telekomunikasyon2770012
United Arab EmiratesDu196269149
United Arab EmiratesEtisalat1700172
United KingdomBE11054712
United KingdomBritish Telecom (BT)285216175661
United KingdomEasynet7520000
United KingdomEclipse Internet1110001
United KingdomPlusNet1810010
United KingdomTalkTalk28619171322
United KingdomVirgin Broadband293814221532
United StatesAT&T38410001
United StatesCable One1320000
United StatesCablevision7690010
United StatesCenturyLink10710001
United StatesCenturyTel Internet1110020
United StatesCharter Internet17400000
United StatesCincinnati Bell1030020
United StatesClearwire1924431
United StatesComcast94770011
United StatesCox21160000
United StatesFrontier Communications of America1230010
United StatesInsight Communications1890010
United StatesMidcontinent Media1020000
United StatesRCN Corporation2460010
United StatesRoadRunner55490000
United StatesSuddenlink Communications2960010
United StatesVerizon20550110
United StatesWindstream Communications1820011
United StatesWOW1970000
VietnamVNPT1240031

4. Where can we download the raw data and analysis scripts?

You can download the raw data collected during Glasnost tests from Measurement Lab. Please follow the instructions on the page.

The tools that we used to analyze the data can be downloaded from here.

We encourage you to analyze the data and inspect our scripts. If you have any comments, questions, or suggestions, we welcome you to contact us.

5. Inferring shaping policies using data cited in New York Times article on Glasnost

Glasnost project was featured in a New York Times article on Novermber 14th, 2011. The article cited the percentage of Glasnost tests for which we detected traffic shaping for ISPs world-wide. When inferring whether an ISP is deploying traffic shaping, it is important to view those percentages in the context of the potential measurement errors, all of which were not emphasized sufficiently in the article. In particular, we wish to emphasize the following:

Note 1: The results cited in the article were obtained using an analysis configuration that detects any traffic shaping that reduces bandwidth by more than 20% (compared to 50% in the table above). With this more aggressive configuration, the chance of false positives are higher, around 4% (compared to 1% in the table above).

Note 2: More importantly, the article cited aggregated results from 4 different BitTorrent tests (BitTorrent uploads, BitTorrent downloads, TCP uploads on BitTorrent ports, and TCP downloads on BitTorrent ports), each of which has an measurement error of around 4%. That is, a user is reported to have observed traffic shaping, even if any one of the four BitTorrent tests she runs detects shaping. So the resulting measurement error in the aggregated results could be as high as 4 x 4% = 16%.

Note 3: When mapping AS names to ISPs, the analysis used an older (2008) version of the mapping table from IANA.

You can download an xls file containing the results cited by the New York Times article here.

As the aggregated results (column H in the .xls file) have a high chance (16%) of potential false positives, it is hard to infer with high confidence that any ISP for which the percentage of users that detected shaping is smaller than 32-48% (a factor of 2 to 3 larger than false positive rate) is actually shaping traffic. A more accurate inference can be drawn from the results of individual BitTorrent tests (columns C through G in the .xls file) which have a significantly lower chance of measurement error at 4%. To further improve the accuracy of inference, we recommend using results presented in the table above, where the measurement error is estimated to be lower than 1%.

For example, for US ISPs like AT&T and Verizon the percentages of tests that were reported to have detected traffic shaping fall within the range of false positives. Results presented in the table above (with tighter bounds on false positives) shows more clearly that there is no evidence of traffic shaping by these US ISPs.

On the other hand, the article correctly reports that BitTorrent traffic shaping appears to be more common in certain European countries like Britain. The percentage of tests that detected traffic shaping are so high (70% or higher) for some European ISPs that they exceed the chance of being accounted for by false positives (16%) alone. In fact, results presented in the table above (with tighter bounds on false positives) also show that some European ISPs are likely deploying BitTorrent traffic shaping.

 

Contact

This page is part of the research on residential broadband networks at the Max Planck Institute for Software Systems.
The results presented on this page were compiled by Marcel Dischinger and Krishna P. Gummadi on 28. November 2011 as part of the Glasnost project.

If you have questions or feedback, you can contact us via e-mail:

broadband @at@ mpi-sws mpg de