Over the past 2 months we've used Amazon's Mechanical Turk to pay users with high speed, mostly residential Internet connections, to run our cloud speedtest. To date, about 1100 unique users in 115 different countries have run the speedtest. This speedtest tracks the amount of time required to download a 1MB test file from 26 different cloud services (servers and content delivery networks), and records the transfer rate for each test and service run. We use MaxMind's geoip database to track where the user is downloading from (country + state/province for US/Canada). The order of the tests is always random and we enforce a time limit such that users with dial up or slow connections are excluded. We also limited individual users to running the speedtest only once per day. To date, about 1300 tests have been performed. About 60% of the tests were performed from US-based connections.
The purpose of these tests is to compare bandwidth speed from various cloud services to high-speed Internet connections. Businesses can use this data to determine which cloud services will provide the best throughput for their customer base.
We got a fairly diverse group of users to run the speedtest from US-based connections covering all 50 states. Voxel, IBM and EC2 (US East) were the top 3 in this category. IBM's service is still in beta and not intended for production use. The bottom 3 are located in Europe, explaining their slower bandwidth.
Non-US Speedtests to Cloud Servers
Most of the non-US speedtests were run by Canadian, European and Indian users. Linode's London data center performed very well in this category followed by EC2 (EU West) and Flexiscale.
Global Speedtests to Cloud ServersThis table shows the aggregate results from all speedtests (both US and non-US). The top 3 are the same as the US-based results, IBM, Voxel, and EC2 (US East).
US Speedtests to CDNs
The only major CDN we were unable to test is Akamai. This is because, unlike other CDNs, Akamai is still strongly opposed to a paygo model and the new realities of CDN pricing, and we are unable to commit to a $200/Mo, 50GB plan ($4GB!). Update: VPS.NET has announced they will be making Akamai CDN available on a zero commit paygo plan. We will add Akamai to the speedtest as soon as we are able to get setup with an account. More info is available here: http://bit.ly/98IQkC
As with our previous pingdom-based tests, CacheFly performed the best in all of our tests. Edgecast also performed very well. Edgecast is available with a zero-commit paygo plan through GoGrid through which it is very good from a price/features/performance perspective. Surprisingly, the #2 CDN (marketshare-wise) Limelight (via RackspaceCloud CDN) performed very poorly in all of our tests.
Non-US Speedtests to CDNs
Outside of the US, both CacheFly and Edgecast performed very well. Amazon's CloudFront came in 3rd, about 18% slower than CacheFly.
Global Speedtests to CDNs
Globally, CacheFly and Edgecast were neck-to-neck, followed by CloudFront at about 15% slower.
Summary
Bandwidth is certainly not the only consideration when choosing a cloud service vendor. However, it is one factor we believe businesses should consider in combination with quality of support, pricing, features and reliability. The combination of these factors should allow users to make a fairly objective decision about which vendor will provide the best overall service for their business.
On a side note, if you intend to use cloud services for internal-facing applications (e.g. virtual private clouds), bandwidth throughput to your business locations should be a very significant factor in your decision making process.
Our speedtest is available publicly in beta release at http://cloudharmony.com/speedtest
How about latency?
ReplyDeleteCan you try this out with the Windows Azure blob CDN service as well? I can help you get started with that if you need help
ReplyDelete- Sriram Krishnan (Windows Azure)
sriramk@microsoft.com
Add MaxCDN.com to the list when you expand the test to inlcude akamai. Uses the Mzima backbone, so it should be pretty fast unless their servers are underprovisioned.
ReplyDeleteHave you contacted Cotendo to include their results?
ReplyDeleteSriram: I emailed you directly regarding some technical limitations with Azure CDN that prohibit us from including it into the speedtest (lack of ability to upload a file to the root CDN path). If there is someway around that, we'd like to include Azure into the speedtest.
ReplyDeleteColin: The speedtest runs via a browser-based flash app which provides very restrictive access to anything other than basic http networking functionality. Because of this, we are not able to measure latency. However, we will be posting latency measurements for our Intracloud bandwidth and network testings.
ReplyDeleteRPM: maxcdn has been added to the speedtest, thanks for bringing that to our attention
ReplyDeleteJim: I just sent them an email
ReplyDeleteThe company was founded after seeing a niche in the market for high quality, high availability hosting, and is one of the most respected hosting companies in the marketplace. We have always been focussed on two things specifically; providing high technical quality services and making sure our clients are completely happy with our services.
ReplyDeleteIt's hard to find a good service. This is great information, thanks for the test results!
ReplyDeleteGet the lowest prices and the best deals of various Internet Service Providers in your area. Also compare DSL internet providers, cable internet providers, fiber -optic broadband and satellite internet providers for rural areas or where DSL and Cable are not possible. Get more details by logging on to http://www.highspeedinternet.com/
ReplyDeleteThanks for the article. It was well explained and it help me test my new Bandwidth Monitoring Software.
ReplyDelete