Caffeinated Bitstream

Bits, bytes, and words.

Bandwidth

The year in bandwidth

I collect data usage statistics on my home broadband connection using a script that polls my router's WAN interface counters via SNMP once a minute. Since I have all this data lying around, I thought it might be neat to chart my broadband usage for 2010 and get an idea of how much of a bandwidth hog I am. My usage includes lots of movie streaming, VoIP phone calls, and work-related applications (since I work from home).

My total usage in 2010 was about 209 GB of download and 24 GB of upload, for a monthly average of about 17 GB and 2 GB, respectively.

HTC EVO and background network activity

HTC EVO, shown with the optional extended-life battery.

Earlier this year, I ditched my antiquated Treo 755p for the new hotness in mobile phones—the HTC EVO, an Android phone available from Sprint. Unlike my previous phone, which would last a full week in standby without recharging, the EVO gulps battery like there's no tomorrow. In fact, the limited battery life has been the primary complaint about the phone. Nobody seems to know for certain why the battery life is so poor. Maybe it's the beefy 1Ghz Snapdragon processor, the large LCD screen, or something else entirely.

I was recently traveling in a rural area, and noticed that my EVO's battery would only last halfway through the day. All mobile phones consume more battery in rural areas where they must transmit with greater power to stay in contact with faraway towers. However, the extent to which the EVO's battery life problems were exacerbated by the higher transmitting power got me thinking that perhaps at least some of the battery drain may be due to software which occasionally causes the radio to transmit.

All modern smartphones have various software components which periodically execute in the background, unbeknownst to the user, and sometimes activate the radio to chat with their buddies in the cloud. Some of this chatter may be useful for the user, such as having fresh mail delivered immediately, while other chatter may provide little or no value.

I got curious about what the phone's software was actually doing while I wasn't around, so I captured and analyzed all the network traffic from my phone over a period of four hours. I accomplished this by turning off the 3G radio and packet sniffing the Wi-Fi network. (I didn't have an easy means of sniffing the 3G traffic directly.) I have my phone syncing with my Google account, Google Latitude, a couple of mail servers, and Facebook, and this is reflected in the captured data. I did not touch the phone at all for the duration of the test, so this traffic represents only non-interactive use. Here are the main traffic categories, sorted by decreasing packet count, with local network maintenance traffic (ARP, DHCP, etc.) removed:

ip address host events packets bytes notes
209.8.115.0/24
66.220.0.0/16
profile.ak.fbcdn.net
api.facebook.com
21353921050 Facebook sync, including downloading profile photos of friends.
74.125.227.0/24mobilemaps.clients.google.com1127889604 Google Latitude updates
-mail_server_1723738572 IMAP/SSL mail traffic
-mail_server_2116787852 IMAP/SSL mail traffic
74.125.47.188yw-in-f188.1e100.netmany747298 Encrypted packets exchanged with Google throughout the period. This is probably the C2DM system which Google uses for push notifications.
-dns_server32649811 Various DNS names are queried throughout the period to support connections to the other services listed here.
184.73.171.111prod.sprintzone.org1326628 The Sprint Zone app periodically communicates with a Sprint server hosted on Amazon EC2 to receive information about promotions and announcements.
216.241.36.232static.ak.fbcdn.net
(via the local Akamai server)
2274728 Fetching the Facebook "no profile" image. (As part of the sync?)
208.17.80.85resources.static.mobitv.com1202296 Sprint TV service. This code seems to be sloppy—two identical HTTP transactions are launched at the same time.
98.137.133.176proxy101.mobile.vip.sp2.yahoo.com1153163 Yahoo Finance Widget: fetch latest stock prices
131.107.13.100time-nw.nist.gov18599 Query the date and time via the Daytime protocol. This server is located at Microsoft.
208.77.19.5us.pool.ntp.org12180 Query the date and time via NTP.

Lots of background network activity can be minimized by tweaking the sync settings.

In total, about 1.2 MB of data was exchanged in 2277 packets: 186 KB (1270 packets) were transmitted, and 1002 KB (1007 packets) were received.

The Facebook sync seems to be the biggest data hog of these background services. Every two hours, it is downloading information about all my Facebook contacts, as well as fetching profile pictures of most of them (whether or not the profile pictures have changed recently). The sync frequency for my "Facebook for HTC Sense" program can be changed in the settings, so I changed it from every two hours to once a day. After Facebook, the biggest consumers seem to be Latitude and my mail servers. I knew what I was getting into when I configured those, so I can't complain. I didn't even realize that I had Yahoo's "Stocks" program set to sync, so I turned that off altogether. The Sprint Zone sync frequency can be changed in the Sprint Zone application itself, although I haven't found a way to limit the Sprint TV syncing. The next time I'm traveling in a rural area, I should remember to turn off the "background data" and "auto-sync" items in the sync settings.

There are a few possible flaws in using this method to predict battery consumption. The exact relationship between network activity and energy usage is not clear to me. For example, is the transmitter only active for just long enough to send a packet, or does it remain on for a bit longer in the hopes of another packet becoming available in the send queue? What would consume more energy—a burst of 100 kilobytes, or 100 short transmissions of 1 kilobyte each, spread throughout a period of time? Is it more meaningful to consider packet counts, or the total byte count? Are some of these services only transmitting because they know the network connection is Wi-Fi and not 3G?


Multimedia and bandwidth

Back in the 1990's, I predicted that all data-oriented utilities — telephone, television, radio, etc. — would eventually become Internet services. Today, the world is rapidly approaching this ideal as people ditch their POTS telephone lines for VoIP and their cable television subscriptions for streaming video. This disruptive technology has created tension in the industry as old-school telecommunications companies cling to their non-Internet services out of fear of their infrastructure turning into “dumb pipes,” while other companies are lining up to provide services over the Internet. Discussion of this disruption usually turns to the subject of bandwidth — the measure of the telecommunications infrastructure's capacity to deliver these services to users.

I became curious about exactly how much bandwidth is used by the multimedia services that I consume, so I decided to graph the bandwidth usage while using these services. I collected the data points from the SNMP network interface counter readings on my router, and compiled them into the graphs below. Each test includes a rough ~10-second lead-in/lead-out, and the data represents 1-second averages unless indicated otherwise.

I was a little surprised to see that Vonage telephone calls use so much bandwidth. I looked at my service settings on the Vonage web site, and my service is indeed configured to use “high quality” audio which uses a nominal bitrate of 90kbps in each direction. This is astonishing, since traditional POTS phone service uses a 64kbps audio channel (8Khz 8-bit sampling). It turns out that by default, Vonage does not use audio compression [1], and encodes audio using the raw PCM format. [2] I'm probably not getting much benefit from this high quality stream, since I'm using a POTS telephone which limits me to the low-quality 8Khz audio that telephone users have experienced for decades. Some day, I would love to ditch the POTS bottleneck altogether, since I've found that hi-fi audio conferencing is much more pleasant.

There's nothing too surprising about the Pandora bandwidth usage. From the looks of the graph, it's downloading each song in a quick burst at some point before the scheduled playback.

YouTube seems to be very front-heavy, filling a large buffer at the start and topping off the buffer periodically. From the length of the quiet area at the tail of the video, I'd guess that the buffer size in this test was about 78 seconds or 7.5MB.

As expected, watching YouTube video with the HD option uses more bandwidth. The buffer in this case seems to be around 123 seconds or 30MB.

The above test was performed with a Roku HD-XR set-top box streaming a feature-length film from Netflix. The bandwidth usage is extremely bursty, with the Roku box taking large gulps from the stream periodically. Note that the interval between bursts does not reveal the buffer size, since the box is only “topping off” its buffer at these times. However, from the quiet area at the tail of the graph, we can surmise a ballpark estimate of about 350 seconds or 64MB.

Because the burstiness of the first Netflix graph doesn't convey the general bandwidth usage very well, I created this second graph which uses 5-minute averages instead of 1-second averages. This graph represents the same viewing session as the first, but with the bandwidth usage smoothed out. The average bitrate is around 1.5Mbps.

Here's another bursty graph representing the viewing of a TV show on Hulu. I believe the empty intervals within the show correspond to advertisement breaks. Interestingly, it seems that Hulu buffers the advertising in advance, and does not use the network for the duration of the advertisement.

Here's the smoothed version of the Hulu graph with 5-minute averages. The average bitrate is about 694kbps, which is quite a bit more efficient than Netflix streaming. However, I think the quality was a little lower, and Hulu certainly demands a lot from my computer. My older laptop with a Centrino/Pentium-M processor cannot play Hulu video without skipping, although my newer Core 2 Duo laptop handles it fine.

All tests were performed at the Caffeinated Bitstream testing laboratories at an undisclosed location in the Rocky Mountain region.

Footnotes

  1. “Vonage utilizes VoIP technology differently from its competition, in that it doesn't compress your voice, so the sound quality is superb.” http://www.vonage.com/how_vonage_works_faq/
  2. “Vonage uses the standard PCM audio codec by default because people prefer it.” http://www.interesting-people.org/archives/interesting-people/200407/msg00231.html