Adventures in Androidland

Analyzing Bandwidth Usage

When making changes to an application that is used by millions worldwide, it is important to be mindful of not only the end user experience, but of the application footprint as well: How much space are we taking up on the device? How much memory do we need while running? How much bandwidth are we consuming? We ask ourselves these and many other questions on a regular basis, and seek ways to answer them in a scalable fashion.

Given our recent international growth and the very real monetary implications of excessive API calls, we sought to answer the question of bandwidth usage. At that moment in time, our workflow was to finish coding a feature, write automated tests for it, perform some rudimentary Charles Proxy checks of the sent requests, and ship it. It is only once the feature reached a full rollout in our production application that we would be able to see its effects on our servers.

This model recently broke down, when we saw a steadily increasing number of API requests in our production application. What turned out to be a caching issue was not caught in development, internal testing, or even our external beta application! We knew that there had to be a better way.

The answer, of course, was automation. Our desired end result was a regularly running job that would measure app bandwidth usage and warn us if something was amiss. To get there, we had to do four things:

1. Create a suite of UI "tests" that would take the app through our core experiences

2. Instrument our network usage with request and response logging

3. Write a script that would parse and summarize these response logs

4. Store the logs and generated summaries for trend tracking

With our work broken down into these high-level tasks, we could begin. 

UI Tests

We currently use Jenkins for running our automated UI and unit testing, so piggy-backing off it was the quickest way to get up and running. Since our bandwidth usage tests are not UI tests in the normal sense, in that they lack failure conditions, it made sense to place them in a separate suite.

For the purposes of testing our caching behaviour, five different application path "tests" were written:

1. Logged-out cold start

2. Logged-out warm start

3. Logged-in cold start

4. Logged-in warm start

5. Generic application path

In all of the above, all screens are visited twice to validate proper caching behaviour. In terms of the start-up tests (1-4), a cold start refers to the application not running when it is invoked. We simulate this with a force kill ADB command:

adb shell am force-stop wp.wattpad

A warm start refers to the application already running when it is invoked. We simulate this by backing out of the application, which has the result of killing our UI, but leaves the application intact in memory.

Lastly, the generic application path is responsible for logging in the user, killing the app, starting the app up again, and then traversing the the majority of our application screens twice. In this way, we should see a vast majority of the bandwidth usage we expect in the application, along with our caching behaviour.

Network Instrumentation

Our application network framework is an extension of Square's OkHttp, so we were able to easily add some custom logging to our network requests. The things we care about most are request duration and payload, and response payload size. You can see some of the added logs in the image below.