Easy and accessible network request analysis

I had to analyse some network requests while I was working on an Android app project. That turned out to be surprisingly easy, using mitmproxy. This article covers the proxy installation and usage by wget and within the source code of an Android app.

Basically it works like a mitm attack. You have to install a proxy server somewhere in your local network and route client requests through it. I’ve chosen mitmproxy, because it’s available for the Linux console, simple to use and its ui is fairly accessible to screen readers like brltty.

Install: sudo apt install mitmproxy

Run: mitmproxy --listen-host 192.168.100.100 --listen-port 22222 --save-stream-file +requests.log

Wget

Now try to send a http request through the proxy:

wget -e https_proxy=192.168.100.100:22222 http://example.org

It should appear in the main view of mitmproxy.

If you want to intercept encrypted requests as well, your client must trust the ssl certificate, which mitmproxy created during its first launch. You may find it under ~/.mitmproxy/mitmproxy-ca-cert.pem. Copy the file onto your client machine and send a second request:

wget -e https_proxy=192.168.100.100:22222 --ca-certificate /path/to/mitmproxy-ca-cert.pem https://example.org

If everything went well, you should see two lines in the mitmproxy main view. Use the arrow keys to select a request and press ENTER to show its details.

Unfortunately the view doesn’t use the hardware console cursor. Therefore screen readers can’t announce the selected list item properly. If you use brltty, you may disable the autom. cursor tracking (CAPSLOCK+ENTER) and follow the selected list item manually. It’s prefixed by “>>”.

The request details view consists of three tabs: request, response and details. Select them with left and right arrow keys or tab through). I was only interested in the request cookie headers, which I found directly in the request tab.

Android app development

First add the same mitmproxy certificate to your apps “res” folder:

mkdir /path/to/android_project/app/src/main/res/raw/
cp ~/.mitmproxy/mitmproxy-ca-cert.pem /path/to/android_project/app/src/main/res/raw/mitm_proxy.pem

Then include the following code into your app to:

1. set the proxy
2. use the SSL certificate from above for the request

import org.example.project.BuildConfig;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.URL;
import java.security.KeyStore;
import java.security.cert.CertificateFactory;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;

URL url = new URL("https://example.org");
String proxyHost = "192.168.100.100";
int proxyPort = 22222;

HttpsURLConnection connection = null;
if (BuildConfig.DEBUG) {

    Proxy proxy = new Proxy(
            Proxy.Type.HTTP, new InetSocketAddress(proxyHost, proxyPort));
    connection = (HttpsURLConnection) url.openConnection(proxy);

    try {
        CertificateFactory cf = CertificateFactory.getInstance("X.509");
        InputStream caInput;
        // Create a KeyStore containing our trusted CAs
        KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        keyStore.load(null, null);
        // load mitm_proxy certificate
        caInput = ApplicationInstance.getContext().getResources().openRawResource(R.raw.mitm_proxy);
        try {
            keyStore.setCertificateEntry("ca1", cf.generateCertificate(caInput));
        } finally {
            caInput.close();
        }
        // Create a TrustManager that trusts the CAs in our KeyStore
        String tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
        TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
        tmf.init(keyStore);
        // Create an SSLContext that uses our TrustManager
        SSLContext sslContext = SSLContext.getInstance("TLS");
        sslContext.init(null, tmf.getTrustManagers(), null);
        connection.setSSLSocketFactory(sslContext.getSocketFactory());
    } catch (Exception e) {
        System.out.println(
                String.format("Cert exception: %1$s", e.getMessage()));
    }

} else {
    connection = (HttpsURLConnection) url.openConnection();
}

Of course it’s also possible to reroute every request from the Android device through your proxy. Then you have to add the certificate to your cert-store (Android settings -> security) and set a system-wide proxy. Unfortunately not every application respects this - for example browsers may define their own proxy settings.

Therefore if you want to debug your own app, go for the direct integration - much easier to implement.