Abstract

This whitepaper delves into the comprehensive exploration of MITMProxy—a pivotal network monitoring and security tool. From its seamless installation process to intricate configurations, the document elucidates every facet of MITMProxy, emphasising its role as a Certificate Authority and its integration with mobile devices. As cyber threats become increasingly sophisticated, understanding and leveraging tools like MITMProxy is paramount for professionals aiming to fortify network security and ensure data integrity.

Introduction

‘mitmproxy’ is an open-source tool that allows you to intercept, inspect, modify, and replay HTTP(S) and other network protocols. The name “mitmproxy” stands for “Man-In-The-Middle Proxy.” When used as a proxy, it sits between a client and a server, intercepting and relaying messages between the two. This capability makes it a powerful tool for debugging, analysing, and manipulating network traffic. Here are some key features and aspects of mitmproxy:

1. Interception: mitmproxy can intercept and display the content of HTTP and HTTPS traffic. This is especially useful for understanding the data being exchanged between a client and a server.
2. Modification: Users can modify requests and responses on-the-fly before they reach their destination. This can be useful for testing how systems react to specific inputs or conditions.
3. Replay: You can capture traffic and replay it later. This helps reproduce bugs or performance testing.
4. Scripting: With its scripting capabilities, users can automate specific actions or manipulations on the traffic. This is done using Python scripts.
5. Multiple Interfaces: It provides multiple interfaces, including:
   • mitmproxy: a terminal-based interface
   • mitmweb: a web-based interface
   • mitmdump: a command-line tool that provides tcpdump-like functionality
6. Transparent Proxying: mitmproxy can be set up as a transparent proxy, which can intercept traffic without requiring explicit configuration on the client’s side.
7. Support for Multiple Protocols: In addition to HTTP and HTTPS, mitmproxy also supports other protocols like WebSocket.
8. Certificate Generation: For HTTPS traffic, mitmproxy generates its own SSL certificates on the fly to intercept and decrypt traffic. Users will need to install the mitmproxy CA certificate on their client devices to avoid SSL warnings.
9. Extensibility: Being open-source, developers can extend its functionalities or integrate it with other tools as needed.
10. Platform Support: It’s available for Windows, macOS, and Linux.
11. Use Cases:
    • Security testing: Identify vulnerabilities in applications by observing their network traffic.
    • Debugging: Understand how an application communicates with back-end services.
    • Performance testing: Analyze the network traffic of applications to optimise their performance.

It’s important to note that while mitmproxy is a valuable tool for developers and security professionals, it can also be misused by malicious actors for nefarious purposes. As such, one should always ensure they have the necessary permissions and operate within legal and ethical boundaries when intercepting and modifying network traffic.

Why MITMProxy?

In a world replete with networking tools and utilities, MITMProxy stands out for several compelling reasons:

1. Unique Value Proposition: While many tools offer network monitoring, few provide the comprehensive interception, modification, and replay capabilities that MITMProxy does, especially with its seamless handling of encrypted traffic.
2. Open-Source and Actively Maintained: MITMProxy is open-source, meaning it’s freely available for anyone to use, modify, and distribute. Its active community ensures the tool is continuously updated with new features and security patches.
3. Community Trust: Being open-source and actively maintained, MITMProxy has garnered trust from a vast community of developers, security researchers, and network analysts. This trust is reflected in its widespread adoption and positive testimonials.
4. Comparative Advantage: When juxtaposed with similar tools, MITMProxy’s versatility—ranging from its scriptable nature to its multiple interfaces—makes it a preferred choice for professionals seeking an all-in-one solution.
5. Relevance in Modern Networking: With the increasing emphasis on HTTPS and secure communications, MITMProxy’s SSL/TLS interception prowess is more relevant than ever. As cyber threats evolve, tools like MITMProxy become indispensable in understanding and mitigating potential vulnerabilities.
6. Empowerment of Users: Beyond its technical capabilities, MITMProxy empowers its users. Whether it’s a developer diagnosing a stubborn application issue, a security researcher uncovering a new vulnerability, or a student gaining hands-on experience with network protocols, MITMProxy facilitates deeper insights and learning.
7. Detailed Insights: With MITMProxy, you can see details of every request and response, including headers, query parameters, and body content. This granularity is crucial for understanding the intricacies of network communication.
8. Interactive and Scriptable: While MITMProxy provides an interactive console for manual inspection and modification of traffic, it also supports scripting, allowing users to automate specific actions or manipulations using Python scripts.
9. SSL/TLS Interception: With the rise of HTTPS, many tools struggled with encrypted traffic. MITMProxy can generate and sign certificates on the fly, allowing it to decrypt and inspect HTTPS traffic once the MITMProxy CA certificate is trusted by the client device.
10. Multiple Interfaces: MITMProxy offers various interfaces, from the classic command-line interface (mitmproxy) to a web-based interface (mitmweb). This flexibility caters to different user preferences and ‘use cases’.
11. Cross-Platform: Whether you’re using Windows, macOS, or Linux, MITMProxy has you covered. Its cross-platform nature ensures that users from different environments can leverage its capabilities.
12. Mobile Integration: With the proliferation of mobile devices, understanding mobile network traffic has never been more crucial. MITMProxy facilitates easy setup for mobile devices to route their traffic through it.
13. Educational Value: For those looking to learn more about network protocols, security, or application behaviour, MITMProxy is an excellent educational tool. By observing real-world traffic, one can gain a deeper understanding of theoretical concepts.
14. Security Research and Penetration Testing: Security professionals can use MITMProxy to identify application vulnerabilities, test how they handle modified data or understand data exfiltration techniques.

While there are myriad tools available for network analysis, MITMProxy’s unique blend of features, community support, and timely relevance makes it an invaluable asset for anyone dealing with network communications.

In summary, MITMProxy is not just a mere traffic interceptor; it’s a comprehensive tool for anyone aiming to understand network traffic, debug issues, enhance security, or learn more about how applications communicate over the network.

Installation: A step-by-step guide to installing MITMProxy on various platforms.

Installing It

On Mac, mitmproxy is easy to install with Brew:

• brew install mitmproxy

Proxy setup:

1. On macOS (venura 13.0.1) go to “System settings” and then search for “proxies.”

2. Click on “Proxies”, then enable “HTTP” and “HTTPS” to redirect all the http and https traffics to a specified proxy server. Here, we set up localhost as a proxy server, I.e.,127.0.0.1 and port 8080, then click “OK.”

Running mitmproxy:

mitmproxy also has a web interface if you prefer the mouse over VIM keybindings. The advanced functionality is a bit more discoverable in the web interface, but the CLI version is convenient for quick capture sessions.

• mitmweb

If you browse any website in any browser, then it will go through the proxy server. As the proxy server’s certificate is not added to certificate verification, every website will show as “Not Secure”.

View all the sessions:

In the web browser, you can view all the unencrypted traffics.

Adding mitmproxy as A Certificate Authority

1. On macOS, the easiest way to add a new CA is to copy it to the desktop and then double-click it.
   • cp ~/.mitmproxy/mitmproxy-ca-cert.cer ~/Desktop
2. Double-click on the certificate, then it will open directly onto the “KeyChain Access.”

3. Double-click on the certificate and, expand “Trust”, and then select “Always Trust.”

To verify whether the proxy is running or not by checking the server certificate.

Write proxy data in human-readable format:

Create a Python script to decode all the mitmproxy data as “save.py”.

save.py

from mitmproxy.net.http.http1.assemble import assemble_request, assemble_response

f = open(‘/tmp/test/output.txt’, ‘w’)

def response(flow):
f.write(assemble_request(flow.request).decode(‘utf-8’))

Then, on the terminal, run the following comment while running mitmproxy –

• mitmdump -s save.py
• vim /tmp/test/output.txt

Setting up proxy forwarding on mobile phone:

In the laptop, start “mitmproxy” or “mitmweb” from a terminal. Then, the proxy will be listening on the 8080 port. Now from the mobile forward the traffic to that computer. But make sure both are on the same network connection (Wi-Fi connection).

1. On the computer, start “mitmweb” from the terminal. Then, on the computer, check for the computer’s IP address. In this example, both mobile and laptop are connected to the same Wi-Fi connection, i.e., “Guest”.

The laptop IP address for the Wi-Fi connection is 10.1.1.43, as shown in the below picture.

2. Now setup proxy forwarding on the mobile phone by the following steps:

Settings –> Network & internet –> tap on setting button of “Guest” –> on proxy option select “manual”
Then, put the computer’s IP address (10.1.1.43) as the “Proxy Hostname” field and 8080 as the “Proxy Port” field. Click the “Save” button.

3. Now, If you try to open “m.youtube.com” from a mobile browser, then it will show “Your connection is not private”. If you check for the certificate details by clicking the “Lock button “ beside the hyperlink, it will show the certificate details.

4. Now download the certificate from the official website, i.e., “mitm.ie” or by connecting another computer to the proxy. In our case, it is “mitmproxy-ca-cert.cer” file. Now add this certificate file to the trusted certificate list of the mobile by following the below steps:
   • Settings –> Advanced settings –> Encryption & credentials –> Install a certificate
     Now select the downloaded certificate and tap on “Install anyway”.

5. Now check the added certificate from the below option.
   • Settings –> Advanced settings –> Encryption & credentials –> Trusted credentials –> “USER” tab.

It will list all the user-trusted certificates added and will show “mitmproxy.” You can view the certificate by clicking on the certificate.

6. Now, if you browse any website, i.e., “m.youtube.com”, then it will run properly. Now if we check for the certificate details by clicking on the lock button beside the URL, it will show as “Connection is secure” and the certificate name as “mitmproxy”.

7. Now, on the laptop, we can view all the traffic which mobile browse. If you are using “mitmweb”, it will show in the browser tab of the laptop. As shown in the below screenshot.

Use Cases and Scenarios

MITMProxy’s capabilities extend far beyond just network monitoring. Its versatility allows it to be employed in a multitude of scenarios across various fields. Here are some real-world examples demonstrating the breadth of its applications:

1. Web Development and Debugging:
   1. Scenario: A web developer notices that a specific API call on their website takes longer than expected. Using MITMProxy, they can intercept this call, view its details, and analyse whether the delay is due to the request payload, the response, or an external factor.
   2. Application: MITMProxy allows developers to diagnose and rectify issues in real time, ensuring optimal website performance and user experience.
2. Security Research and Penetration Testing:
   1. Scenario: A security researcher is testing an application for potential vulnerabilities. By manipulating request parameters via MITMProxy, they can test how the application responds to unexpected or malicious input.
   2. Application: MITMProxy becomes an essential tool for uncovering security flaws and ensuring that applications are resistant to potential cyberattacks.
3. Mobile Application Analysis:
   1. Scenario: A company analyses a third-party mobile application before integrating it into its ecosystem. Using MITMProxy, they can monitor the data being sent and received by the application, ensuring it doesn’t leak sensitive information.
   2. Application: MITMProxy helps audit mobile applications, ensuring data privacy and compliance with company policies.
4. Educational and Training Purposes:
   1. Scenario: In a networking class, students learn about HTTP and HTTPS protocols. The instructor uses MITMProxy to showcase real-time traffic, helping students understand the intricacies of request-response mechanisms.
   2. Application: MITMProxy serves as an invaluable hands-on tool for students and educators in the realm of networking and cybersecurity.
5. Quality Assurance and Testing:
   1. Scenario: A QA engineer is testing a new feature in an application. They use MITMProxy to simulate different network conditions and responses, ensuring the application behaves correctly in various scenarios.
   2. Application: With MITMProxy, QA teams can create a wide range of test scenarios, ensuring robust application performance under all conditions.
6. Regulatory and Compliance Audits:
   1. Scenario: A financial institution must ensure its new application complies with data protection regulations. By routing traffic through MITMProxy, auditors can verify that sensitive data is transmitted securely and without leaks.
   2. Application: MITMProxy aids in regulatory audits, ensuring that applications adhere to industry standards and legal requirements.

In essence, MITMProxy’s multifaceted capabilities make it a tool of choice for professionals across various domains. From debugging and security research to education and compliance audits, its applications are vast, underlining its versatility and indispensability in the modern digital landscape.

Conclusion

In this whitepaper, we embarked on a comprehensive exploration of MITMProxy, a vital tool in the realm of network monitoring and security. Through detailed insights into its installation, configuration, and advanced usage techniques, we have unveiled the profound capabilities that this open-source tool offers to developers, security researchers, and network analysts alike.

The significance of MITMProxy in today’s digital landscape cannot be overstated. As cyber threats become increasingly sophisticated, tools that enable granular inspection, modification, and understanding of network traffic become indispensable. One can ensure robust network security and data integrity across platforms by adding MITMProxy as a Certificate Authority and integrating it seamlessly with mobile devices.

Moreover, the ability to write proxy data in a human-readable format and the provision to set up proxy forwarding on mobile phones further underline MITMProxy’s versatility and wide applicability.

In conclusion, whether you’re a developer looking to debug network issues, a security professional aiming to fortify defences, or simply an enthusiast eager to understand the intricacies of network communication, MITMProxy emerges as an invaluable asset. As the digital world continues to evolve, tools like MITMProxy will play a pivotal role in shaping secure, efficient, and transparent network ecosystems.

Appendix: Adding a Trusted Certificate on iOS Devices

For detailed steps on how to manually install and trust an SSL certificate on an iOS device, readers can refer to the following external guide: Installing an SSL Certificate on an iOS Device Manually. https://support.n4l.co.nz/s/article/Installing-an-SSL-Certificate-on-an-iOS-Device-Manually

References:

Laptop:

• How to Man in the Middle HTTPS Using mitmproxy (https://earthly.dev/blog/mitmproxy/)
• https://stackoverflow.com/questions/50207103/mitmproxy-make-output-human-readable-to-file
• https://discourse.mitmproxy.org/t/loop-decrypted-traffic-through-an-external-gateway/185
• https://discourse.mitmproxy.org/t/har-mongo-dump-script/901
• https://alternativeto.net/software/polarproxy/
• https://docs.mitmproxy.org/stable/

Mobile:

• https://support.n4l.co.nz/s/article/Installing-an-SSL-Certificate-on-an-iOS-Device-Manually
• https://support.google.com/pixelphone/answer/2844832?hl=en