Capture and monitor http(s) traffic from Android device through Fiddler using Wi-Fi and ProxyDroid

Developing your own application or making a security assessment of 3rd party application – it is very important to know, what traffic flows from an Android application? There are several ways to do it. We may use different proxies, we may utilize a real device or an emulator, and we might intercept air traffic or direct it through USB cable.

It is often that we need to intercept and analyze traffic from a real device and want to use Wi-Fi for that.
Here is how we may do that using Fiddler.


For this method to work you need a rooted Android device and an infrastructure which allows you to perform socket connections from your phone to your PC – Wi-Fi connection and an ordinary laptop with Wi-Fi module with would do just fine.

2. FIDDLER

Download the free version of Fiddler from Telerik. When running it the first time, make sure you allow Fiddler to accept network connections from the intra- or internet, depending on your infrastructure setup.

Configure Fiddler for incoming connections by going to Tools - Fiddler options... - Connections.. Set the following properties:

• Fiddler listens on port: 8888 or any other port not in use at your computer
• Allow remote computers to connect: on

Hit OK and restart Fiddler. 

2. PROXY DROID

Download Proxy Droid to your Android phone. Run it and configure it to point at your Fiddler instance by entering the IP of your computer and its port (8888 by default). Note that you must use the local IP address (often starting with 192.168.* or 10.*).

To test if your PC is reachable from your phone, you may simply ping it using ADB, given that it is connected with USB and have developer mode enabled. Example:
adb shell ping 192.168.1.6
(You need to have enabled reply to ICMP pings in your firewall for this to work. Fiddler may still work even if you get no ping replies)

You can also ping your phone from your PC. Acquire its IP address through this command:
adb shell netcfg

Usually the device eth0 will have the IP address you need.

3. TESTING

Simply start your favorite Android browser and navigate to a random web page. You should get the page presented normally in the browser, and see the HTTP requests line up in Fiddler. All apps will use this proxy now – not only the web browser. 

Fiddler with HTTP traffic log from the device

Whether you’re using this for policing other apps, or for debugging your own applications or 3rd party APIs, this is a technique that should be in every Android developer’s toolbox.


4. TROUBLESHOOTING

And what if the traffic from the targeted Android Applicaion is SSL encrypted?
It might be a very serious problem, if it uses its own SSL infrastructure (i.e. SSL pinning).

However, it might also be an issue with Android HttpsUrlConnection pipeline implementation - and Fiddler, happily, may fix it!
See a recipe in my previous post: How to decrypt SSL traffic from an Android app

How to decrypt SSL traffic from an Android app using Fiddler

There are plenty of tutorials on how you can intercept HTTP(s) traffic from Android using Fiddler.
Try this one: http://docs.telerik.com/fiddler/configure-fiddler/tasks/configureforandroid 

 However, it will fail when you try to intercept and decrypt Android SSL traffic coming from an application, and not from a browser.

 It might be that the application uses a certificate pinning – and you are probably cannot decipher this connection. Lost cause!
But more probably, the reason is a bug in the HttpsUrlConnection pipeline implementation.

 To solve the issue, please proceed with the following steps:

1 In Fiddler click "Rules->Customize Rules";
2 Find function OnBeforeResponse in the script
3 Add following code to the function body:

if (oSession.oRequest["User-Agent"].indexOf("Dalvik") > -1 && oSession.HTTPMethodIs("CONNECT")) { oSession.oResponse.headers["Connection"] = "Keep-Alive"; }

4 Save the file and restart Fiddler.

How to use PHP in RAILS application

Sometimes it is necessary to integrate PHP code into existing rails application. Using open source and time-tested PHPBB3 is a good example.

To do that, simply disable Passenger for the directory of your PHP application.

      Alias /forum /var/.../forum
      <Location /forum>
        # This allows PHP.
        PassengerEnabled off
        DirectoryIndex index.php 
        allow from all
        Options +Indexes
      </Location>

Enjoy!

WEBrick is so slow! why?

WEBrick is doing a reverse DNS lookup on connecting IPs by default. In other words, it's trying to see if your IP address is associated with a domain name. This is unnecessary and takes too long, so you can disable it.

Open the file "l/ruby/lib/ruby/1.9.1/webrick/config.rb" and locate the line with ":DoNotReverseLookup => nil".
Change nil to true.

Enjoy!

Installing Webmin on Ubuntu

Webmin is a perfect way to administer an Unix or Linux server. Some say it is too easy, and we, IT guys, should never stoop to it!

Hope nobody ever reads these words: I use Webmin a lot. If you are also lazy and of low moral standards, you may install it on Ubuntu using this simple instructions:


1. Download the latest webmin from http:www.webmin.com .
wget http://www.webmin.com/download/deb/webmin-current.deb

2. Execute webmin package. It will fail but it’s easier for us, since we’ll just force installing dependencies, rather than installing them one by one.

sudo dpkg -i webmin-current.deb

3. Install missing dependencies and it will automatically recompile webmin

sudo apt-get -f install

Enjoy webmin on https://[serverIP]:10000 .

LIKE on PostgreSQL - how to use an index

Using Indexes on LIKE queries on PostgreSQL might be tricky.

First, starting your search with '%' will always lead to full table scan.
OK, this is easy to understand and there is a recommended way to fix it - fulltextsearch.

Second, context-independent search ILIKE will also use table scan instead of index.

But if database is initialized with utf8 you will also get painfully slow table scan, until create an index, with an option, designed for this particular purpose: text_pattern_ops

Here is an example:
CREATE INDEX  ON MyTable (myField text_pattern_ops ASC NULLS LAST);

In non-C locales (which you're almost certainly using if you're using

UTF8) the ordering which the normal text operations use can be quite complex.

Just as an example most locales have spaces being entirely insignificant. 

So no range can reliably match a prefix LIKE pattern. The text_pattern_ops use

simple character-by-character ordering which are useful for LIKE but not for

regular < and > comparisons. They're just two different orderings.

Some sources also recommend indexes with  lower option:

CREATE INDEX  ON MyTable (lower(myField) text_pattern_ops ASC NULLS LAST);

For me, it did not work with postgresql 9.0. Just stay with regular text_pattern_ops.

Customizing JSON services in Rails

In my previous sample I got this result from a JSON service:

[
{"created_at":"2012-08-12T13:56:42Z","id":1,"time":"2012-08-12T13:30:00Z","title":"Test Event \u21161","updated_at":"2012-08-12T13:57:40Z"},
{"created_at":"2012-08-12T13:57:07Z","id":2,"time":"2012-08-17T16:00:00Z","title":"Test Event \u21162","updated_at":"2012-08-12T13:57:07Z"},
{"created_at":"2012-08-12T13:57:29Z","id":3,"time":"2012-08-20T10:00:00Z","title":"Test Event \u21161","updated_at":"2012-08-12T13:57:29Z"}]

But what if I want to customize the result? Here is when function overriding comes handy.
Override as_json function in data model Event, like this:

class Event < ActiveRecord::Base
  attr_accessible :time, :title
 def as_json(options = {})
  result = {id: self.id, EventName: self.title, WillTakePlace: self.time }
 end
end

Viola! This is customised JSON data (as you see, I chose only three attributes and renamed two of them):

[
{"id":1,"EventName":"Test Event \u21161","WillTakePlace":"2012-08-12T13:30:00Z"},
{"id":2,"EventName":"Test Event \u21162","WillTakePlace":"2012-08-17T16:00:00Z"},
{"id":3,"EventName":"Test Event \u21161","WillTakePlace":"2012-08-20T10:00:00Z"}]

and what if I need something more complicated? Probably, I would call to RABL