Saturday, July 24, 2010

How to count instances of an IP address in a log file

I was looking for a quick and painless way to determine what IP addresses used our web app the most and stumbled across this excellent post. I got reminded of the power of the command-line interface and that often, I underestimate the way these commands are supposed to make a sysad's life easier.

So here's a quick tip for counting unique visitors to your web app. Locate your web app's log file and do a quick scan of the last few lines. In my case, this is what I saw: - - [10/Jul/2010:00:57:29 -0400] "POST /v2/api/execute.php?method=upload_time&ver=win1.1.15 HTTP/1.1" 200 116 "-" "Mozilla/5.0" - - [10/Jul/2010:00:57:30 -0400] "POST /v2/api/execute.php?method=timestat&ver=win1.1.15 HTTP/1.1" 200 288 "-" "Mozilla/5.0" - - [10/Jul/2010:00:57:29 -0400] "POST /v2/api/execute.php?method=offtime&ver=win1.1.15 HTTP/1.1" 200 19 "-" "Mozilla/5.0" - - [10/Jul/2010:00:57:32 -0400] "POST /v2/api/execute.php?method=get_defaults&ver=win1.1.15 HTTP/1.1" 200 77 "-" "Mozilla/5.0"

In this log format, the IP address appears first, so we will only need to get the first set of characters, separated by a space. In order to count instances of an IP address, we sort the IP addresses so similar IP addresses are grouped together, then count them. Then we sort them again so that the sums are arranged. The number of unique IPs may give you pages of standard output, so optionally, if you only want to see which IP addresses accessed the web app the most, we look at the top results only.

So these are the commands used, piped one after the other, with the output:

[root@server2 ~]# cat /var/log/httpd/access_ssl.log | awk '{print $1}' | sort | uniq -c | sort -nr | head>

[root@server2 ~]#

Happy counting, err reporting!

The things Miguel does on a usual day

In a month or so he'll be too big to fit in his laundry basket. He has his eye on the bigger hamper in our room.

He runs around and does the "happy feet". He also likes running while on top of the air bed, but I can't get any video of that yet as I'm too busy going after him lest he bounce too hard and hit the wall.

And he demands his time for "computer work". :)

Google Apps Certified Deployment Specialist

Back in November last year, the Apps Services Partner that I help with Google Apps Support told me that I was required to take the Google Ap...