Skip to main content

How to count instances of an IP address in a log file

I was looking for a quick and painless way to determine what IP addresses used our web app the most and stumbled across this excellent post. I got reminded of the power of the command-line interface and that often, I underestimate the way these commands are supposed to make a sysad's life easier.


So here's a quick tip for counting unique visitors to your web app. Locate your web app's log file and do a quick scan of the last few lines. In my case, this is what I saw:


112.198.79.223 - - [10/Jul/2010:00:57:29 -0400] "POST /v2/api/execute.php?method=upload_time&ver=win1.1.15 HTTP/1.1" 200 116 "-" "Mozilla/5.0"
117.241.112.254 - - [10/Jul/2010:00:57:30 -0400] "POST /v2/api/execute.php?method=timestat&ver=win1.1.15 HTTP/1.1" 200 288 "-" "Mozilla/5.0"
80.249.84.105 - - [10/Jul/2010:00:57:29 -0400] "POST /v2/api/execute.php?method=offtime&ver=win1.1.15 HTTP/1.1" 200 19 "-" "Mozilla/5.0"
117.241.112.254 - - [10/Jul/2010:00:57:32 -0400] "POST /v2/api/execute.php?method=get_defaults&ver=win1.1.15 HTTP/1.1" 200 77 "-" "Mozilla/5.0"


In this log format, the IP address appears first, so we will only need to get the first set of characters, separated by a space. In order to count instances of an IP address, we sort the IP addresses so similar IP addresses are grouped together, then count them. Then we sort them again so that the sums are arranged. The number of unique IPs may give you pages of standard output, so optionally, if you only want to see which IP addresses accessed the web app the most, we look at the top results only.


So these are the commands used, piped one after the other, with the output:


[root@server2 ~]# cat /var/log/httpd/access_ssl.log | awk '{print $1}' | sort | uniq -c | sort -nr | head>


71612 216.157.78.237
34094 120.28.205.244
30091 119.93.97.54
25078 120.28.214.104
18949 120.28.195.151
17929 58.107.64.40
14877 212.98.174.235
14752 117.241.112.143
14429 120.28.247.74
12864 117.241.113.183
[root@server2 ~]#


Happy counting, err reporting!

Comments

Popular posts from this blog

Family Visits Singapura

My parents and in-laws traveled to Singapore last September and it was truly memorable! I'm trying out some code to display a Google Photos album and it's quite cool that I'm able to embed the album here. Credits to Pavex for the great tool. He details out how to use it here . Check out the album below!
March was a good month. Not just because it's my birthday month, but I've been able to do more physical exercises as far as I'm allowed, and I've also been able to get through my first month back in the job. For the first couple of weeks, I thought I was going to do 4 hours a day at most, and turn up the heat bit by bit until I'm able to do 100% again. But I suppose having to rest over a month does have its toll. It took significant effort to try not go all in with guns blazing on the first day back at work. LOL Anyway, I'm trying something new this year. We've migrated our applications to a better platform, and we'd like to push that to the cloud this year. It'll obviously need a lot of liaising and cooperative work with other teams and domains so I'm going to do more engagement and solutioning than the usual engineering this time. Last week, we did a platform-related hackathon and a critical refresh across clusters. There's going to be a lo

Being a Systems Administrator @37signals (Podcast)

 I use Basecamp a lot. Kudos to these guys for making my job as a Systems Administrator more bearable than usual. :p Episode #12: Being a Systems Administrator at 37signals Time: 22:50 | 04/13/2010 |  Download MP3  |  Link to this episode Mark, Joshua, and John on life as a 37signals Sys Admin The Sys Admin team discusses hosting the 37signals apps, working with programmers, helping support, telecommuting, dealing with vendors, improving speeds in Europe, and more. Related links: Rackspace Dell Definitions: Redundancy Facebook Just Gets It Right via  37signals.com