Squid is one of the most popular proxy software in Linux world. It’s powerful, it has a lot of features and of course, it’s free. Squid has been used by ISP’s since the early 1990 to provide faster download speeds and reduce latency. For common proxy use, I believe squid is implemented in most companies around the world.
Squid provides access.log to record all user activities which through it. IT Administrator can parse the file to see what happens there. But access.log is a raw file. You really to read it carefully to get valuable information. Since access.log file is a raw file, a third party software is needed to process it into a human readable information.
There are a lot of squid-parser software out there. On the Squid-Cache website, there are more than 20 software for squid log-analysis. If you are confused of which is suitable for your needs, we will show you one of squid log-analysis software called Squid-Analyzer.
What is Squid-Analyzer
Squid Analyzer parses Squid proxy access log and reports general statistics about hits, bytes, users, networks, top URLs, and top second level domains. Statistic reports are oriented toward user and bandwidth control.
As an Analyzer, this application has a great graphical reports. You will be surprised if you compare Squid Analyzer with other applications. Let’s start to install it.
Download the source
The latest version of Squid Analyzer now is version 5.2. You can download it from here. After that, you can extract the source file.
tar zxcf squidanalyzer-5.2.tar.gz
Install the software
Squid Analyzer is depends on Perl software. You need to make sure that you have it on your system. To check it whether your system have Perl or not, type :
You will see Perl version as the output. If you don’t see it, you need to install Perl first.
On Debian / Ubuntu based, you can use this command :
sudo apt-get install perl
On RedHat / CentOS based, you can use this command :
sudo yum install perl
Once the Perl is installed, then you can continue to install Squid Analyzer. Go to the folder where the Squid Analyzer file is stored. Assume that the Squid Analyzer is located in folder /home/squidanalyzer-5.2/ , then type this command to install it (you may need to be root):
Configuring the web server
Squid Analyzer is a web based tool. So we need to configure our web server to recognize it. If you are using Apache web server, here’s the minimal configuration to make Squid Analyzer run.
Alias /squidreport /var/www/squidanalyzer
Options -Indexes FollowSymLinks MultiViews
Deny from all
Allow from 127.0.0.1
Don’t forget to restart the Apache web server to take effect.
Configuring Squid Analyzer
Squid Analyzer has a it’s own configuration file. By default, you don’t have to change anything. But if you want to custom it, the file is located in /etc/squidanalyzer/squidanalyzer.conf
Test it Running
If you found no error at installation stage, browse to http://myhost.com/squidreports
You will be taken to the front page of Squid Analyzer
Creating Cron Job
Squid log files are almost never stopping running. The log content will grow from time to time. To parse the log files, we need to run Squid Analyzer periodically. Cron is best tool to do it. Here’s a sample to run it for everyday at 02:00 AM
Inside Squid Analyzer
Squid Analyzer front page is quite simple. You will see Global Statistics which grouped by Years. Click the Years you want to see. Then you will be taken to the Cache Statistics of that Year. Assume we will use year 2013.
There will be a calendar 2013 at the top right area. The months which are covered by Squid Analyzer can be clicked.
Let say we click on Oct month then you will see beautiful graphs about Daily Requests statistics and Daily Mega Bytes statistics on October. The calender at top right is also changed into a daily view.
To view graphs in day-to-day view, click on the date on the calender.
Squid Analyzer have 5 menus. There are Mime Types, Networks, Users, Top URLs, and Top Domains.
Here are the details :
This menu will only show you a list of MIME-Types which is recognized by Squid Analyzer. Just for information, from MIME is an Internet media type, a standard identifier used on Internet to indicate the type of data that a file contains.
As you can guess from it’s name, Networks will give you information about network statistics.
It will give you detail information from IP Addresses to the accessed URL. The interesting part is you can select the IP Address there to view information more detail related to that IP Address. When you click it, Squid Analyzer will show you information about this :
- A graph shows daily requests based on amount of hit and date
- A graph shows daily megabyte transferred based on date
- A list of IP Address in the same subnet who used the proxy
Once again, you can click the list of IP Addresses there to view more details.
On Users menu, you will see a list of users that accessed the proxy. If you see line such as “User Statistis on 2013-10” it’s mean that you are seeing a list of Users on October 2013.
If your Squid proxy is using proxy-authentication such as LDAP, you will see some of users are displayed by name. You can also click at the IP Addresses or names on that list to view information more detail.
You will see the most accessed URLs in a year or a specific month. If you hover your mouse cursor on top the URL, you will see a box pop-up. It contains the information about who access that URL and how many times the users access it.
If you click on the URL, you will be taken to it’s website.
Top Domains is similar with Top URLs. The difference is it just show you the domains. If you access http://www.youtube.com URL, Top Domains just display *.youtube.com
But in Top Domains, you will see pie-chart graphs.
Below those graphs, you will see a list of Top Domains. Similar with Top URL, you can also hover you mouse cursor on top the domains to see a quick view of Users and Count.
Squid Analyzer may not the best Squid access.log parser. But it gives you a useful information about what happen in your proxy. And it’s Open Source and free. It got 4,9 stars of 5 stars from Sourceforge.net. You may explore Squid Analyzer in more detail from SquidAnalyzer website.