Setup Graylog2 Log Analyzer to Store and Search Log Errors

September 21, 2016 | By in UBUNTU HOWTO
| 2 Replies More

Graylog2 is an open-source log analyzer tool that makes use of MongoDB and ElasticSearch for storing and searching through log errors. It’s mainly used by developers to detect and fix errors in their applications.  Graylog is a more finished and "enterprise-ready" product out of the box as compared with other log analyzer tools.

Versions used:

Oracle Java: 1.8.0_101
Graylog: 2.0.3
Elasticsearch: 2.3.3
MongoDB:  3.2.9

1. Prerequisite

The installation described in this tutorial requires Ubuntu 16 with at least 4GB of RAM. If your system is constrained by RAM size then you can consider adding swap memory of your server.

Let us start by updating and upgrading the Ubuntu 16.04 ( Xenial Xerus )

root@ip-172-31-18-24:~# apt-get update && apt-get upgrade

Next configure FQDN of the server by updating /etc/hostname and /etc/hosts . We will choose hostname as graylog2 and domain name as . You can choose these two values according to your choice.

root@ip-172-31-18-24:~# vi /etc/hostname

Update /etc/hosts

root@ip-172-31-18-24:~# vi /etc/hosts localhost graylog2

If you are using any cloud based services then make sure you have changed the value of preserve_hostname to true from false.

root@ip-172-31-18-24:~# vim /etc/cloud/cloud.cfg
preserve_hostname: true

Reboot the server to apply changes.

root@ip-172-31-18-24:~# reboot

After rebooting, check the fully qualified domain name of your server.

ubuntu@graylog2:~$ hostname
ubuntu@graylog2:~$ hostname -f

2. Instal JDK

Elasticsearch needs Java so we will install oracle Java 8 since this is recommended by Elastic. However, it works well with OpenJDK also.

Add the private package archive webupd8team/java

root@graylog2:~# sudo add-apt-repository ppa:webupd8team/java
root@graylog2:~# sudo apt-get update

Install Oracle Java8

root@graylog2:~# sudo apt-get install oracle-java8-installer

Accept license while installing the above and  setup Oracle Java8 to be the default JVM

root@graylog2:~# sudo apt-get install oracle-java8-set-default

Now check the version of JAVA

root@graylog2:~# java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

3. Install Elasticsearch

Start installing elasticsearch by getting the GPG signing key.

root@graylog2:~# wget -qO - | sudo apt-key add -

Add Eleasticsearch repository to the apt database by executing the following command.

root@graylog2:~#  echo "deb stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch.list
deb stable main

Update apt database cache and install Elasticsearch

root@graylog2:~# apt-get update && sudo apt-get install elasticsearch

Make Elasticsearch to start automatically on system startup.

root@graylog2:~# systemctl enable elasticsearch

While configuring elasticsearch it is important to set a cluster name such as  "linoxide" .  The only other two parameters that we will add are and . Add the following parameter in the configuration file of Elasticsearch and save it.

root@graylog2:~# vi /etc/elasticsearch/elasticsearch.yml linoxide [""]

Restart the Elasticsearch service to read the new configurations.

root@graylog2:~# service elasticsearch restart

Wait for few seconds to let the Elasticsearch get fully restarted. Elastisearch listens on port 9200 for processing HTTP request. Test elasticsearch using CURL.

root@graylog2:~#  curl -X GET http://localhost:9200
"name" : "Royal Roy",
"cluster_name" : "linoxide",
"version" : {
"number" : "2.4.0",
"build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55",
"build_timestamp" : "2016-08-29T09:14:17Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
"tagline" : "You Know, for Search"

Ensure that cluster name shows above as "linoxide"

Now test the health of Elasticsearch cluster using CURL.

root@graylog2:~#  curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
"cluster_name" : "linoxide",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0

The above output should show the status as "green"

4. Install MongoDB

Get started installing MongoDB by importing the public key

root@graylog2:~# apt-key adv --keyserver hkp:// --recv EA312927
Executing: /tmp/tmp.iUI8C11LIe/ --keyserver hkp:// --recv
gpg: requesting key EA312927 from hkp server
gpg: key EA312927: public key "MongoDB 3.2 Release Signing Key <>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

Add mongodb repository in the apt database by creating the /etc/apt/sources.list.d/mongodb-org.list file using following command.

root@graylog2:~#  echo "deb wheezy/mongodb-org/3.2 main" | sudo tee /etc/apt/sources.list.d/mongodb-org.list
deb wheezy/mongodb-org/3.2 main

Now install MongoDB using the following command.

root@graylog2:~#  sudo apt-get update
root@graylog2:~#  sudo apt-get install mongodb-org

Start MongoDB using any of the following command.

root@graylog2:~# systemctl start mongod
root@demohost:/etc/init.d# /etc/init.d/mongod start
* Starting database mongod                                                                                                 [ OK ]

Finally enable it during system start-up.

root@graylog2:~# systemctl enable mongod
mongod.service is not a native service, redirecting to systemd-sysv-install
Executing /lib/systemd/systemd-sysv-install enable mongod

5. Install Graylog

To install graylog2, download and Install graylog 2.x repository.

root@graylog2:~# wget
root@graylog2:~# dpkg -i graylog-2.0-repository_latest.deb

HTTPS support for apt comes with pre-installed in Ubuntu 16.04 , if it is missing then install it and update the apt database.

root@graylog2:~# sudo apt-get install apt-transport-https
root@graylog2:~# apt-get update

Now install Graylog server using the following command.

root@graylog2:~# sudo apt-get install  graylog-server

While configuring Graylog2 server, we need to provide password for root user (admin) and a secret for securing user's password. Therefore we need to install password generator to generate password for us. Install pwgen using the following command. You need the admin password to login into the web interface of Graylog2. Remember, You cannot change the admin password using web interface. you need to edit this variable manually to change the admin's password.

root@graylog2:~# apt-get install pwgen

First generate the secret one.

root@graylog2:~# pwgen -N 1 -s 96

Now generate password for root user i.e admin

root@graylog2:~# echo -n password.123 | sha256sum
90dd9a873ed29902c543fe5cbb0a01268e7a7adadfc91bb135e800e1260f5cb2  -

The main part of configuration of graylog2 server is configuring server.conf inside /etc/graylog/server/

root@graylog2:~# vim /etc/graylog/server/server.conf

We will start by setting the admin password and secret key in /etc/graylog/server/server.conf . Paste the above two passwords in password_secret and root_password_sha2 parameter respectively.

password_secret = GSaULswcGz31ZCdd7aKhLNNx1aIflUSItH8TS1mY2Vnl8r4IkKTQKV4T9Jw3C1Jzmo7Jd1R1oqmRBavncIP8ExAqtijnfA68
root_password_sha2 = 90dd9a873ed29902c543fe5cbb0a01268e7a7adadfc91bb135e800e1260f5cb2

The following URI will be used to receive messages and must be accessible for all collectors.

rest_listen_uri =

The following URI is REST API transport address. Defaults to the value of rest_listen_uri

rest_transport_uri =

Web interface listen URI, this will be used to access the graylog interface in the browser.

web_listen_uri =

elasticsearch_shards defines the number of nodes in the Elasticsearch cluster, we have only one node, set it value as 1.

elasticsearch_shards = 1

This the number of replicas for your indices, we have only one node in cluster. set its value as 0.

elasticsearch_replicas = 0

Elasticsearch cluster name that you have set in configuring elasticsearch.

elasticsearch_cluster_name = linoxide

Graylog server will try to find the Elasticsearch nodes automatically using multicast mode. But for larger network it is recommended to use unicast mode which is best suited for production.

elasticsearch_discovery_zen_ping_unicast_hosts =

Disable multicast

elasticsearch_discovery_zen_ping_multicast_enabled = false

Bind addresses for the Elasticsearch client in Graylog. You can skip this step letting Elasticsearch choose these values automatically.

elasticsearch_network_host =
elasticsearch_network_bind_host =
elasticsearch_network_publish_host =

Restart Graylog2 and wait for few seconds to start it fully.

root@graylog2:~# systemctl daemon-reload
root@graylog2:~# systemctl restart graylog-server
root@demohost:~# /etc/init.d/graylog-server  start

6. Configure Firewall

Open TCP port no 9000 ( Graylog web interface ) and UDP port no 514 ( Rsyslog ) to allow traffic to Graylog2 web interface and Rsyslog respectively.

For IPTABLES users

root@demohost:~# vi /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 9000 -j ACCEPT
-A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT
root@demohost:~# iptables-save > /etc/iptables/rules.v4
root@demohost:~# service iptables-persistent restart

For UFW users

root@demohost:~# sudo ufw allow 9000/tcp
root@demohost:~# sudo ufw allow 514/udp
root@demohost:~# sudo ufw reload

7. Configure Rsyslog

Rsyslog comes along with ubuntu, therefore you don't have to install it. List this package in your server using the following command.

root@graylog2:~# dpkg -l rsyslog

If the above command returns a blank output then install rsyslog by executing following command in the terminal.

root@graylog2:~# apt-get install rsyslog
root@graylog2:~# systemctl start rsyslog
root@graylog2:~# netstat -alnp |grep 514
tcp 0 0* LISTEN 13549/rsyslogd
tcp6 0 0 :::514 :::* LISTEN 13549/rsyslogd

Once we are sure that rsyslog is running in the Graylog2 server, let us configure it so that Rsyslog sends the log to Graylog2.

Edit /etc/rsyslog.conf and remove the comments from the following lines.

root@graylog2:~# vi /etc/rsyslog.conf

module(load="imuxsock") # provides support for local system logging
module(load="imklog")   # provides kernel logging support
module(load="immark")  # provides --MARK-- message capability

# provides UDP syslog reception
input(type="imudp" port="514")

# provides TCP syslog reception
input(type="imtcp" port="514")

Configure syslog

Next edit /etc/rsyslog.d/50-default.conf and comment the following standard log files to disable the local logs.

#auth,authpriv.* /var/log/auth.log
#*.*;auth,authpriv.none -/var/log/syslog
#cron.* /var/log/cron.log
#daemon.* -/var/log/daemon.log
#kern.* -/var/log/kern.log
#lpr.* -/var/log/lpr.log
#mail.* -/var/log/mail.log
#user.* -/var/log/user.log

Now Add the following line at the end of the file so that rsyslog sends log data to the graylog server's port no 5140.


Configure default rsyslog

Next create a template file for rsyslog by the name 90-graylog2.conf and add the following information.

root@graylog2:~# vi /etc/rsyslog.d/90-graylog2.conf

$PreserveFQDN on

Create template for syslog

Restart Rsyslog

root@graylog2:~# systemctl restart rsyslog

Before accessing Graylog2 web interface, make sure every processes like elasticsearch, mongoDB, Graylog server, its web interface has started. Check it using netstat command.

root@graylog2:~# netstat -pltn

netstat command

8. Access Graylog web interface

Open your favourite browser and point it to http://YOUR-FQDN:9000. We will point the browser to

Login to Graylog

Login as admin and password that you have configured in step 5

Check elasticsearch cluster

Click System->Overview and make sure that elasticsearch cluster as green.

Graylog launch syslog udp input

Click System->Input, Select Input as "Syslog UDP" and click "Launch new input". Give a title name, bind address as either or, port no as 5140 that we have defined in /etc/rsyslog.conf while configuring Rsyslog in step 7. Click "Save" at the bottom.

System input running

Once saved, make sure it is running.

Now to test Graylog2, we will make use of logger which is a part of the util-linux package to send automated log data to the Graylog server's UDP port no 514.

thegeek@demohost:~$ logger --server --port 514 test Final message from VULTR123

Send more automated log data like above.

Click show messagesOnce you have sent few log data, Network IO in Throughput section will show the size of the log data till received. Click "Show received messages"

Show received messages

We have successfully able to send log data to Graylog2 in Ubuntu 16


Graylog is more targeted towards developers than other open source log management tools. If you want a strong alerting function in your log analyzer tool then consider using Graylog . It can handle an extensive range of data formats through its simple interface using REST API. As compared with other log analyzer tools Graylog is not management friendly in the dashboard front and its reporting functionalities are also a bit lacking. Since Graylog is an open source project, we can expect these drawbacks to be removed in future releases.


Share This :

Free Linux Ebook to Download

Comments (2)

Trackback URL | Comments RSS Feed

  1. Webuser says:

    logger --server --port 514 test Final message from VULTR123
    logger: getaddrinfo Name or service not known

  2. John says:

    How to send apache logs to graylog

Leave a Reply

Commenting Policy:
Promotion of your products ? Comment gets deleted.
All comments are subject to moderation.