In this article, I'll explain how to install latest Splunk in on Ubuntu 18.04 server. Splunk is aimed to process the data to make it useful for the user without manipulating the original data. It is one of the most powerful tools for analyzing, exploring and searching data. It is one of the easiest way to index, search, collect and visualize massive data streams in real-time from the application, web servers, databases, server platforms, Cloud-networks and many more.
Splunk Architecture
There are three main components in Splunk as shown below:
- Splunk Forwarder
- Splunk Indexer
- Splunk Search head
As you can see Splunk Forwarder is used for data forwarding. It is the component which is used for collecting the logs. Splunk Indexer is the one used for Parsing and Indexing the data. Splunk instance transforms the incoming data into events and stores it in indexes for performing search operations efficiently. Lastly the Splunk Search Head which is a Graphical interface used for searching, analyzing and reporting.
Installing Splunk on Ubuntu 18.04
Create a Splunk account and download the Splunk Enterprise Software from their official website here.
Now upload the downloaded file to your Ubuntu 18.04 server and place it in a temporary directory. Next, we can run the dpkg command to install the Splunk server.
~# dpkg -i splunk-7.1.0-2e75b3406c5b-linux-2.6-amd64.deb Selecting previously unselected package splunk. (Reading database ... 66600 files and directories currently installed.) Preparing to unpack splunk-7.1.0-2e75b3406c5b-linux-2.6-amd64.deb ... Unpacking splunk (7.1.0) ... Setting up splunk (7.1.0) ... complete
Secondly, we need to create the init.d script so that we can easily start and stop Splunk. Change to the Splunk binary directory at /opt/splunk/bin/ and run Splunk executable with the below arguments.
#cd /opt/splunk/bin/ # ./splunk enable boot-start Splunk Software License Agreement 04.24.2018 Do you agree with this license? [y/n]: y Do you agree with this license? [y/n]: y This appears to be your first time running this version of Splunk. An Admin password must be set before installation proceeds. Password must contain at least: * 8 total printable ASCII character(s). Please enter a new password: Please confirm new password: Copying '/opt/splunk/etc/openldap/ldap.conf.default' to '/opt/splunk/etc/openldap/ldap.conf'. Generating RSA private key, 2048 bit long modulus .......+++ ................+++ e is 65537 (0x10001) writing RSA key Generating RSA private key, 2048 bit long modulus .............................................................+++ ............+++ e is 65537 (0x10001) writing RSA key Moving '/opt/splunk/share/splunk/search_mrsparkle/modules.new' to '/opt/splunk/share/splunk/search_mrsparkle/modules'. Init script installed at /etc/init.d/splunk. Init script is configured to run at boot.
During this process, you can Press the Spacebar to go through the license agreement and then type Y to accept it as shown in the installation logs. Finally, we can start the Splunk Service with the command below:
# service splunk start
Now you can access your Splunk Web interface at http://Server-IP:8000/ or http://Server-hostname:8000 . You need to make sure this port 8000 is open on your server firewall.
You can provide the admin login credentials created during the installation phase to access your Splunk GUI interface. Once you logged in, you will have your Splunk Dashboard ready to use.
There are different categories listed over on the home page. You can choose the required one and start Splunking.
Adding a task
I'm adding an example for a simple task which is been added to the Splunk system. Just see my snapshots to understand how I added it. My task is to add /var/log folder to the Splunk system for monitoring.
Step1 :
Open up the Splunk Web interface and choose the Add Data option to start with.
Step 2 :
The Add Data Tab opens up with three options: Upload, Monitor, and Forward. Each option is self-explanatory with a short description of the purpose. Here our task is to monitor a folder, so we go ahead with Monitor.
In the Monitor option, there are four categories as below:
Files & Directories: To monitor files/folders
HTTP Event Collector: Monitor data streams over HTTP
TCP/ UDP: Monitor traffic over the TCP/UDP ports
Scripts: Monitor Custom Scripts or Commands
Step 3 :
According to our purpose, I choose the Files & Directories option.
Step 4 :
Now, I'm browsing the exact folder path /var/log from the server to monitor. Once you select the settings, you can click Next and Review.
Once all your settings are reviewed, you can click 'Submit' to conclude.
Step 5 :
Now you have added successfully your data source to Splunk for monitoring. You can start searching and monitoring the log file as required. I have narrowed down the logs to Apache application on the server.
This is just a simple example for Splunking, you can add as many tasks to this and explore your local or remote server data. It also provides you with tools to create tables and visualizations using multiple fields and metrics depending on your log analysis.
Read also:
Splunk is an absolute admin solution for log processing. It's an ultimate log collection and analysis tool. I hope this article is informative and useful for you. I recommend your valuable suggestions and comments on this. Happy Splunking!!
