In this guide, you will learn to install Elastic stack on Ubuntu 18.04. Elastic stack, formerly known as ELK stack is a collection or stack of free and opensource software from Elastic Company designed for centralized logging.

在本指南中，您将学习在Ubuntu 18.04上安装Elastic stack。 Elastic stack （以前称为ELK堆栈）是Elastic Company的用于集中式日志记录的免费或开源软件的集合或堆栈。

It enables the searching, analyzing and visualization of logs from different sources in a myriad of formats. Centralized logging helps in identification of server or application issues from a common point.

它可以以多种格式搜索，分析和可视化来自不同来源的日志。 集中式日志记录有助于从公共角度识别服务器或应用程序问题。

弹性堆叠组件 (Elastic Stack Components)

Elastic Stack comprises of 4 main components.

弹性堆栈包括4个主要组件。

1. Elasticsearch: This is a RESTful search engine that stores or holds all of the collected data
   Elasticsearch ：这是一个RESTful搜索引擎，用于存储或保存所有收集的数据
2. Logstash: This is the component that processes the data and parses it to elastic search
   Logstash ：这是处理数据并将其解析为弹性搜索的组件
3. Kibana: This is a web interface that visualizes logs
   Kibana ：这是一个可视化日志的Web界面
4. Beats: These are lightweight data shippers that ship logs from hundreds/thousands of servers to the central server on which ELK is configured.
   Beats ：这些是轻量级的数据传送器，可将日志从成百上千的服务器传送到配置了ELK的中央服务器。

Let’s now see how you can install the Elastic stack on Ubuntu 18.04.

现在让我们看看如何在Ubuntu 18.04上安装Elastic Stack。

先决条件 (Prerequisites)

Before you begin the installation ensure you should have the following infrastructure.

在开始安装之前，请确保您具有以下基础结构。

With that said, let’s dive in and begin the installation of the Elastic stack on Ubuntu.

话虽如此，让我们深入研究并开始在Ubuntu上安装Elastic stack。

1.在Ubuntu上安装Elasticsearch (1. Install Elasticsearch on Ubuntu)

First off, we are going to import Elasticsearch’s public GPG key into APT. Elastic stack packages are usually signed with Elasticsearch signing key to protect your system against package spoofing. In addition, authenticated packages are considered trusted by the package manager.

首先，我们将把Elasticsearch的公共GPG密钥导入APT。 Elastic Stack软件包通常使用Elasticsearch签名密钥进行签名，以保护您的系统免受软件包欺骗的侵害。 另外，通过身份验证的程序包被程序包管理器认为是受信任的。

To import the GPG key run:

要导入GPG密钥，请运行：

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Next, add Elastic repository to the sources.list.d directory using the command below.

接下来，使用以下命令将Elastic存储库添加到sources.list.d目录。

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

The output of the two commands is as shown:

这两个命令的输出如下所示：

Output

输出量

Now update the system’s repository using the command below.

现在，使用以下命令更新系统的存储库。

sudo apt update

Sample Output

样本输出

Now, install Elasticsearch using the command below.

现在，使用以下命令安装Elasticsearch。

sudo apt install elasticsearch

Output

输出量

2.在Ubuntu上配置Elasticsearch (2. Configure Elasticsearch on Ubuntu)

Elasticsearch listens on port 9200. However, we are going to limit outside access so that outside parties cannot access data and shut down the elastic cluster. That said, we are going to make a few modifications to the Elasticsearch configuration file as shown below

Elasticsearch侦听端口9200。但是，我们将限制外部访问，以使外部方无法访问数据并关闭弹性集群。 也就是说，我们将对Elasticsearch配置文件进行一些修改，如下所示

sudo nano /etc/elasticsearch/elasticsearch.yml

Find the network.host attribute and uncomment it and add localhost as its value. Also uncomment the http.port attribute.

查找network.host属性，然后取消注释，并添加localhost作为其值。 还要取消注释http.port属性。

Output

输出量

network.host: localhosthttp.port: 9200

Next, start and enable Elasticsearch service as shown.

接下来，启动并启用Elasticsearch服务，如图所示。

sudo systemctl start elasticsearchsudo systemctl enable elasticsearch

Output

输出量

At this point, Elasticsearch should be up and running. You can verify this by running the command below.

此时，Elasticsearch应该已启动并正在运行。 您可以通过运行以下命令来验证这一点。

systemctl status elasticsearch

Output

输出量

You can also use the netstat command as shown.

您也可以使用netstat命令，如图所示。

netstat -pnltu

Also, you can run the curl command as shown.

另外，您可以运行所示的curl命令。

curl -X GET "localhost:9200"

Output

输出量

Great! We have finalized the installation and configuration of Elasticsearch. Next, we are going to install and configure Logstash.

大！ 我们已经完成了Elasticsearch的安装和配置。 接下来，我们将安装和配置Logstash。

3.安装和配置Logstash (3. Installing and configuring Logstash)

The second component of Elastic stack that we are going to install is Logstash. Logstash will be responsible for collecting and centralizing logs from various servers using filebeat data shipper. It will then filter and relay syslog data to Elasticsearch.

我们将要安装的Elastic stack的第二个组件是Logstash。 Logstash将负责使用filebeat数据发送程序从各种服务器收集和集中日志。 然后它将过滤系统日志数据并将其中继到Elasticsearch。

First, Let’s confirm that OpenSSL is running. To do that, run.

首先，让我们确认OpenSSL正在运行。 为此，请运行。

openssl version -a

Output

输出量

To install Logstash, run the command below.

要安装Logstash，请运行以下命令。

sudo apt install logstash -y

Output

输出量

Next, edit the /etc/hosts file and append the following.

接下来，编辑/etc/hosts文件并添加以下内容。

18.224.44.11  elk-master

Where 18.224.44.11 is the IP address of the masterELk server.

其中18.224.44.11是masterELk服务器的IP地址。

We are then going to generate the SSL certificate key to secure the log data transfer from the client filebeat to the logstash server.

然后，我们将生成SSL证书密钥，以保护从客户端文件拍到Logstash服务器的日志数据传输安全。

To do this, first, create a new SSL directory under the logstash configuration directory ‘/etc/logstash’ and navigate into that directory.

为此，首先，在logstash配置目录“ / etc / logstash”下创建一个新的SSL目录，并导航到该目录。

mkdir -p /etc/logstash/sslcd /etc/logstash/

Now you can generate the SSL certificate as shown below.

现在，您可以生成SSL证书，如下所示。

openssl req -subj '/CN=elk-master/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout ssl/logstash-forwarder.key -out ssl/logstash-forwarder.crt

Next, we are going to create new configuration files for logstash. We will create a configuration file ‘filebeat-input.conf’ as input file from filebeat, ‘syslog-filter.conf’ for syslog processing, and lastly a ‘output-elasticsearch.conf’ file to define the Elasticsearch output.

接下来，我们将为logstash创建新的配置文件。 我们将创建一个配置文件'filebeat-input.conf'作为来自filebeat的输入文件，创建'syslog-filter.conf'用于进行syslog处理，最后创建一个'output-elasticsearch.conf'文件来定义Elasticsearch输出。

Navigate to Logstash directory and create a ‘filebeat-input.conf’ in the ‘conf.d’ directory.

导航到Logstash目录，然后在“ conf.d”目录中创建“ filebeat-input.conf”。

cd /etc/logstash/vim conf.d/filebeat-input.conf

Paste the following configuration.

粘贴以下配置。

input {  beats {    port => 5443    type => syslog    ssl => true    ssl_certificate => "/etc/logstash/ssl/logstash-forwarder.crt"    ssl_key => "/etc/logstash/ssl/logstash-forwarder.key"  }}

Save and exit the text editor.

保存并退出文本编辑器。

For the syslog processing log data, we are using the filter plugin named ‘grok’ for parsing of the syslog files.

对于处理日志数据的系统日志，我们使用名为“ grok”的过滤器插件来解析系统日志文件。

Create a new configuration ‘syslog-filter.conf’.

创建一个新的配置'syslog-filter.conf'。

vim conf.d/syslog-filter.conf

Paste the configuration below.

粘贴下面的配置。

filter {  if [type] == "syslog" {    grok {      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }      add_field => [ "received_at", "%{@timestamp}" ]      add_field => [ "received_from", "%{host}" ]    }    date {      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]    }  }}

Save and exit the text editor.

保存并退出文本编辑器。

Finally, create a configuration file named ‘output-elasticsearch.conf’ for elasticsearch output.

最后，为弹性搜索输出创建一个名为“ output-elasticsearch.conf”的配置文件。

vim conf.d/output-elasticsearch.conf

Paste the following content.

粘贴以下内容。

output {  elasticsearch { hosts => ["localhost:9200"]    hosts => "localhost:9200"    manage_template => false    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"    document_type => "%{[@metadata][type]}"  }}

Save and exit the text editor.

保存并退出文本编辑器。

When that is said and done, enable and start the Logstash service.

完成上述步骤后，启用并启动Logstash服务。

sudo systemctl enable logstashsudo systemctl start logstash

To verify that Logstash is running, run the command.

要验证Logstash是否正在运行，请运行命令。

sudo systemctl status logstash

Sample Output

样本输出

You can also use the netstat command as shown.

您也可以使用netstat命令，如图所示。

netstat -pnltu

4.在Ubuntu上安装和配置Kibana (4. Install and configure Kibana on Ubuntu)

Next, we are going to install Kibana using the command below.

接下来，我们将使用以下命令安装Kibana。

sudo apt install kibana -y

Output

输出量

Next, we are going to make a few modifications to the kibana configuration file.

接下来，我们将对kibana配置文件进行一些修改。

vim /etc/kibana/kibana.yml

Locate and uncomment the following attributes.

找到并取消注释以下属性。

server.port: 5601server.host: "localhost"elasticsearch.url: "https://localhost:9200"

Save and exit the text editor.

保存并退出文本编辑器。

Then enable and start the Kibana service:

然后启用并启动Kibana服务：

sudo systemctl enable kibanasudo systemctl start kibana

Output

输出量

You can confirm that kibana is running on it default port 5601 using the netstat command as shown.

您可以使用netstat命令确认kibana是否在默认端口5601上运行。

netstat -pnltu

Output

输出量

5.安装和配置NGINX作为Kibana的反向代理 (5. Installing and configuring NGINX as a reverse proxy for Kibana)

We are using NGINX as a reverse proxy to kibana dashboards. You need to install Nginx and ‘Apache2-utils’ as shown below.

我们将NGINX用作kibana仪表板的反向代理。 您需要安装Nginx和“ Apache2-utils”，如下所示。

sudo apt install nginx apache2-utils -y

Output

输出量

Next, create a new virtual host file named kibana.

接下来，创建一个名为kibana的新虚拟主机文件。

vim /etc/nginx/vim sites-available/kibana

Paste the following content into the virtual host file

将以下内容粘贴到虚拟主机文件中

server {    listen 80;     server_name localhost;     auth_basic "Restricted Access";    auth_basic_user_file /etc/nginx/.kibana-user;     location / {        proxy_pass https://localhost:5601;        proxy_http_version 1.1;        proxy_set_header Upgrade $http_upgrade;        proxy_set_header Connection 'upgrade';        proxy_set_header Host $host;        proxy_cache_bypass $http_upgrade;    }}

Save and exit the text editor. Recommended Read: .

保存并退出文本编辑器。 推荐阅读： 。

Next, create a basic authentication for the kibana dashboard using the htpasswd command as shown.

接下来，使用htpasswd命令为kibana仪表板创建基本身份验证，如图所示。

sudo htpasswd -c /etc/nginx/.kibana-user elasticType the elastic user password

Output

输出量

In the above example, the username is elastic and the password will be what you provide.

在上面的示例中，用户名是弹性的 ，密码将由您提供。

Next, activate the Kibana virtual host configuration and test Nginx configuration.

接下来，激活Kibana虚拟主机配置并测试Nginx配置。

ln -s /etc/nginx/sites-available/kibana /etc/nginx/sites-enabled/nginx -t

Output

输出量

With no errors, enable and restart Nginx server.

没有错误，启用并重新启动Nginx服务器。

systemctl enable nginxsystemctl restart nginx

6.安装和配置Filebeat (6. Installing and Configuring Filebeat)

In this step, we are going to configure filebeat data shipper on our elk-master server. This will relay all the syslog messages to logstash which will get processed and visualized by kibana.

在此步骤中，我们将在elk-master服务器上配置filebeat数据发送程序。 这会将所有系统日志消息中继到logstash，后者将由kibana处理并可视化。

To install filebeat run:

要安装filebeat，请运行：

sudo apt install filebeat

Next, open the filebeat configuration file.

接下来，打开filebeat配置文件。

sudo vim /etc/filebeat/filebeat.yml

We are going to use Logstash to perform additional processing on the data collected by Filebeat. Filebeat will not be needed to send any data directly to Elasticsearch. Therefore, locate and Comment the elasticsearch section as shown.

我们将使用Logstash对Filebeat收集的数据执行其他处理。 不需要Filebeat将任何数据直接发送到Elasticsearch。 因此，如图所示，找到elasticsearch部分并对其进行注释。

#output.elasticsearch:  # Array of hosts to connect to.  #hosts: ["localhost:9200"]

Next, head out to the Logstash section and uncomment as shown.

接下来，前往Logstash部分并取消注释，如图所示。

output.logstash:  # The Logstash hosts  hosts: ["localhost:5044"]

Enable the filebeat prospectors by changing the ‘enabled’ line value to ‘true’.

通过将“ enabled”行值更改为“ true”来启用filebeat探矿者。

enabled: true

Specify the system log files to be sent to the logstash server. In this example, we will add the ssh log file ‘auth.log’ and the syslog file.

指定要发送到logstash服务器的系统日志文件。 在此示例中，我们将添加ssh日志文件'auth.log'和syslog文件。

paths:    - /var/log/auth.log    - /var/log/syslog

Save and Exit.

保存并退出。

Finally, copy the logstash certificate file – logstash-forwarder.crt – to /etc/filebeat directory.

最后，将logstash证书文件logstash-forwarder.crt复制到/etc/filebeat目录。

cp ~/logstash-forwarder.crt /etc/filebeat/logstash-forwarder.crt

Now start and enable filebeat.

现在启动并启用filebeat。

systemctl start filebeatsystemctl enable filebeat

To check the status of filebeat run:

要检查filebeat的状态，请运行：

systemctl status filebeat

Output

输出量

7.测试Elasticsearch Stack (7. Testing Elasticsearch Stack)

To test our Elastic stack, Open your browser and browse your server’s IP followed by port 5601 which is the port kibana listens to.

要测试我们的弹性堆栈，请打开浏览器并浏览服务器的IP，然后浏览端口5601，这是kibana侦听的端口。

ip-address:5601

Enter the username and password and later, the following screen will be displayed.

输入用户名和密码，稍后将显示以下屏幕。

Click on the ‘discover’ tab and click on ‘Filebeat’ The following interface will appear giving you live streaming of visualized data.

单击“发现”选项卡，然后单击“ Filebeat”。将出现以下界面，为您提供实时可视化数据流。

Congratulations! You have successfully installed and configured The Elastic Stack and the Elastic Beat ‘Filebeat’ on your Ubuntu 18.04 system.

恭喜你！ 您已在Ubuntu 18.04系统上成功安装并配置了Elastic Stack和Elastic Beat'Filebeat'。

翻译自: