Netdata: Difference between revisions

From charlesreid1
 
(17 intermediate revisions by the same user not shown)
Line 3: Line 3:
=Installing=
=Installing=


==Prerequisites and dependencies==
{{Main|Netdata/Installing}}


<pre>
We recommend installing Netdata on Linux using the quick one-liner:
apt-get install -y zlib1g-dev uuid-dev libmnl-dev gcc make \
    autoconf autoconf-archive autogen automake pkg-config curl
 
apt-get install -y python python-yaml python-mysqldb \
    python-psycopg2 nodejs lm-sensors netcat
</pre>
 
==Installing Netdata on Linux==
 
Here's a quick one-liner to avoid checking out the github repository.


<pre>
<pre>
Line 21: Line 11:
</pre>
</pre>


==What It Do==
The [[Netdata/Installing]] page has more details about installation steps.
 
The script does the following:
* Create users/groups as needed
* Get logrotate
* Set permissions
* Download default netdata configuration
 
==Output from Install Process==
 
Here is the output from the installation process, which contains some useful information:
 
{{Scrollbox|
<pre>
--- Restore user edited netdata configuration files ---
--- Fix generated files permissions ---
[/usr/src/netdata.git]# find ./system/ -type f -a \! -name \*.in -a \! -name Makefile\* -a \! -name \*.conf -a \! -name \*.service -a \! -name \*.logrotate -exec chmod 755 \{\} \;                                       
OK 
 
--- Add user netdata to required user groups ---
Adding netdata user group ...
[/usr/src/netdata.git]# groupadd -r netdata
OK 
 
Adding netdata user account ...
[/usr/src/netdata.git]# useradd -r -g netdata -c netdata -s /usr/sbin/nologin -d / netdata
OK 
 
Group 'docker' does not exist.
Group 'nginx' does not exist.
Group 'varnish' does not exist.
Group 'haproxy' does not exist.
Adding netdata user to the adm group ...
[/usr/src/netdata.git]# usermod -a -G adm netdata
OK 
 
Group 'nsd' does not exist.
Adding netdata user to the proxy group ...
[/usr/src/netdata.git]# usermod -a -G proxy netdata
OK 
 
Group 'squid' does not exist.
--- Install logrotate configuration for netdata ---
[/usr/src/netdata.git]# cp system/netdata.logrotate /etc/logrotate.d/netdata
OK 
 
[/usr/src/netdata.git]# chmod 644 /etc/logrotate.d/netdata
OK 
 
--- Read installation options from netdata.conf ---
--- Fix permissions of netdata directories (using user 'netdata') ---
[/usr/src/netdata.git]# chown -R root:netdata /etc/netdata
OK 
 
[/usr/src/netdata.git]# find /etc/netdata -type f -exec chmod 0640 \{\} \;
OK 
 
[/usr/src/netdata.git]# find /etc/netdata -type d -exec chmod 0755 \{\} \;
OK 
 
[/usr/src/netdata.git]# chown -R netdata:netdata /usr/share/netdata/web
OK 
 
[/usr/src/netdata.git]# find /usr/share/netdata/web -type f -exec chmod 0664 \{\} \;
OK 
 
[/usr/src/netdata.git]# find /usr/share/netdata/web -type d -exec chmod 0775 \{\} \;
OK 
 
[/usr/src/netdata.git]# chown -R netdata:netdata /var/lib/netdata
OK 
 
[/usr/src/netdata.git]# chown -R netdata:netdata /var/cache/netdata
OK 
 
[/usr/src/netdata.git]# chown -R netdata:netdata /var/log/netdata
OK 
 
[/usr/src/netdata.git]# chmod 755 /var/log/netdata
OK 
 
[/usr/src/netdata.git]# chown netdata:root /var/log/netdata
OK 
 
[/usr/src/netdata.git]# chown -R root /usr/libexec/netdata
OK 
 
[/usr/src/netdata.git]# find /usr/libexec/netdata -type d -exec chmod 0755 \{\} \;
OK 
 
[/usr/src/netdata.git]# find /usr/libexec/netdata -type f -exec chmod 0644 \{\} \;
OK 
 
[/usr/src/netdata.git]# find /usr/libexec/netdata -type f -a -name \*.plugin -exec chmod 0755 \{\} \;
OK 
 
[/usr/src/netdata.git]# find /usr/libexec/netdata -type f -a -name \*.sh -exec chmod 0755 \{\} \;
OK 
 
[/usr/src/netdata.git]# chown root:netdata /usr/libexec/netdata/plugins.d/apps.plugin
OK 
 
[/usr/src/netdata.git]# chmod 0750 /usr/libexec/netdata/plugins.d/apps.plugin
OK 
 
[/usr/src/netdata.git]# setcap cap_dac_read_search\,cap_sys_ptrace+ep /usr/libexec/netdata/plugins.d/apps.plugin                                                                                                           
OK 
 
[/usr/src/netdata.git]# chown root:netdata /usr/libexec/netdata/plugins.d/cgroup-network
OK 
 
[/usr/src/netdata.git]# chmod 4750 /usr/libexec/netdata/plugins.d/cgroup-network
OK 
 
[/usr/src/netdata.git]# chown root /usr/libexec/netdata/plugins.d/cgroup-network-helper.sh
OK 
 
[/usr/src/netdata.git]# chmod 0550 /usr/libexec/netdata/plugins.d/cgroup-network-helper.sh
OK 
 
[/usr/src/netdata.git]# chmod a+rX /usr/libexec
OK 
 
[/usr/src/netdata.git]# chmod a+rX /usr/share/netdata
OK 
 
--- Install netdata at system init ---
Installing systemd service...
[/usr/src/netdata.git]# cp system/netdata.service /etc/systemd/system/netdata.service
OK 
 
[/usr/src/netdata.git]# systemctl daemon-reload
OK 
 
[/usr/src/netdata.git]# systemctl enable netdata
Created symlink /etc/systemd/system/multi-user.target.wants/netdata.service -> /etc/systemd/system/netdata.service.
OK 
 
--- Start netdata ---
[/usr/src/netdata.git]# /bin/systemctl stop netdata
OK 


[/usr/src/netdata.git]# /bin/systemctl restart netdata
=Netdata Web Dashboard=
OK 


OK. NetData Started!
Navigate to port 19999 on the server running netdata, and you'll see a slick live dashboard detailing the status of the server netdata is running on:


[[Image:NetdataDashboardExample.png|400px]]


-------------------------------------------------------------------------------
Netdata errs on the side of too much info, rather than too little. It is also designed to display the current state of the machine, it does not put any effort into archiving or data storage or memory other than a backend plugin to continuously export data.


Downloading default configuration from netdata...
=Database Backends=
[/usr/src/netdata.git]# curl -s -o /etc/netdata/netdata.conf.new http://localhost:19999/netdata.conf
OK 


[/usr/src/netdata.git]# mv /etc/netdata/netdata.conf.new /etc/netdata/netdata.conf
Here we are interested in dumping data from netdata to a database.
OK 


OK  New configuration saved for you to edit at /etc/netdata/netdata.conf
Netdata wiki page on backends: https://github.com/firehol/netdata/wiki/netdata-backends


[/usr/src/netdata.git]# chown netdata /etc/netdata/netdata.conf
==Backend API URLs==
OK 


[/usr/src/netdata.git]# chmod 0664 /etc/netdata/netdata.conf
We'll use this below, but check out how to use URLs to access the Netdata backend API:
OK 


--- Check KSM (kernel memory deduper) ---
http://192.168.25.236:19999/api/v1/allmetrics?format=json&help=yes


Memory de-duplication instructions
[[Image:NetdataBackendJSON.png|500px]]


You have kernel memory de-duper (called Kernel Same-page Merging,
http://192.168.25.236:19999/api/v1/allmetrics?format=prometheus&help=yes
or KSM) available, but it is not currently enabled.


To enable it run:
[[Image:NetdataBackendPrometheus.png|500px]]


    echo 1 >/sys/kernel/mm/ksm/run
This helpful information was pointed out by this guide to setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack
    echo 1000 >/sys/kernel/mm/ksm/sleep_millisecs


If you enable it, you will save 40-60% of netdata memory.
==Prometheus Backend==


--- Check version.txt ---
Following this guide to setting up Prometheus backend, which is on the Netdata wiki: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack
--- Check apps.plugin ---
--- Generate netdata-uninstaller.sh ---
--- Basic netdata instructions ---  


netdata by default listens on all IPs on port 19999,
[[Netdata/Prometheus]]
so you can access it with:


  http://this.machine.ip:19999/
==Json Backend==


To stop netdata run:
I had some trouble getting Netdata to work with MongoDB, a JSON database. I pointed Netdata to a particular port, and then set up a simple echo socket listener in Python, but saw no activity on the port.


  systemctl stop netdata
I tried this using the following <code>/etc/netdata/netdata.conf</code>:
 
To start netdata run:
 
  systemctl start netdata
 
 
Uninstall script generated: ./netdata-uninstaller.sh
Update script generated  : ./netdata-updater.sh
 
netdata-updater.sh can work from cron. It will trigger an email from cron
only if it fails (it does not print anything when it can update netdata).
--- Installing netdata-updater at cron ---
[/usr/src/netdata.git]# ln -s /usr/src/netdata.git/netdata-updater.sh /etc/cron.daily/netdata-updater
OK 
 
 
--- We are done! ---
 
  ^
  |.-.  .-.  .-.  .-.  .-.  .  netdata                          .-.  .-
  |  '-'  '-'  '-'  '-'  '-'  is installed and running now!  -'  '-' 
  +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->
 
  enjoy real-time performance and health monitoring...
 
OK 
</pre>
}}
 
=Usage=
 
==Netdata web dashboard==
 
To use netdata's web dashboard, open a browser and navigate to localhost:19999
 
You should now see a web dashboard with live statistics about the system.
 
Spend plenty of time here, as there is a lot of information available.
 
==Database Backend==
 
Here we are interested in dumping data from netdata to a MongoDB database. To do this, we use the json document backend, mentioned here: https://github.com/firehol/netdata/wiki/netdata-backends
 
Using the json document backend, metrics are sent to the database in JSON format.
 
To do this, edit <code>/etc/netdata/netdata.conf</code> and tell it where the database server host/port are.


<pre>
<pre>
Line 273: Line 66:
         send names instead of ids = yes
         send names instead of ids = yes
</pre>
</pre>
=Securing Netdata=
To secure the Netdata portal, set up a VPN and bind Netdata to the VPN IP address.
See [[Netdata/Security]]


=Resources=
=Resources=
Line 279: Line 78:


Netdata wiki: https://github.com/firehol/netdata/wiki
Netdata wiki: https://github.com/firehol/netdata/wiki
Netdata wiki article on setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack


Netdata command line options: https://github.com/firehol/netdata/wiki/Command-Line-Options
Netdata command line options: https://github.com/firehol/netdata/wiki/Command-Line-Options
Line 291: Line 92:


[[Category:Linux]]
[[Category:Linux]]
[[Category:Netdata]]
[[Category:Logging]]
{{DashboardFlag}}

Latest revision as of 00:12, 11 February 2018

Server monitoring/dashboard solution.

Installing

Main article: Netdata/Installing

We recommend installing Netdata on Linux using the quick one-liner: 

bash <(curl -Ss https://my-netdata.io/kickstart.sh)

The Netdata/Installing page has more details about installation steps.

Netdata Web Dashboard

Navigate to port 19999 on the server running netdata, and you'll see a slick live dashboard detailing the status of the server netdata is running on:

NetdataDashboardExample.png

Netdata errs on the side of too much info, rather than too little. It is also designed to display the current state of the machine, it does not put any effort into archiving or data storage or memory other than a backend plugin to continuously export data.

Database Backends

Here we are interested in dumping data from netdata to a database.

Netdata wiki page on backends: https://github.com/firehol/netdata/wiki/netdata-backends

Backend API URLs

We'll use this below, but check out how to use URLs to access the Netdata backend API:

http://192.168.25.236:19999/api/v1/allmetrics?format=json&help=yes

NetdataBackendJSON.png

http://192.168.25.236:19999/api/v1/allmetrics?format=prometheus&help=yes

NetdataBackendPrometheus.png

This helpful information was pointed out by this guide to setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack

Prometheus Backend

Following this guide to setting up Prometheus backend, which is on the Netdata wiki: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack

Netdata/Prometheus

Json Backend

I had some trouble getting Netdata to work with MongoDB, a JSON database. I pointed Netdata to a particular port, and then set up a simple echo socket listener in Python, but saw no activity on the port.

I tried this using the following /etc/netdata/netdata.conf: 

[backend]
	enabled = yes
	type = json
	destination = localhost:27017
	data source = average
	prefix = netdata
	update every = 10
	buffer on failures = 10
	timeout ms = 20000
        send charts matching = *
        send names instead of ids = yes

Securing Netdata

To secure the Netdata portal, set up a VPN and bind Netdata to the VPN IP address.

See Netdata/Security

Resources

Netdata Github repo: https://github.com/firehol/netdata

Netdata wiki: https://github.com/firehol/netdata/wiki

Netdata wiki article on setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack

Netdata command line options: https://github.com/firehol/netdata/wiki/Command-Line-Options

Databases

Replicating databases across multiple netdata instances: https://github.com/firehol/netdata/wiki/Replication-Overview

Archiving data to database backends: https://github.com/firehol/netdata/wiki/netdata-backends

Flags



CSFlag.jpg
Dashboards and Monitoring
tools for creating dashboards and monitoring applications


Time Series Databases:

MongoDB: MongoDB  · Category:MongoDB

Graphite: Graphite  · Category:Graphite

Prometheus: Prometheus  · Category:Prometheus


System Monitoring Tools:

Netdata: Netdata  · Netdata/Prometheus  · Netdata/Security  · Category:Netdata

Collectd: Collectd  · Collectd/Mongo  · Category:Collectd


Dashboard Visualization Tools:

Standalone: Grafana  · Carbon/Graphite

Javascript: D3

Python: Bokeh


Using Docker:

Docker/System Stats


Category:Monitoring  · Category:Docker


Flags  · Template:DashboardFlag  · e



Retrieved from "https://charlesreid1.com/w/index.php?title=Netdata&oldid=24142"