Netdata: Difference between revisions

From charlesreid1
 
(18 intermediate revisions by the same user not shown)
Line 3: Line 3:
=Installing=
=Installing=


==Prerequisites and dependencies==
{{Main|Netdata/Installing}}


<pre>
We recommend installing Netdata on Linux using the quick one-liner:
apt-get install -y zlib1g-dev uuid-dev libmnl-dev gcc make \
    autoconf autoconf-archive autogen automake pkg-config curl
 
apt-get install -y python python-yaml python-mysqldb \
    python-psycopg2 nodejs lm-sensors netcat
</pre>
 
==Installing Netdata on Linux==
 
Here's a quick one-liner to avoid checking out the github repository.


<pre>
<pre>
Line 21: Line 11:
</pre>
</pre>


==What It Do==
The [[Netdata/Installing]] page has more details about installation steps.
 
The script does the following:
* Create users/groups as needed
* Get logrotate
* Set permissions
* Download default netdata configuration
 
==Output from Install Process==
 
Here is the output from the installation process, which contains some useful information:
 
{{Scrollbox|
<pre>
--- Restore user edited netdata configuration files ---
--- Fix generated files permissions ---
[/usr/src/netdata.git]# find ./system/ -type f -a \! -name \*.in -a \! -name Makefile\* -a \! -name \*.conf -a \! -name \*.service -a \! -name \*.logrotate -exec chmod 755 \{\} \;                                       
OK 
 
--- Add user netdata to required user groups ---
Adding netdata user group ...
[/usr/src/netdata.git]# groupadd -r netdata
OK 


Adding netdata user account ...
=Netdata Web Dashboard=
[/usr/src/netdata.git]# useradd -r -g netdata -c netdata -s /usr/sbin/nologin -d / netdata
OK 


Group 'docker' does not exist.
Navigate to port 19999 on the server running netdata, and you'll see a slick live dashboard detailing the status of the server netdata is running on:
Group 'nginx' does not exist.
Group 'varnish' does not exist.
Group 'haproxy' does not exist.
Adding netdata user to the adm group ...
[/usr/src/netdata.git]# usermod -a -G adm netdata  
OK 


Group 'nsd' does not exist.
[[Image:NetdataDashboardExample.png|400px]]
Adding netdata user to the proxy group ...
[/usr/src/netdata.git]# usermod -a -G proxy netdata
OK 


Group 'squid' does not exist.
Netdata errs on the side of too much info, rather than too little. It is also designed to display the current state of the machine, it does not put any effort into archiving or data storage or memory other than a backend plugin to continuously export data.
--- Install logrotate configuration for netdata ---
[/usr/src/netdata.git]# cp system/netdata.logrotate /etc/logrotate.d/netdata
OK 


[/usr/src/netdata.git]# chmod 644 /etc/logrotate.d/netdata
=Database Backends=
OK 


--- Read installation options from netdata.conf ---
Here we are interested in dumping data from netdata to a database.
--- Fix permissions of netdata directories (using user 'netdata') ---
[/usr/src/netdata.git]# chown -R root:netdata /etc/netdata
OK 


[/usr/src/netdata.git]# find /etc/netdata -type f -exec chmod 0640 \{\} \;
Netdata wiki page on backends: https://github.com/firehol/netdata/wiki/netdata-backends
OK 


[/usr/src/netdata.git]# find /etc/netdata -type d -exec chmod 0755 \{\} \;
==Backend API URLs==
OK 


[/usr/src/netdata.git]# chown -R netdata:netdata /usr/share/netdata/web
We'll use this below, but check out how to use URLs to access the Netdata backend API:
OK 


[/usr/src/netdata.git]# find /usr/share/netdata/web -type f -exec chmod 0664 \{\} \;
http://192.168.25.236:19999/api/v1/allmetrics?format=json&help=yes
OK 


[/usr/src/netdata.git]# find /usr/share/netdata/web -type d -exec chmod 0775 \{\} \;
[[Image:NetdataBackendJSON.png|500px]]
OK 


[/usr/src/netdata.git]# chown -R netdata:netdata /var/lib/netdata
http://192.168.25.236:19999/api/v1/allmetrics?format=prometheus&help=yes
OK 


[/usr/src/netdata.git]# chown -R netdata:netdata /var/cache/netdata
[[Image:NetdataBackendPrometheus.png|500px]]
OK 


[/usr/src/netdata.git]# chown -R netdata:netdata /var/log/netdata
This helpful information was pointed out by this guide to setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack
OK 


[/usr/src/netdata.git]# chmod 755 /var/log/netdata
==Prometheus Backend==
OK 


[/usr/src/netdata.git]# chown netdata:root /var/log/netdata
Following this guide to setting up Prometheus backend, which is on the Netdata wiki: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack
OK 


[/usr/src/netdata.git]# chown -R root /usr/libexec/netdata
[[Netdata/Prometheus]]
OK 


[/usr/src/netdata.git]# find /usr/libexec/netdata -type d -exec chmod 0755 \{\} \;
==Json Backend==
OK 


[/usr/src/netdata.git]# find /usr/libexec/netdata -type f -exec chmod 0644 \{\} \;
I had some trouble getting Netdata to work with MongoDB, a JSON database. I pointed Netdata to a particular port, and then set up a simple echo socket listener in Python, but saw no activity on the port.  
OK 


[/usr/src/netdata.git]# find /usr/libexec/netdata -type f -a -name \*.plugin -exec chmod 0755 \{\} \;
I tried this using the following <code>/etc/netdata/netdata.conf</code>:
OK 
 
[/usr/src/netdata.git]# find /usr/libexec/netdata -type f -a -name \*.sh -exec chmod 0755 \{\} \;
OK 
 
[/usr/src/netdata.git]# chown root:netdata /usr/libexec/netdata/plugins.d/apps.plugin
OK 
 
[/usr/src/netdata.git]# chmod 0750 /usr/libexec/netdata/plugins.d/apps.plugin
OK 
 
[/usr/src/netdata.git]# setcap cap_dac_read_search\,cap_sys_ptrace+ep /usr/libexec/netdata/plugins.d/apps.plugin                                                                                                           
OK 
 
[/usr/src/netdata.git]# chown root:netdata /usr/libexec/netdata/plugins.d/cgroup-network
OK 
 
[/usr/src/netdata.git]# chmod 4750 /usr/libexec/netdata/plugins.d/cgroup-network
OK 
 
[/usr/src/netdata.git]# chown root /usr/libexec/netdata/plugins.d/cgroup-network-helper.sh
OK 
 
[/usr/src/netdata.git]# chmod 0550 /usr/libexec/netdata/plugins.d/cgroup-network-helper.sh
OK 
 
[/usr/src/netdata.git]# chmod a+rX /usr/libexec
OK 
 
[/usr/src/netdata.git]# chmod a+rX /usr/share/netdata
OK 
 
--- Install netdata at system init ---
Installing systemd service...
[/usr/src/netdata.git]# cp system/netdata.service /etc/systemd/system/netdata.service
OK 
 
[/usr/src/netdata.git]# systemctl daemon-reload
OK 
 
[/usr/src/netdata.git]# systemctl enable netdata
Created symlink /etc/systemd/system/multi-user.target.wants/netdata.service -> /etc/systemd/system/netdata.service.
OK 
 
--- Start netdata ---
[/usr/src/netdata.git]# /bin/systemctl stop netdata
OK 
 
[/usr/src/netdata.git]# /bin/systemctl restart netdata
OK 
 
OK. NetData Started!
 
 
-------------------------------------------------------------------------------
 
Downloading default configuration from netdata...
[/usr/src/netdata.git]# curl -s -o /etc/netdata/netdata.conf.new http://localhost:19999/netdata.conf
OK 
 
[/usr/src/netdata.git]# mv /etc/netdata/netdata.conf.new /etc/netdata/netdata.conf
OK 
 
OK  New configuration saved for you to edit at /etc/netdata/netdata.conf
 
[/usr/src/netdata.git]# chown netdata /etc/netdata/netdata.conf
OK 
 
[/usr/src/netdata.git]# chmod 0664 /etc/netdata/netdata.conf
OK 
 
--- Check KSM (kernel memory deduper) ---
 
Memory de-duplication instructions
 
You have kernel memory de-duper (called Kernel Same-page Merging,
or KSM) available, but it is not currently enabled.
 
To enable it run:
 
    echo 1 >/sys/kernel/mm/ksm/run
    echo 1000 >/sys/kernel/mm/ksm/sleep_millisecs
 
If you enable it, you will save 40-60% of netdata memory.
 
--- Check version.txt ---
--- Check apps.plugin ---
--- Generate netdata-uninstaller.sh ---
--- Basic netdata instructions ---
 
netdata by default listens on all IPs on port 19999,
so you can access it with:
 
  http://this.machine.ip:19999/
 
To stop netdata run:
 
  systemctl stop netdata
 
To start netdata run:
 
  systemctl start netdata
 
 
Uninstall script generated: ./netdata-uninstaller.sh
Update script generated  : ./netdata-updater.sh
 
netdata-updater.sh can work from cron. It will trigger an email from cron
only if it fails (it does not print anything when it can update netdata).
--- Installing netdata-updater at cron ---
[/usr/src/netdata.git]# ln -s /usr/src/netdata.git/netdata-updater.sh /etc/cron.daily/netdata-updater
OK 
 
 
--- We are done! ---
 
  ^
  |.-.  .-.  .-.  .-.  .-.  .  netdata                          .-.  .-
  |  '-'  '-'  '-'  '-'  '-'  is installed and running now!  -'  '-' 
  +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->
 
  enjoy real-time performance and health monitoring...
 
OK 
</pre>
}}
 
=Usage=
 
==Web dashboard==
 
To use netdata's web dashboard, open a browser and navigate to localhost:19999
 
You should now see a web dashboard with live statistics about the system.
 
==Database Backend==
 
Here we are interested in dumping data from netdata to a MongoDB database. To do this, we use the json document backend, mentioned here: https://github.com/firehol/netdata/wiki/netdata-backends
 
Using the json document backend, metrics are sent to the database in JSON format.
 
To do this, edit <code>/etc/netdata/netdata.conf</code> and tell it where the database server host/port are.


<pre>
<pre>
Line 271: Line 66:
         send names instead of ids = yes
         send names instead of ids = yes
</pre>
</pre>
=Securing Netdata=
To secure the Netdata portal, set up a VPN and bind Netdata to the VPN IP address.
See [[Netdata/Security]]


=Resources=
=Resources=
Line 277: Line 78:


Netdata wiki: https://github.com/firehol/netdata/wiki
Netdata wiki: https://github.com/firehol/netdata/wiki
Netdata wiki article on setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack


Netdata command line options: https://github.com/firehol/netdata/wiki/Command-Line-Options
Netdata command line options: https://github.com/firehol/netdata/wiki/Command-Line-Options
Line 289: Line 92:


[[Category:Linux]]
[[Category:Linux]]
[[Category:Netdata]]
[[Category:Logging]]
{{DashboardFlag}}

Latest revision as of 00:12, 11 February 2018

Server monitoring/dashboard solution.

Installing

Main article: Netdata/Installing

We recommend installing Netdata on Linux using the quick one-liner: 

bash <(curl -Ss https://my-netdata.io/kickstart.sh)

The Netdata/Installing page has more details about installation steps.

Netdata Web Dashboard

Navigate to port 19999 on the server running netdata, and you'll see a slick live dashboard detailing the status of the server netdata is running on:

NetdataDashboardExample.png

Netdata errs on the side of too much info, rather than too little. It is also designed to display the current state of the machine, it does not put any effort into archiving or data storage or memory other than a backend plugin to continuously export data.

Database Backends

Here we are interested in dumping data from netdata to a database.

Netdata wiki page on backends: https://github.com/firehol/netdata/wiki/netdata-backends

Backend API URLs

We'll use this below, but check out how to use URLs to access the Netdata backend API:

http://192.168.25.236:19999/api/v1/allmetrics?format=json&help=yes

NetdataBackendJSON.png

http://192.168.25.236:19999/api/v1/allmetrics?format=prometheus&help=yes

NetdataBackendPrometheus.png

This helpful information was pointed out by this guide to setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack

Prometheus Backend

Following this guide to setting up Prometheus backend, which is on the Netdata wiki: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack

Netdata/Prometheus

Json Backend

I had some trouble getting Netdata to work with MongoDB, a JSON database. I pointed Netdata to a particular port, and then set up a simple echo socket listener in Python, but saw no activity on the port.

I tried this using the following /etc/netdata/netdata.conf: 

[backend]
	enabled = yes
	type = json
	destination = localhost:27017
	data source = average
	prefix = netdata
	update every = 10
	buffer on failures = 10
	timeout ms = 20000
        send charts matching = *
        send names instead of ids = yes

Securing Netdata

To secure the Netdata portal, set up a VPN and bind Netdata to the VPN IP address.

See Netdata/Security

Resources

Netdata Github repo: https://github.com/firehol/netdata

Netdata wiki: https://github.com/firehol/netdata/wiki

Netdata wiki article on setting up Prometheus backend: https://github.com/firehol/netdata/wiki/Netdata,-Prometheus,-and-Grafana-Stack

Netdata command line options: https://github.com/firehol/netdata/wiki/Command-Line-Options

Databases

Replicating databases across multiple netdata instances: https://github.com/firehol/netdata/wiki/Replication-Overview

Archiving data to database backends: https://github.com/firehol/netdata/wiki/netdata-backends

Flags



CSFlag.jpg
Dashboards and Monitoring
tools for creating dashboards and monitoring applications


Time Series Databases:

MongoDB: MongoDB  · Category:MongoDB

Graphite: Graphite  · Category:Graphite

Prometheus: Prometheus  · Category:Prometheus


System Monitoring Tools:

Netdata: Netdata  · Netdata/Prometheus  · Netdata/Security  · Category:Netdata

Collectd: Collectd  · Collectd/Mongo  · Category:Collectd


Dashboard Visualization Tools:

Standalone: Grafana  · Carbon/Graphite

Javascript: D3

Python: Bokeh


Using Docker:

Docker/System Stats


Category:Monitoring  · Category:Docker


Flags  · Template:DashboardFlag  · e



Retrieved from "https://charlesreid1.com/w/index.php?title=Netdata&oldid=24142"