MongoDB: Difference between revisions

From charlesreid1
 
(17 intermediate revisions by one other user not shown)
Line 1: Line 1:
=Setting Up=
The short version: Just use docker. MongoDB authentication documentation is sloppy.


==Installing==
https://git.charlesreid1.com/docker/d-mongodb


===Debian/Ubuntu===
https://git.charlesreid1.com/docker/d-mongoexpress


MongoDB provides instructions for installing on Debian/Ubuntu. The short version: don't do <code>apt-get install mongodb</code>.
=Summary=


Here's what you do:
The brief summary:
* Add the mongodb aptitude repositories to your aptitude
* MongoDB provides a nosql unstructured data store for arbitrarily complicated json structures
* Update your aptitude
* Listens on port 27017
* Install a mongodb package from mongodb.org
* Install from mongodb.org debian repos
* Config handles file paths, logging, security, networking
* Multiple ways to interface (command line shell in Javascript, or via language bindings)
* Users must be created per-database, or a system-wide admin account added
* Enable user access controls, expose to private management LAN interfaces


<pre>
=Installing=
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
sudo apt-get update
sudo apt-get install -y mongodb-org
</pre>


These assume you have ubuntu xenial, see link [https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/] for other LTS releases.
==Native Installation==


===Fixing Stupid Issues===
[[MongoDB/Manual Installation]] - installing MongoDB manually/natively on the OS


On Ubuntu there is a stupid issue with the startup service - a mistake.
==Docker Installation==


Edit the file <code>/lib/systemd/system/mongod.service</code>
To run MongoDB using Docker, I recommend using a docker-pod that has both MongoDB and MongoExpress (web frontend for MongoDB).


Change the line
Links:
* https://git.charlesreid1.com/docker/pod-mongo - docker pod that uses docker containers defined in the repos below to run the docker pod
* https://git.charlesreid1.com/docker/d-mongodb - docker container to run MongoDB
* https://git.charlesreid1.com/docker/d-mongoexpress - docker container to run MongoExpress


<pre>
[[MongoDB/Docker]] - installing/running MongoDB in a docker pod
ExecStart=/usr/bin/mongod -f /etc/mongod.conf
</pre>


to the following (note the name of the conf file):
=Configuring=


<pre>
[[MongoDB/Configuration]] - notes on configuring MongoDB
ExecStart=/usr/bin/mongod -f /etc/mongodb.conf
</pre>


Re-load the service from the edited file by running:
MongoDB documentation on configuration: https://docs.mongodb.com/manual/reference/configuration-options/


<pre>
=Startup Service=
sudo systemctl start mongod
</pre>


Pretty stupid, ey?
[[MongoDB/Startup]] - notes on creating a MongoDB startup service


===Homebrew===
=Access Control=


Was able to install this ok with Homebrew: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-with-homebrew
MongoDB offers two access control mechanisms: user authentication, and network access.


<pre>
First, MongoDB allows you to create an admin user, which can be used to create various user accounts with different permissions levels for different data. This provides a fine-grained access control mechanism around MongoDB.
brew update
brew install mongodb
</pre>


or to install the development version:
[[MongoDB/Users]] - guide to setting up admin/regular users in MongoDB to control access to data in database


<pre>
Second, like any network service, MongoDB can bind to a particular network interface, allowing the network firewall to be used to restrict access to MongoDB.
brew update
brew install mongodb --devel
</pre>


==Configuring==
[[MongoDB/Network Access]] - guide to setting up the network to access (or not allow access) to MongoDB
 
Link to documentation page on config options: https://docs.mongodb.com/manual/reference/configuration-options/
 
By default, MongoDB will not require a config file, and if you don't specify one, it makes some weird decisions.
 
To start mongodb with a specified config file, use the --config or -f options:
 
<pre>
mongod --config /etc/mongod.conf
mongod -f /etc/mongod.conf
</pre>
 
Core mongodb config sections:
* systemLog
* net
 
===systemLog===
 
<pre>
# default:
 
systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true
</pre>
 
can also set verbosity (0-5):
 
<pre>
systemLog:
    destination: file
    path: /var/log/mongodb/mongod.log
    verbosity: 2
    logRotate: rename
</pre>
 
If using logrotate util, set <code>logRotate: reopen</code>
 
Can further customize log behavior for specific components (access, commands, etc.)
 
===net===
 
<pre>
net:
    port: 27017
    bindIp: 10.0.0.1
    ipv6: True
</pre>
 
There are also several options for SSL. Those go into an ssl subsection of the net section of the config file.
 
==Starting==
 
MongoDB can be started with systemd, or using the init.d startup scripts. I hate systemd so I went with the latter.
 
Start by creating the directory where MongoDB will keep all of its data. For example, I used <code>/opt/mongodb</code>. Set the permissions so that the mongodb user/group can read/write to this directory:
 
<pre>
sudo chown -R mongodb:mongodb mongodb/
</pre>
 
Now start the service, which is defined in <code>/etc/init.d/mongodb</code>:
 
<pre>
sudo service mongodb start
</pre>
 
You can issue the status command in place of the start command to check if the process is running:
 
<pre>
$ sudo service mongodb status
● mongodb.service - LSB: An object/document-oriented database
  Loaded: loaded (/etc/init.d/mongodb; bad; vendor preset: enabled)
  Active: active (running) since Tue 2018-01-30 16:59:10 PST; 1min 38s ago
    Docs: man:systemd-sysv-generator(8)
  Process: 1962 ExecStart=/etc/init.d/mongodb start (code=exited, status=0/SUCCESS)
Main PID: 13596 (code=exited, status=0/SUCCESS)
  CGroup: /system.slice/mongodb.service
          └─1973 /usr/bin/mongod --config /etc/mongodb.conf
 
Jan 30 16:59:09 jupiter systemd[1]: Starting LSB: An object/document-oriented database...
Jan 30 16:59:09 jupiter mongodb[1962]:  * Starting database mongodb
Jan 30 16:59:10 jupiter mongodb[1962]:    ...done.
Jan 30 16:59:10 jupiter systemd[1]: Started LSB: An object/document-oriented database.
</pre>
 
You can also turn on logging, and look at the log files in <code>/var/log/mongodb/mongodb.log</code>
 
===Create Users and Enable Authentication===
 
It is a good idea to set up users and user authentication to control access to the data in the database.
 
To create a system-wide mongodb user admin, create a user with the role userAdminAnyDatabase (no other roles!).
 
Start the mongo shell from localhost (which will not require authentication to begin with) or using --noauth flag by using the mongo command:
 
<pre>
$ mongo
>
</pre>
 
Now create a user for the admin database (run these commands directly in the shell). This will create a user "darthvader" with password "secretpass":
 
<pre>
use admin
db.createUser(
  {
    user: "darthvader",
    pwd: "secretpass",
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)
</pre>
 
Now you can enable client access control.
 
As per the mongodb documentation [https://docs.mongodb.com/v3.0/reference/configuration-options/#security.authorization], to enable authorization you can either pass --auth when starting mongod or you can set security.authorization in the mongodb config file as follows:
 
<pre>
security:
    authorization: enabled
</pre>
 
==Selecting an Interface==
 
The first thing you have to decide before interacting
with the database is how you want to interact.
 
The mongodb shell is a javascript shell that can
be used from a command line on the mongodb server.
 
Mongodb also has python language bindings.
there are multiple non-mongo-provided
third party APIs and libraries too, so there
are multiple options.


=Basic CRUD Operations=
=Basic CRUD Operations=
Line 205: Line 59:
MongoDB performs CRUD (create, read, update, delete) transactions/operations on the data that it stores.
MongoDB performs CRUD (create, read, update, delete) transactions/operations on the data that it stores.


==Create (Insert)==
[[Mongo/CRUD]]


To insert documents into a collection:
==Advanced CRUD Operations==
* <code>db.collection.insertOne()</code>
* <code>db.collection.insertMany()</code>


Example: <code>db.users.insertOne({name:"Sue", age:26})</code>
Spelunking in a MongoDB database to see what's there: [[Mongo/Spelunking]]


==Read (Query)==
=Basic Collections Operations=


To read documents from a collection,
Basic operations on collections:
use the find function:
*  <code>db.collection.find()</code>


Example: <code>db.users.find({age:{$gt:18}})</code>
[[Mongo/Collections]]


==Update==
=Basic Database Operations=


To update documents in a collection:
Notes on basic database operations:
* <code>db.collection.updateOne()</code>
* <code>db.collection.updateMany()</code>
* <code>db.collection.replaceOne()</code>


Example: <code>db.users.updateMany( {age:{$lt:18}, $set: {status: "reject"}} )</code>
[[Mongo/Databases]]
 
==Delete==
 
Delete documents one at a time or en masse:
* <code>db.collection.deleteOne()</code>
* <code>db.collection.deleteMany()</code>


=Monitoring=
=Monitoring=
Line 246: Line 87:
==MongoDB as a Monitoring Data Store==
==MongoDB as a Monitoring Data Store==


The Write_MongoDB plugin provides a plugin for collectd to write its data to MongoDB.
collectd has a Write_MongoDB plugin to allow collectd to write its data to MongoDB.


Plugin link: https://collectd.org/wiki/index.php/Plugin:Write_MongoDB
Plugin link: https://collectd.org/wiki/index.php/Plugin:Write_MongoDB
=APIs=
Python API: [[Pymongo]]
Java API: [[MongoDB/Java]]


=References=
=References=
Line 257: Line 104:


Cheat sheet: https://blog.codecentric.de/files/2012/12/MongoDB-CheatSheet-v1_0.pdf
Cheat sheet: https://blog.codecentric.de/files/2012/12/MongoDB-CheatSheet-v1_0.pdf


=Related Page=
=Related Page=
Line 264: Line 110:


=Flags=
=Flags=
{{DashboardFlag}}


[[Category:MongoDB]]
[[Category:MongoDB]]
[[Category:Python]]
[[Category:Python]]
[[Category:2018]]
[[Category:February 2018]]

Latest revision as of 19:08, 17 August 2020

The short version: Just use docker. MongoDB authentication documentation is sloppy.

https://git.charlesreid1.com/docker/d-mongodb

https://git.charlesreid1.com/docker/d-mongoexpress

Summary

The brief summary:

  • MongoDB provides a nosql unstructured data store for arbitrarily complicated json structures
  • Listens on port 27017
  • Install from mongodb.org debian repos
  • Config handles file paths, logging, security, networking
  • Multiple ways to interface (command line shell in Javascript, or via language bindings)
  • Users must be created per-database, or a system-wide admin account added
  • Enable user access controls, expose to private management LAN interfaces

Installing

Native Installation

MongoDB/Manual Installation - installing MongoDB manually/natively on the OS

Docker Installation

To run MongoDB using Docker, I recommend using a docker-pod that has both MongoDB and MongoExpress (web frontend for MongoDB).

Links:

MongoDB/Docker - installing/running MongoDB in a docker pod

Configuring

MongoDB/Configuration - notes on configuring MongoDB

MongoDB documentation on configuration: https://docs.mongodb.com/manual/reference/configuration-options/

Startup Service

MongoDB/Startup - notes on creating a MongoDB startup service

Access Control

MongoDB offers two access control mechanisms: user authentication, and network access.

First, MongoDB allows you to create an admin user, which can be used to create various user accounts with different permissions levels for different data. This provides a fine-grained access control mechanism around MongoDB.

MongoDB/Users - guide to setting up admin/regular users in MongoDB to control access to data in database

Second, like any network service, MongoDB can bind to a particular network interface, allowing the network firewall to be used to restrict access to MongoDB.

MongoDB/Network Access - guide to setting up the network to access (or not allow access) to MongoDB

Basic CRUD Operations

MongoDB performs CRUD (create, read, update, delete) transactions/operations on the data that it stores.

Mongo/CRUD

Advanced CRUD Operations

Spelunking in a MongoDB database to see what's there: Mongo/Spelunking

Basic Collections Operations

Basic operations on collections:

Mongo/Collections

Basic Database Operations

Notes on basic database operations:

Mongo/Databases

Monitoring

MongoDB as a Monitoring Target

MongoDB has several mechanisms for monitoring the state of the database (per second operations, cache sizes, disk and memory usage, etc.)

Utilities like Netdata and Collectd have plugins written for MongoDB that can collect this information as part of scraping the system status.

MongoDB as a Monitoring Data Store

collectd has a Write_MongoDB plugin to allow collectd to write its data to MongoDB.

Plugin link: https://collectd.org/wiki/index.php/Plugin:Write_MongoDB

APIs

Python API: Pymongo

Java API: MongoDB/Java

References

pymodm: https://pymodm.readthedocs.io/en/latest/getting-started.html

Database design patterns: https://docs.mongodb.com/manual/applications/data-models/

Cheat sheet: https://blog.codecentric.de/files/2012/12/MongoDB-CheatSheet-v1_0.pdf

Related Page

Flags



CSFlag.jpg
Dashboards and Monitoring
tools for creating dashboards and monitoring applications


Time Series Databases:

MongoDB: MongoDB  · Category:MongoDB

Graphite: Graphite  · Category:Graphite

Prometheus: Prometheus  · Category:Prometheus


System Monitoring Tools:

Netdata: Netdata  · Netdata/Prometheus  · Netdata/Security  · Category:Netdata

Collectd: Collectd  · Collectd/Mongo  · Category:Collectd


Dashboard Visualization Tools:

Standalone: Grafana  · Carbon/Graphite

Javascript: D3

Python: Bokeh


Using Docker:

Docker/System Stats


Category:Monitoring  · Category:Docker


Flags  · Template:DashboardFlag  · e



Retrieved from "https://charlesreid1.com/w/index.php?title=MongoDB&oldid=28459"