Docker/Pods/Deep Learning - Revision history

Admin: /* GPU-Based Platform */

2017-09-23T19:14:12Z

GPU-Based Platform

← Older revision		Revision as of 19:14, 23 September 2017
Line 169:		Line 169:

	<pre>		<pre>
	nvidia-docker run -it -p 8888:8888 -p 6006:6006 -v ~/:/host waleedka/modern-deep-learning:gpu		$ nvidia-docker run -it -p 8888:8888 -p 6006:6006 -v ~/:/host waleedka/modern-deep-learning:gpu
	</pre>		</pre>

2017-09-23T19:13:43Z

Using GPU Based Platform

← Older revision		Revision as of 19:13, 23 September 2017
Line 64:		Line 64:

	<pre>		<pre>
	$ docker run -it -p 8888:8888 -v ~/:/host waleedka/modern-deep-learning		$ nvidia-docker run -it -p 8888:8888 -v ~/:/host waleedka/modern-deep-learning
	</pre>		</pre>

2017-09-23T19:13:01Z

Docker container from dockerhub

← Older revision		Revision as of 19:13, 23 September 2017
Line 18:		Line 18:
	==Docker container from dockerhub==		==Docker container from dockerhub==

	To get various containers set up, we can use a container created by Github user waleedka:		To get various containers set up, we can use a container created by Github user [https://github.com/waleedka @waleedka]:
	* Github: https://github.com/waleedka/modern-deep-learning-docker		* Github: https://github.com/waleedka/modern-deep-learning-docker
	* Dockerhub: https://hub.docker.com/r/waleedka/modern-deep-learning/		* Dockerhub: https://hub.docker.com/r/waleedka/modern-deep-learning/
Line 24:		Line 24:
	This Github repo provides a Dockerfile that installs pretty much every item we wanted from the list above, plus a few other things (Java).		This Github repo provides a Dockerfile that installs pretty much every item we wanted from the list above, plus a few other things (Java).

	There is also the floydhub snake-oil salesman, who nonetheless has some interesting materials: https://github.com/floydhub/dl-docker		There is also the @floydhub snake-oil salesman, who nonetheless has some interesting materials: https://github.com/floydhub/dl-docker

	==Setting Up Docker Container==		==Setting Up Docker Container==

2017-05-07T18:28:43Z

Custom image

← Older revision		Revision as of 18:28, 7 May 2017
Line 149:		Line 149:

	<pre>		<pre>
	$ docker build -t ~~cmr_stunnel_http~~		$ docker build -t deep_learning
	</pre>		</pre>

Line 155:		Line 155:

	<pre>		<pre>
	$ docker run -it -p 8888:8888 ~~deep-learning~~		$ docker run -it -p 8888:8888 deep_learning
	</pre>		</pre>

2017-05-07T18:28:23Z

Custom image

← Older revision		Revision as of 18:28, 7 May 2017
Line 138:		Line 138:

	===Custom image===		===Custom image===

			Or, use the custom image:

	<pre>		<pre>
	$ git clone https://charlesreid1.com:3000/docker/d-deep-learning.git		$ git clone https://charlesreid1.com:3000/docker/d-deep-learning.git
	$ cd d-deep-learning		$ cd d-deep-learning
	$		</pre>
	$ docker run -it -p 8888:8888 ~~waleedka/modern-~~deep-learning
			Build it:

			<pre>
			$ docker build -t cmr_stunnel_http
			</pre>

			Run it:

			<pre>
			$ docker run -it -p 8888:8888 deep-learning
	</pre>		</pre>

2017-05-07T17:55:07Z

CPU-Based Platform (Macbook Pro)

← Older revision		Revision as of 17:55, 7 May 2017
Line 135:		Line 135:
	import matplotlib		import matplotlib
	import keras		import keras
			</pre>

			===Custom image===

			<pre>
			$ git clone https://charlesreid1.com:3000/docker/d-deep-learning.git
			$ cd d-deep-learning
			$
			$ docker run -it -p 8888:8888 waleedka/modern-deep-learning
	</pre>		</pre>

2017-05-07T17:52:16Z

CPU-Based Platform (Macbook Pro)

← Older revision		Revision as of 17:52, 7 May 2017
Line 106:		Line 106:
	===Stock image===		===Stock image===

	Start out by running Docker.app in the Applications folder. This will run the Docker daemon in the background.		Start out by running Docker.app in the Applications folder. This will run the Docker daemon in the background.

	Now ~~run docker pull to~~ get the ~~stock deep learning Docker~~ container:		Now get the docker container, run it, and start a notebook.

	<pre>		<pre>
	$ docker pull waleedka/modern-deep-learning		$ docker pull waleedka/modern-deep-learning
	~~</pre>~~

	~~Now take~~ it ~~for a test drive. Start the container~~:		$ docker run -it -p 8888:8888 waleedka/modern-deep-learning

	~~<pre>~~
	~~$ docker run -it -p 8888:8888 waleedka/modern-deep-learning~~
	root@a944863bc1e6:~#		root@a944863bc1e6:~#
	~~</pre>~~

	~~Now you can start the Jupyter notebook, and access it from the host at port 8888:~~

	~~<pre>~~
	root@a944863bc1e6:~# jupyter notebook		root@a944863bc1e6:~# jupyter notebook
	</pre>		</pre>
Line 143:		Line 136:
	import keras		import keras
	</pre>		</pre>


	==GPU-Based Platform==		==GPU-Based Platform==

2017-05-07T17:51:23Z

Docker Deep Learning Workflow

← Older revision		Revision as of 17:51, 7 May 2017
Line 72:		Line 72:

	To customize it, I created a git repository: https://git.charlesreid1.com/docker/d-deep-learning		To customize it, I created a git repository: https://git.charlesreid1.com/docker/d-deep-learning

	~~=Docker Deep Learning Workflow=~~

	==Data Volumes Strategy==		==Data Volumes Strategy==

2017-05-07T17:51:10Z

Training Workflow vs Prediction Workflow

← Older revision		Revision as of 17:51, 7 May 2017
Line 74:		Line 74:

	=Docker Deep Learning Workflow=		=Docker Deep Learning Workflow=

	~~==Training Workflow vs Prediction Workflow==~~

	The training workflow consists of providing a large number of containers the same training set, training many different models on the data set, and outputting the results of each model to a host directory or database. Containers take training data in, and dump machine models out.

	Outside of the Docker container workflow, there is a separate step in which each model's performance is evaluated, using whatever criteria is most appropriate. Maybe it is a single metric, maybe is a weighted average of multiple metrics. Whatever it is, the giant pile of models that was generated in the training workflow is whittled down to one or a few models that are useful.

	The prediction workflow is used once that final model has been picked. In the prediction workflow, each container loads up the same model, and applies it to different data sets. It is the opposite of the training workflow. Containers take a machine model in, and dump data (predictions) out.

	==Data Volumes Strategy==		==Data Volumes Strategy==

2017-05-07T17:50:51Z

Customizing Docker Container

← Older revision		Revision as of 17:50, 7 May 2017
Line 69:		Line 69:
	==Customizing Docker Container==		==Customizing Docker Container==

	~~If you want to use~~ the ~~docker~~ image as-is~~, you can~~ just ~~grab the Dockerfile from Dockerhub, build it, and go. However, if you are interested in modifying~~ the image~~, you'll want to grab the Dockerfile directly~~ from Github~~. Here is the link again~~.		Using the Docker image as-is just requires getting the image from Github or Docker hub.

	https://~~github~~.com/~~waleedka~~/~~modern~~-deep-learning~~-docker~~		To customize it, I created a git repository: https://git.charlesreid1.com/docker/d-deep-learning

	=Docker Deep Learning Workflow=		=Docker Deep Learning Workflow=