Fuel - Revision history

Admin: /* Wrapping Custom Datasets with Fuel */

2017-10-15T21:43:55Z

Wrapping Custom Datasets with Fuel

← Older revision		Revision as of 21:43, 15 October 2017
Line 38:		Line 38:
	=Wrapping Custom Datasets with Fuel=		=Wrapping Custom Datasets with Fuel=

	Main ~~page: [[~~Fuel/Custom Datasets]]		{{Main\|Fuel/Custom Datasets}}

	Basically, the process of wrapping a custom data set with fuel looks like this:		Basically, the process of wrapping a custom data set with fuel looks like this:

Admin: /* Basic Usage */

2017-10-15T21:43:34Z

Basic Usage

← Older revision		Revision as of 21:43, 15 October 2017
Line 28:		Line 28:
	=Basic Usage=		=Basic Usage=

	~~See [[~~Fuel/Usage]]		{{Main\|Fuel/Usage}}

	Summary:		Summary:

Admin: /* Basic Usage */

2017-10-15T21:42:42Z

Basic Usage

← Older revision		Revision as of 21:42, 15 October 2017
Line 32:		Line 32:
	Summary:		Summary:
	* [[Fuel/Usage#Datasets\|Datasets]] are the principal interface to data, but are abstract classes		* [[Fuel/Usage#Datasets\|Datasets]] are the principal interface to data, but are abstract classes
	* [[Fuel/Usage#IterableDataset Example\|IterableDatasets]] allow sequential access to data in specified order only		* [[Fuel/Usage#IterableDataset Example\|IterableDatasets]] (less powerful) allow sequential access to data in specified order only
	* [[Fuel/Usage#IndexableDataset Example\|IndexableDatasets]] allow random access to data		* [[Fuel/Usage#IndexableDataset Example\|IndexableDatasets]] (more powerful) allow random access to data
	* [[Fuel/Usage#Iteration Schemes\|Schemes]] allow iterating through IndexablelDatasets in various orders (batch, sequential, shuffle, etc.)		* [[Fuel/Usage#Iteration Schemes\|Schemes]] allow iterating through IndexablelDatasets in various orders (batch, sequential, shuffle, etc.)

Admin: /* Flags */

2017-10-14T23:08:41Z

Flags

← Older revision		Revision as of 23:08, 14 October 2017
Line 55:		Line 55:
	=Flags=		=Flags=

	~~[[Category:Fuel]]~~		{{FuelFlag}}

	~~[[Category:Data Engineering]]~~
	~~[[Category:NN]]~~
	~~[[Category:ML]]~~

Admin: /* Wrapping Custom Datasets with Fuel */

2017-10-14T23:08:31Z

Wrapping Custom Datasets with Fuel

← Older revision		Revision as of 23:08, 14 October 2017
Line 38:		Line 38:
	=Wrapping Custom Datasets with Fuel=		=Wrapping Custom Datasets with Fuel=

	~~Repo by github user dribnet illustrates how to wrap a new dataset using~~ Fuel~~: https://github.com/dribnet~~/~~lfw_fuel~~		Main page: [[Fuel/Custom Datasets]]

	~~Advantages~~:		Basically, the process of wrapping a custom data set with fuel looks like this:
	* ~~Only takes one command to download~~ the data and ~~import it~~ into fuel		* Specify how the original data should be downloaded, processed, and turned into a fuel data set
	* ~~Then it only takes one command to import the library that wraps~~ the data~~, and~~ be ~~able to turn it into training/testing X and Y~~		* Specify how the fuel data set should be loaded

	~~Disadvantages~~:		The first step - defining how to turn original data into fuel data:
	* ~~One~~-~~size-fits-all; importing~~ data ~~using load_data~~() ~~can take~~ a ~~REALLY long time~~, and ~~must be done every time you run the script~~ (~~not persistent in memory~~)		* Create a download wrapper - this tells fuel how to download the original data ("briq" download?)
	* ~~Complicated~~ to ~~extend~~		* Define a way to load a single piece of data (e.g., parameterized by name) and, optionally, paired/related pieces of data (e.g., two related images)
	* Removes some of the nicer options of fuel		* Convert function to extract all data and assemble it all into an HDF5 file (and remove original data when finished)

	~~Here is what~~ the ~~final payoff looks like~~:		The second step - specifying how the fuel data set should be loaded:
			* Create a fuel Datasets object (inheriting from, e.g., H5PYDataset)
	~~<pre>~~		* Define a way for that data to be loaded (example: make a universally-available load_data method in a package specific to your data set, as in lfw_fuel)
	~~from keras.models import Sequential~~
	from ~~lfw_fuel import lfw~~

	~~# the data~~, ~~shuffled and split between train and test sets~~
	~~(X_train, y_train), (X_test, y_test) = lfw~~.~~load_data(format="deepfunneled")~~

	~~# (build the perfect model here)~~

	~~model~~.~~fit(X_train, Y_train, show_accuracy=True, validation_data=(X_test~~, ~~Y_test)~~)
	~~score = model.evaluate~~(~~X_test~~, ~~Y_test, show_accuracy=True, verbose=0~~)
	~~</pre>~~

	=Flags=		=Flags=

Admin: /* Flags */

2017-10-14T22:59:17Z

Flags

← Older revision		Revision as of 22:59, 14 October 2017
Line 66:		Line 66:
	=Flags=		=Flags=

			[[Category:Fuel]]

	[[Category:Data Engineering]]		[[Category:Data Engineering]]
	[[Category:NN]]		[[Category:NN]]
	[[Category:ML]]		[[Category:ML]]

Admin: /* Install Fuel from Source */

2017-10-14T22:26:20Z

Install Fuel from Source

← Older revision		Revision as of 22:26, 14 October 2017
Line 22:		Line 22:
	$ git clone git@github.com:/mila-udem/fuel.git		$ git clone git@github.com:/mila-udem/fuel.git
	$ cd fuel		$ cd fuel
	$ python setup.py build && python setup.py install		$ python setup.py build
			$ python setup.py install
	</pre>		</pre>

Admin: /* Install */

2017-10-14T22:26:12Z

Install

← Older revision		Revision as of 22:26, 14 October 2017
Line 17:		Line 17:
	Now you can install Fuel.		Now you can install Fuel.

	==Install==		==Install Fuel from Source==

	<pre>		<pre>
Line 24:		Line 24:
	$ python setup.py build && python setup.py install		$ python setup.py build && python setup.py install
	</pre>		</pre>





	=Basic Usage=		=Basic Usage=

Admin: /* Basic Usage */

2017-10-14T22:25:00Z

Basic Usage

Show changes

Admin: /* Basic Usage */

2017-10-14T22:22:03Z

Basic Usage

← Older revision		Revision as of 22:22, 14 October 2017
Line 30:		Line 30:

	=Basic Usage=		=Basic Usage=

			See [[Fuel/Usage]]


	==Datasets==		==Datasets==

← Older revision		Revision as of 21:42, 15 October 2017
Line 32:		Line 32:
	Summary:		Summary:
	* [[Fuel/Usage#Datasets\|Datasets]] are the principal interface to data, but are abstract classes		* [[Fuel/Usage#Datasets\|Datasets]] are the principal interface to data, but are abstract classes
	* [[Fuel/Usage#IterableDataset Example\|IterableDatasets]] allow sequential access to data in specified order only		* [[Fuel/Usage#IterableDataset Example\|IterableDatasets]] (less powerful) allow sequential access to data in specified order only
	* [[Fuel/Usage#IndexableDataset Example\|IndexableDatasets]] allow random access to data		* [[Fuel/Usage#IndexableDataset Example\|IndexableDatasets]] (more powerful) allow random access to data
	* [[Fuel/Usage#Iteration Schemes\|Schemes]] allow iterating through IndexablelDatasets in various orders (batch, sequential, shuffle, etc.)		* [[Fuel/Usage#Iteration Schemes\|Schemes]] allow iterating through IndexablelDatasets in various orders (batch, sequential, shuffle, etc.)