Gitea/Dump - Revision history

Admin: /* Format of Output */

2018-02-25T09:19:01Z

Format of Output

← Older revision		Revision as of 09:19, 25 February 2018
Line 27:		Line 27:

	Just as the .git folder stores the entire history and log of the repository normally, so too the zip file of resulting repositories can be used to get all historical information about all repositories in gitea. Most of the git utilities are designed to work with .git directories with arbitrary names in arbitrary locations, so that works to our advantage.		Just as the .git folder stores the entire history and log of the repository normally, so too the zip file of resulting repositories can be used to get all historical information about all repositories in gitea. Most of the git utilities are designed to work with .git directories with arbitrary names in arbitrary locations, so that works to our advantage.

			==Location of Output and -t Flag==

			Let's talk about how to control where the dump file goes.

			The gitea dump command has a -t flag, which is described as:

			<pre>
			--tempdir value, -t value Temporary dir path (default: "/tmp")
			</pre>

			Let's clear up what this means for the gitea dump process.

			===Gitea Dump Process===

			The dump process is as follows:
			* Create a gitea temporary directory to dump things in
			* Zip up the gitea temporary directory
			* Move the zip file to the directory where the gitea dump command was run
			* Leave the temporary gitea directory in place

			The gitea temporary directory is inside of the temporary directory specified by the -t flag. As mentioned, this is /tmp by default. The gitea temporary directory will have the name <code>gitea-dump-<random number></code> (e.g., <code>gitea-dump-110153823</code>)

			The contents of the dump directory are the gitea database dump and all the repositories: <code>gitea-db.sql, gitea-repo.zip</code>

	==Extracting Git Log Info==		==Extracting Git Log Info==

Admin: /* Logs */

2018-02-17T08:55:44Z

Logs

← Older revision		Revision as of 08:55, 17 February 2018
Line 28:		Line 28:
	Just as the .git folder stores the entire history and log of the repository normally, so too the zip file of resulting repositories can be used to get all historical information about all repositories in gitea. Most of the git utilities are designed to work with .git directories with arbitrary names in arbitrary locations, so that works to our advantage.		Just as the .git folder stores the entire history and log of the repository normally, so too the zip file of resulting repositories can be used to get all historical information about all repositories in gitea. Most of the git utilities are designed to work with .git directories with arbitrary names in arbitrary locations, so that works to our advantage.

	==~~Logs~~==		==Extracting Git Log Info==

	~~Goal is~~ to extract log information from each repository. To do ~~this~~, ~~move into a~~ repository directory and ~~execute~~ the git ~~log command. Control formatting to make info easier to extract~~.		Our goal was to extract log information from each repository, specifically to create a count of commits and the date and time of each one so that those could finally be assembled into a D3 calendar. To do that, the git log command was run from each repository directory, and the output processed to assemble CSV data about git commits.

	===Formatting git log using preconfigured format===		===Formatting git log using preconfigured format===

Admin: /* Format of Output */

2018-02-17T08:54:10Z

Format of Output

← Older revision		Revision as of 08:54, 17 February 2018
Line 17:		Line 17:
	==Format of Output==		==Format of Output==

	The gitea dump dumps out the log files, and a zip file containing all of the repositories.		Notes on formatting of output:
			* The gitea dump dumps out the log files, and a zip file containing all of the repositories.

	The directory structure is as follows:		The directory structure is as follows:
			* The repositories folder contains all repositories.
			* Within the repositories folder, there is a folder for each user and organization.
			* Inside each user or organization folder, there is one folder for each repository that user owns.
			* The folders are called <code><reponame>.git</code>, and the contents are the contents of the .git folder in that repo

	~~The repositories folder contains all repositories.~~		Just as the .git folder stores the entire history and log of the repository normally, so too the zip file of resulting repositories can be used to get all historical information about all repositories in gitea. Most of the git utilities are designed to work with .git directories with arbitrary names in arbitrary locations, so that works to our advantage.

	~~Within~~ the ~~repositories folder, there is a folder for each user and organization~~.

	~~Inside each user or organization~~ folder~~, there is one folder for each repository that user owns.~~

	~~The folders are called <code><reponame>.git</code>~~ and ~~are the contents~~ of the ~~.git folder for that~~ repository.

	~~The .git folder~~ can be used to get ~~any and~~ all ~~info~~ about ~~the repository history~~. Most of the git utilities are designed to work with git directories in arbitrary locations, so that works to our advantage.

	==Logs==		==Logs==

Admin: /* Formatting git log as JSON */

2018-02-17T08:48:27Z

Formatting git log as JSON

← Older revision		Revision as of 08:48, 17 February 2018
Line 56:		Line 56:
	===Formatting git log as JSON===		===Formatting git log as JSON===

	~~To format~~ the output ~~of the git log as JSON~~, ~~use~~ the --pretty=format ~~argument when calling~~ git ~~log~~, ~~along with~~ the ~~extremely extensive git log documentation~~.		There are many, many formatting stings to choose from, and you can get very fancy with the output. For example, this technique uses the pretty format to print each git commit item as a JSON in curlybrackets {}, and then wraps the resulting output in square brackets [] using perl.

	<pre>		<pre>
Line 66:		Line 66:
	</pre>		</pre>

	https://gist.github.com/textarcana/1306223		Via https://gist.github.com/textarcana/1306223

	==Flags==		==Flags==

Admin: /* Formatting git log using string */

2018-02-17T08:47:02Z

Formatting git log using string

← Older revision		Revision as of 08:47, 17 February 2018
Line 51:		Line 51:

	<pre>		<pre>
			git log --pretty=%H %ai %s
	</pre>		</pre>

Admin: /* Logs */

2018-02-14T17:45:19Z

Logs

← Older revision		Revision as of 17:45, 14 February 2018
Line 33:		Line 33:
	==Logs==		==Logs==

	Goal is to extract log information from each repository.		Goal is to extract log information from each repository. To do this, move into a repository directory and execute the git log command. Control formatting to make info easier to extract.

	~~Use this gist~~ to ~~get~~ a ~~one-liner~~ git log to ~~json output: https~~:~~//gist.github.com/textarcana/1306223~~		===Formatting git log using preconfigured format===

			Git already provides several formats for dumping out the logs:
			* oneliner (commit and summary only)
			* short (commit, author, title)
			* medium (commit, author, date, title, commit message)
			* full (commit, author, committer, title, commit message)
			* email (from sha, from author, from date, subject, commit message)

			Alternatively, you can customize the output format exactly by using output strings to control what text goes where. See below for examples.

			===Formatting git log using string===

			To use a custom string to format the output of git log, pass a format string to pretty:

	<pre>		<pre>
	#!/~~usr/bin/env bash~~		</pre>

			===Formatting git log as JSON===

	~~# Use this one~~-~~liner to produce a JSON literal from~~ the ~~Git~~ log:		To format the output of the git log as JSON, use the --pretty=format argument when calling git log, along with the extremely extensive git log documentation.

			<pre>
	git log \		git log \
	--pretty=format:'{%n "commit": "%H",%n "author": "%aN <%aE>",%n "date": "%ad",%n "message": "%f"%n},' \		--pretty=format:'{%n "commit": "%H",%n "author": "%aN <%aE>",%n "date": "%ad",%n "message": "%f"%n},' \
Line 49:		Line 65:
	</pre>		</pre>

			https://gist.github.com/textarcana/1306223

	==Flags==		==Flags==

Admin: /* Logs */

2018-02-14T17:13:53Z

Logs

← Older revision		Revision as of 17:13, 14 February 2018
Line 48:		Line 48:
	perl -pe 's/},]/}]/'		perl -pe 's/},]/}]/'
	</pre>		</pre>



			==Flags==

			[[Category:Gitea]]
			[[Category:Git]]
			[[Category:Charlesreid1]]

Admin at 17:13, 14 February 2018

2018-02-14T17:13:15Z

← Older revision		Revision as of 17:13, 14 February 2018
Line 1:		Line 1:
	Via Gitea documentation: https://docs.gitea.io/en-us/command-line/		Via Gitea documentation: https://docs.gitea.io/en-us/command-line/

	~~Can~~ dump the entire contents of Gitea's database to a zip ~~directory using the gitea dump command~~.		==What Dump is For==

			Running the gitea dump command will dump the entire contents of Gitea's internal database and reposiitories to disk and zip it up.

	The way I have described installing [[Gitea]], it is important to run this command as the correct user, in this case the user git:		The way I have described installing [[Gitea]], it is important to run this command as the correct user, in this case the user git:

Admin at 17:08, 14 February 2018

2018-02-14T17:08:29Z

@@ Line 13: / Line 13: @@
 This will begin the backup process, which will take a few minutes.
-==Format==
+==Format of Output==
 The gitea dump dumps out the log files, and a zip file containing all of the repositories.
@@ Line 28: / Line 28: @@
 The .git folder can be used to get any and all info about the repository history. Most of the git utilities are designed to work with git directories in arbitrary locations, so that works to our advantage.

Admin at 09:30, 14 February 2018

2018-02-14T09:30:44Z

← Older revision		Revision as of 09:30, 14 February 2018
Line 12:		Line 12:

	This will begin the backup process, which will take a few minutes.		This will begin the backup process, which will take a few minutes.

			==Format==

			The gitea dump dumps out the log files, and a zip file containing all of the repositories.

			The directory structure is as follows:

			The repositories folder contains all repositories.

			Within the repositories folder, there is a folder for each user and organization.

			Inside each user or organization folder, there is one folder for each repository that user owns.

			The folders are called <code><reponame>.git</code> and are the contents of the .git folder for that repository.

			The .git folder can be used to get any and all info about the repository history. Most of the git utilities are designed to work with git directories in arbitrary locations, so that works to our advantage.