Compressed data files - Revision history

Administrator: /* Dealing with truncated compressed files */

2014-10-29T20:30:00Z

Dealing with truncated compressed files

← Older revision		Revision as of 15:30, 29 October 2014
Line 10:		Line 10:
	=== Dealing with truncated compressed files ===		=== Dealing with truncated compressed files ===

	A situation sometimes encountered is that a compressed file is truncated (this most frequently occurs when a program aborts and a file buffer was not flushed). Naive application of <tt>gzip -d</tt> (or <tt>gunzip</tt>) will lead to a "unexpected end of file" error message. However, it is easy to access most of the data in this file by ~~compressing~~ "on the fly" and redirecting the data stream to a file. For example, if <tt>output.dat.gz</tt> is truncated, use		A situation sometimes encountered is that a compressed file is truncated (this most frequently occurs when a program aborts and a file buffer was not flushed). Naive application of <tt>gzip -d</tt> (or <tt>gunzip</tt>) will lead to a "unexpected end of file" error message. However, it is easy to access most of the data in this file by decompressing "on the fly" and redirecting the data stream to a file. For example, if <tt>output.dat.gz</tt> is truncated, use
	<pre>		<pre>
	gzip -dc output.dat.gz > output.dat		gzip -dc output.dat.gz > output.dat

Huanxin: fixed one small typo

2014-10-29T20:29:59Z

fixed one small typo

← Older revision		Revision as of 15:29, 29 October 2014
Line 15:		Line 15:
	</pre>		</pre>

	This will yield the same error message as <tt>gzip -d output.dat.gz</tt> but now the decompressed file contents (until close to the truncation point) will be saved to <tt>~~dump~~.dat</tt>		This will yield the same error message as <tt>gzip -d output.dat.gz</tt> but now the decompressed file contents (until close to the truncation point) will be saved to <tt>output.dat</tt>

	=== Avoiding decompression of compressed files ===		=== Avoiding decompression of compressed files ===

Administrator at 20:00, 29 October 2014

2014-10-29T20:00:24Z

@@ Line 1: / Line 1: @@
 * Save disk/tape space
@@ Line 5: / Line 8: @@
 * Easier transfer to other computer systems
 gzip -dc output.dat.gz > output.dat

Administrator at 17:31, 29 October 2014

2014-10-29T17:31:52Z

← Older revision		Revision as of 12:31, 29 October 2014
Line 4:		Line 4:
	* File corruption is detected more easily		* File corruption is detected more easily
	* Easier transfer to other computer systems		* Easier transfer to other computer systems

			A situation sometimes encountered is that a compressed file is truncated (this most frequently occurs when a program aborts and a file buffer was not flushed). Naive application of "gzip -d" (or "gunzip") will lead to a "unexpected end of file" error message. However, it is easy to access most of the data in this file by compressing "on the fly" and redirecting the data stream to a file. For example, if "output.dat.gz" is truncated, use

			gzip -dc output.dat.gz > output.dat

			This will yield the same error message as "gzip -d output.dat.gz" but now the decompressed file contents until close to the truncation point will be saved to "dump.dat"

Administrator: Created page with " * Save disk/tape space * Faster writing of data (the CPU time expenditure for the compression is negligible compared to the time save by writing smaller files) * File corrup..."

2014-10-29T17:27:13Z

Created page with " * Save disk/tape space * Faster writing of data (the CPU time expenditure for the compression is negligible compared to the time save by writing smaller files) * File corrup..."

New page

* Save disk/tape space
* Faster writing of data (the CPU time expenditure for the compression is negligible compared to the time save by writing smaller files)
* File corruption is detected more easily
* Easier transfer to other computer systems