ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-19 21:17:54 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

pyfec: add bragging about how fec is way faster than "par2"

Browse Source
This commit is contained in:
Zooko O'Whielacronx 2007-04-14 17:47:48 -07:00
parent 049d8d1576
commit 502c979307
1 changed files with 22 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
pyfec/README.txt
View File

@ -78,12 +78,12 @@ input to the encoding step.
The bin/ directory contains two Unix-style, command-line tools "fec" and
"unfec". Execute "fec --help" or "unfec --help" for usage instructions.
Note: a Unix-style tool like "fec" does only one thing -- in this case erasure
coding -- and leaves other tasks to other tools. Other Unix-style tools that
go well with "fec" include "GNU tar" for packaging up multiple files and
directories into one bundle, "rzip" for compression, and "GNU Privacy Guard"
for encryption. It is best to do things in that order: first package, then
compress, then encrypt, then erasure code.
Note: a Unix-style tool like "fec" does only one thing -- in this case
erasure coding -- and leaves other tasks to other tools. Other Unix-style
tools that go well with "fec" include "GNU tar" for packaging up multiple
files and directories into one bundle, "rzip" for compression, and "GNU
Privacy Guard" for encryption. It is important to do things in order: first
package, then compress, then encrypt, then erasure code.
* API
@ -157,18 +157,30 @@ objects (e.g. Python strings) to hold the data that you pass to pyfec.
* Utilities
The filefec.py module which has a utility function for efficiently reading a
file and encoding it piece by piece.
The filefec.py module has a utility function for efficiently reading a file
and encoding it piece by piece. This module is used by the "fec" and "unfec"
command-line tools from the bin/ directory.
* Dependencies
A C compiler is required. For the Python API, we have tested it with Python
v2.4 and v2.5.
A C compiler is required. To use the Python API or the command-line tools a
Python interpreter is also required. We have tested it with Python v2.4 and
v2.5.
* Performance Measurements
On my Athlon 64 2.4 GHz workstation (running Linux), the "fec" command-line
tool encoded a 160 MB file with m=100, k=94 (about 6% redundancy) in 3.9
seconds, where the "par2" tool encoded the file with about 6% redundancy in
27 seconds. "fec" encoded the same file with m=12, k=6 (100% redundancy) in
4.1 seconds, where par2 encoded it with about 100% redundancy in 7 minutes
and 56 seconds.
The underlying C library in benchmark mode encoded from a file at about
4.9 million bytes per second and decoded at about 5.8 million bytes per second.
On Peter's fancy Intel Mac laptop (2.16 GHz Core Duo), it encoded from a file
at about 6.2 million bytes per second.
@ -178,10 +190,6 @@ at about 6.8 million bytes per second.
On my old PowerPC G4 867 MHz Mac laptop, it encoded from a file at about 1.3
million bytes per second.
On my Athlon 64 2.4 GHz workstation (running Linux), it encoded from a file at
about 4.9 million bytes per second and decoded at about 5.8 million bytes per
second.
* Acknowledgements