Commit Graph

9659 Commits

Author SHA1 Message Date
Zooko O'Whielacronx
10f3312ad6 pyfec: fix docstring 2007-01-30 10:46:40 -07:00
Zooko O'Whielacronx
37e234ebc2 pyfec: fix preconditions, tighten internal C types, fix bugs in the file-encoding and benchmarking utility functions 2007-01-30 10:37:35 -07:00
Zooko O'Whielacronx
96a1cc8e93 pyfec: documentation and licensing 2007-01-30 10:36:42 -07:00
Zooko O'Whielacronx
9e49ea0249 pyfec: COPYING 2007-01-30 10:28:26 -07:00
Zooko O'Whielacronx
1a4ff66557 pyfec: add fec/__init__.py to import all the right names into the fec module namespace 2007-01-26 20:25:36 -07:00
Zooko O'Whielacronx
559411f96d pyfec: delete m-k of the tempfiles at random in the benchmark of the to/from files 2007-01-26 20:22:13 -07:00
Zooko O'Whielacronx
1f1b2a03b7 pyfec: execute tests on import of test_pyfec.py only if it is the __main__ 2007-01-26 20:17:59 -07:00
Zooko O'Whielacronx
9a39ec9f4e pyfec: weaken preconditions -- you can have up to 256 shares, and you don't have to pad the individual shares except to make them all the same length 2007-01-26 20:16:12 -07:00
Zooko O'Whielacronx
57779e2796 pyfec: move benchmark code from test_pyfec.py into the new bench_pyfec.py and add more benchmark code 2007-01-26 20:15:49 -07:00
Zooko O'Whielacronx
f377f0e466 pyfec: add utility functions to encode into a bunch of temp files and to decode from a bunch of tempfiles 2007-01-26 20:15:27 -07:00
Zooko O'Whielacronx
fd8ebcaf6d pyfec: trivial formatting tweak 2007-01-26 20:10:53 -07:00
Zooko O'Whielacronx
81a4d96533 pyfec: remove (optimize) unused code 2007-01-26 20:10:28 -07:00
Zooko O'Whielacronx
afa4d8a46f pyfec: make README.txt much more detailed and rename some internal variables and add some docstrings 2007-01-26 20:08:50 -07:00
Zooko O'Whielacronx
0556e2dba8 rename the C extension from "fec" to "_fec" 2007-01-26 19:58:04 -07:00
Brian Warner
e4870a432c fix some of the filetree/workqueue interface definitions 2007-01-26 17:31:24 -07:00
Zooko O'Whielacronx
f62c5f9c5d pyfec: add precondition checks on the values of k and m to constructors 2007-01-25 18:47:04 -07:00
Zooko O'Whielacronx
3e6ef757d3 pyfec: trivial formatting 2007-01-25 18:46:57 -07:00
Zooko O'Whielacronx
54fdcc2626 add README.txt 2007-01-25 18:46:37 -07:00
Zooko O'Whielacronx
643c4994c0 pyfec: improve package metadata 2007-01-25 18:08:29 -07:00
Zooko O'Whielacronx
8d8cf553c3 add copyright notices, license, attributions 2007-01-25 18:02:16 -07:00
Zooko O'Whielacronx
d339b95895 add dummy function to see how fast we can read in a file and invoke a Python function on each segment 2007-01-25 17:50:26 -07:00
Zooko O'Whielacronx
d893e656df add benchmark 2007-01-25 17:33:48 -07:00
Zooko O'Whielacronx
b398a8210b fix tests to not require pyutil and other tweaks 2007-01-25 15:25:34 -07:00
Zooko O'Whielacronx
2347c94106 add utility method file_fec 2007-01-25 15:25:19 -07:00
Zooko O'Whielacronx
b4f30afe3d use buffers as generic "read" buffers instead of char buffers
This is a typing kludge -- if your buffers have elements of size > 1 then we will be processing only a subset of the elements and treating each byte of the element as a separate entry.
Oh well.
2007-01-25 15:24:01 -07:00
Zooko O'Whielacronx
dcfdee5f25 allow any sequence as input, not just List 2007-01-25 15:23:19 -07:00
Zooko O'Whielacronx
b9c9b9f7b4 add warning of subtle gotcha that can happen with mutable data buffers 2007-01-25 15:22:43 -07:00
Brian Warner
ef73ebaf0a download: update all users to match Zooko's change to ICodecDecoder.decode (as it now returns a list instead of a single string) 2007-01-24 17:23:22 -07:00
Zooko O'Whielacronx
833ef86a88 update interfaces and docs for codec
It now takes a sequence of buffers instead of a single string for both encode and decode, and it also takes a separate sequence of shareids for decode instead of a sequence of tuples, and it returns a sequence of buffers instead of a single string.
2007-01-24 15:34:02 -07:00
Zooko O'Whielacronx
020dd3f6d2 add comment 2007-01-24 15:21:45 -07:00
Zooko O'Whielacronx
06835e5a2b nicer API -- you don't have to shuffle the shares into place before calling decode_all()
Instead it does that shuffling in C inside fecmodule.
2007-01-24 15:20:53 -07:00
Zooko O'Whielacronx
2b03427f19 stricter typing -- using unsigned char for indexes into shares 2007-01-24 15:17:51 -07:00
Brian Warner
c27564a3e2 hush pyflakes 2007-01-24 15:29:22 -07:00
Brian Warner
bbf188d2c6 filetree: make delete() work 2007-01-24 15:10:53 -07:00
Zooko O'Whielacronx
37ede96167 pyfec v0.9
Here is the change history from the first darcs era, in reverse chronological
order:

Mon Jan 22 16:12:56 MST 2007  "Zooko O'Whielacronx <zooko@zooko.com>"
  * move everything into a subdirectory so that I can merge this darcs repo with the tahoe darcs repo
  

     ./fec -> ./pyfec/fec
     ./setup.py -> ./pyfec/setup.py
    A ./pyfec/

Mon Jan 22 16:10:17 MST 2007  "Zooko O'Whielacronx <zooko@zooko.com>"
  * clean up and minimize fec.c
   * strip out unused code
   * hard-code GF_BITS to 8
   * reindent and reformat curly bracket placement
  

    M ./fec/fec.c -655 +324
    M ./fec/fec.h -25

Mon Jan 22 14:24:32 MST 2007  "Zooko O'Whielacronx <zooko@zooko.com>"
  * change API to allow a subset of the shares to be produced, and to just pass back pointers to primary shares instead of copying them

    M ./fec/fec.c -24 +40
    M ./fec/fec.h -5 +17
    M ./fec/fecmodule.c -63 +144
    M ./fec/test/test_pyfec.py -16 +25
    M ./setup.py -2 +27

Tue Jan 16 23:01:44 MST 2007  "Zooko O'Whielacronx <zooko@zooko.com>"
  * split encoder from decoder

    M ./fec/fecmodule.c -48 +161
    M ./fec/test/test_pyfec.py -3 +4

Tue Jan 16 14:35:25 MST 2007  "Zooko O'Whielacronx <zooko@zooko.com>"
  * it compiles now!

     ./fec.c -> ./pyfec/fec.c
     ./fec.h -> ./pyfec/fec.h
     ./fecmodule.c -> ./pyfec/fecmodule.c
     ./pyfec -> ./fec
    M ./fec/fec.c -109 +85 r13
    M ./fec/fec.h -3 +2 r13
    M ./fec/fecmodule.c -23 +241 r13
    A ./fec/test/
    A ./fec/test/test_pyfec.py
    A ./pyfec/
    A ./setup.py

Tue Jan  9 10:47:58 MST 2007  zooko@zooko.com
  * start of new fecmodule.c

    A ./fecmodule.c

Mon Jan  1 15:00:04 MST 2007  zooko@zooko.com
  * tidy up error handling

    M ./fec.c -26 +16

Mon Jan  1 14:06:30 MST 2007  zooko@zooko.com
  * remove the on-the-fly encoding option
  We don't currently need it.

    M ./fec.c -68
    M ./fec.h -22

Mon Jan  1 13:53:28 MST 2007  zooko@zooko.com
  * original import from Mnet project

    A ./fec.c
    A ./fec.h
2007-01-22 17:17:31 -07:00
Brian Warner
3c1bfbe746 Makefile: add target to count lines of code and TODOs 2007-01-22 01:17:39 -07:00
Brian Warner
22731125f3 filetree: change the way addpath works, now we add workqueue steps for all involved subtrees at about the same time, rather than letting one step add the next when it runs. Finally add a (passing) test for uploading files to CHK-based directories 2007-01-22 01:06:09 -07:00
Brian Warner
9dc1c0cfc0 filetree: add vdrive upload/download test, change workqueue relative-filename semantics 2007-01-21 16:03:15 -07:00
Brian Warner
5453e0f022 queen.sendOnly: ignore DeadReferenceError too 2007-01-21 16:01:09 -07:00
Brian Warner
81d093b649 move IWorkQueue into allmydata.interfaces, give VirtualDrive an uploader 2007-01-21 15:15:31 -07:00
Brian Warner
430b3a03fc move upload/download interfaces to allmydata.interfaces, let SubTreeMaker assert IDownloader-ness of its 'downloader' argument 2007-01-21 15:01:34 -07:00
Brian Warner
a8ecaf45b6 test_filetree: more vdrive._get_file_uri() coverage 2007-01-21 13:39:40 -07:00
Brian Warner
124d531b41 filetree: mark leaf nodes by adding is_leaf_subtree(), stop traversing when we hit them, to make vdrive._get_file_uri() work 2007-01-21 13:31:16 -07:00
Brian Warner
9e7dbf20bc filetree: test NoSuchDirectoryError in vdrive.list() 2007-01-21 04:50:37 -07:00
Brian Warner
7b8c524d7c more filetree, workqueue-boxes now hold serialized Nodes, move NodeMaker out to a separate module 2007-01-21 04:18:54 -07:00
Brian Warner
324033c9e2 filetree: start testing IVirtualDrive, beginning with list() 2007-01-20 21:14:41 -07:00
Brian Warner
8c7d33f4a2 filetree: put SubTreeMaker and NodeMaker in separate classes 2007-01-20 17:04:56 -07:00
Brian Warner
ce4610c3e6 more filetree work, more tests now pass 2007-01-20 15:50:21 -07:00
Brian Warner
b61a4ff371 more filetree, it's actually starting to make sense now 2007-01-20 14:41:51 -07:00
Brian Warner
48216ce272 filetable: oops, os.listdir() does not guarantee sorted results 2007-01-20 13:02:26 -07:00