Commit Graph

7 Commits

Author SHA1 Message Date
3b26ffef65 support multiple corpus accounts (#334)
Add support for sharding across multiple storage accounts for blob containers used for corpus management.

Things to note:

1. Additional storage accounts must be in the same resource group, support the "blob" endpoint, and have the tag `storage_type` with the value `corpus`.  A utility is provided (`src/utils/add-corpus-storage-accounts`), which adds storage accounts. 
2. If any secondary storage accounts exist, they are used by default for containers.
3. Storage account names are cached in memory the Azure Function instance forever.   Upon adding new storage accounts, the app needs to be restarted to pick up the new accounts.
2021-01-06 23:11:39 +00:00
e47e89609a Use Storage Account types, rather than account_id (#320)
We need to move to supporting data sharding.

One of the steps towards that is stop passing around `account_id`, rather we need to specify the type of storage we need.
2020-11-18 14:06:14 +00:00
b5578381ce default TTL for queued messages to infinite (#259) 2020-11-04 15:41:05 -05:00
d4c584342a address multiple issues found by pylint (#206) 2020-10-26 12:24:50 -04:00
e308a4ae1e refactor node state to fully put the agent in charge (#90) 2020-10-03 02:43:04 -04:00
f72543dde2 cleanup logging in multiple functions (#73) 2020-10-01 23:31:08 -04:00
d3a0b292e6 initial public release 2020-09-18 12:21:04 -04:00