In practice, Application Insights can take up to 3 minutes before something sent to it is available via KQL.
This PR logs a start and stop marker such that the integration tests only search for logs during the integration tests. This reduces the complexity when using the integration tests during the development process.
Note: this migrated the new functionality from #356 into the latest integration test tools.
Add support for sharding across multiple storage accounts for blob containers used for corpus management.
Things to note:
1. Additional storage accounts must be in the same resource group, support the "blob" endpoint, and have the tag `storage_type` with the value `corpus`. A utility is provided (`src/utils/add-corpus-storage-accounts`), which adds storage accounts.
2. If any secondary storage accounts exist, they are used by default for containers.
3. Storage account names are cached in memory the Azure Function instance forever. Upon adding new storage accounts, the app needs to be restarted to pick up the new accounts.
We need to move to supporting data sharding.
One of the steps towards that is stop passing around `account_id`, rather we need to specify the type of storage we need.
