2148 Commits

Author SHA1 Message Date
bmc-msft
a02e084522
split out node, scaleset, and pool code (#507) 2021-02-04 19:07:49 -05:00
bmc-msft
81263c9065
Release 2.2.0 (#509) 2.2.0 2021-02-04 17:49:35 -05:00
bmc-msft
c5bb0f0588
Update Proxy heartbeat & logging (#502) 2021-02-04 15:38:17 -05:00
bmc-msft
4abfbeca56
mark connection refused errors as an ephemeral error (#506) 2021-02-04 15:27:27 -05:00
Cheick Keita
a50eb94db5
Making input_tester and expand immutable (#500) 2021-02-04 15:18:15 -05:00
bmc-msft
220fe46f9d
disable check_debugger by default (#505) 2021-02-04 14:23:51 -05:00
bmc-msft
1adee4ea83
update pools before nodes or scalesets (#503) 2021-02-04 13:26:19 -05:00
bmc-msft
e0cd3d535b
rename proxy queue endpoint (#504) 2021-02-04 12:45:36 -05:00
bmc-msft
cdfdc2be84
split telemetry into it's own crate (#501)
Splits out telemetry crate such that it can be reused by other components (specifically the proxy-manager) easily.
2021-02-04 14:46:35 +00:00
bmc-msft
0a4278110d
update azure-mgmt-compute to 18.2.0 (#499) 2021-02-04 12:34:29 +00:00
bmc-msft
3094ebb846
Release 2.1.1 (#496) 2.1.1 2021-02-02 16:02:12 -05:00
bmc-msft
02721f3ed9
address clippy issues in agent (#490) 2021-02-02 14:41:27 -05:00
bmc-msft
e60d697040
add telemetry stats utility (#493) 2021-02-02 13:56:34 +00:00
bmc-msft
5e2e9448df
add security auditing of python code using Bandit during CICD (#491) 2021-02-01 16:51:03 -05:00
bmc-msft
0f70ffa3e2
try pushing updates to scaleset configs frequently until the push succeeds (#489) 2021-02-01 10:09:40 -05:00
bmc-msft
e894a523ab
Release 2.1.0 (#487) 2.1.0 2021-01-29 16:26:00 -05:00
Cheick Keita
cc1c79b27d
Fix bug when checking if the default cli application exists (#488) 2021-01-29 15:32:17 -05:00
Cheick Keita
0a1021447b
Return an error when download_input fails (#485) 2021-01-29 17:13:36 +00:00
bmc-msft
5acb59e5b9
Add task_id & job_id to variable expansion (#481)
Fixes #479 

Note, this is built on top of #480
2021-01-29 15:35:59 +00:00
bmc-msft
9c7eb33149
Expand tools_dir accessibility in variable expansion (#480)
Fixes #478
2021-01-29 11:33:23 +00:00
bmc-msft
bcf42485eb
sync new inputs found by supervisor tasks (#484) 2021-01-28 21:33:30 -05:00
bmc-msft
a46f7b4193
expose supervisor tasks that are fully self-contained fuzzing tasks in the service (#474)
Exposes the functionality added in #454 to the service & CLI.

Fixes #439
2021-01-29 00:01:59 +00:00
bmc-msft
14fc1ca51f
remove unused Event generation from the pre-2.0.0 SignalR integration (#477)
Remove a vestige of the adhoc events used by the previous SingalR integration for container updates.
2021-01-28 21:56:31 +00:00
bmc-msft
f155ad625f
reimage long-lived nodes (#476)
This helps keep nodes on scalesets that use `latest` OS image SKUs reasonably up-to-date with OS patches without disrupting running fuzzing tasks with patch reboot cycles.

In combination with the already-merged #416, this PR closes #414.
2021-01-28 20:36:40 +00:00
bmc-msft
98b6626a84
use license rather than license_file for python packages (#472)
Use the correct field for specifying the license for the onefuzz python pacakges following [setup tools documentation](https://packaging.python.org/guides/distributing-packages-using-setuptools/#license)
2021-01-28 17:56:29 +00:00
bmc-msft
07760bf967
Update expansion documentation to describe nested options as implemented (#475)
This expands the documentation to demonstrate how nested lists are handled in practice, as well as to provide examples for how to deal with the nested list expansion.
2021-01-28 17:12:48 +00:00
bmc-msft
2e44a5a22f
update azure-mgmt-network to the latest version (#471) 2021-01-28 07:04:52 +00:00
bmc-msft
81e3ea9895
force set-versions.sh to run in Bash (#473)
Co-authored-by: Brian Caswell <bmc@shmoo.com>
2021-01-27 09:06:12 -08:00
nharper285
24685ca8df
Updating Windows Default Image from RS5-Pro to 20H2-Pro (#469)
RS5-Pro is no longer updated in the Azure Marketplace. In order to ensure the Windows 10 VMs are regularly updated, we need to switch the default image to 20H2-Pro, which is regularly maintained.
2021-01-27 13:46:46 +00:00
bmc-msft
cfcf493a23
add context to command failures (#466)
Fixes #465
2021-01-26 21:29:59 +00:00
bmc-msft
de67c9db63
Release 2.0.0 (#464) 2.0.0 2021-01-26 15:41:29 -05:00
bmc-msft
5027745ee2
simplify get/delete for scalesets (#468) 2021-01-26 14:43:14 -05:00
bmc-msft
165257e989
update python prereqs (#427)
Updates the following libraries in the service:
* azure-core
* azure-functions
* azure-identity
* azure-keyvault-keys
* azure-keyvault-secrets
* azure-mgmt-compute
* azure-mgmt-core
* azure-mgmt-loganalytics
* azure-mgmt-network
* azure-mgmt-resource
* azure-mgmt-storage
* azure-mgmt-subscription
* azure-storage-blob
* azure-storage-queue
* pydantic
* requests
* jsonpatch

Removes the following libraries in the service:
* azure-cli-core
* azure-cli-nspkg
* azure-mgmt-cosmosdb
* azure-servicebus

Updates the following libraries in the CLI:
* requests
* semver
* asciimatics
* pydantic
* tenacity

Updates the following libraries in onefuzztypes:
* pydantic

The primary "legacy" libraries are [azure-graphrbac](https://pypi.org/project/azure-graphrbac/) and azure-cosmosdb-table.  The former has not been updated to use azure-identity yet. The later is being rewritten as [azure-data-tables](https://pypi.org/project/azure-data-tables/), but is still in early beta.
2021-01-25 20:53:40 +00:00
bmc-msft
31ea71e8b6
use the unique-string based keyvault names (#462) 2021-01-25 15:02:12 -05:00
bmc-msft
4bc90a7564
set max stdout/stderr size (#460) 2021-01-25 13:07:35 -05:00
bmc-msft
3ae70cea6b
add keyvault management to administrative role (#389)
This is a prerequisite for #326
2021-01-25 16:40:36 +00:00
Cheick Keita
3f2883d38e
Storing secrets in azure keyvault (#326) 2021-01-25 11:12:07 -05:00
bmc-msft
dc31ffc92b
add support for fully self-contained fuzzers (#454) 2021-01-22 18:20:22 -05:00
bmc-msft
e4ecf7e230
remove early-exit from cleanup_nodes that broke dead node cleanup (#458) 2021-01-22 18:04:50 -05:00
bmc-msft
42337587ad
Update QueueClient with latest SAS URL upon renewing registration (#459) 2021-01-22 17:41:42 -05:00
Cheick Keita
ec982c68c5
Update the registration logic to print manual steps when adal authentication fails (#447)
Mitigate the deployment issue related to the conditional access policy.
The registration logic is updated to use the old rbac python library when possible. 
The deployment will print some manual step for operations that cannot be automated
2021-01-22 22:21:43 +00:00
bmc-msft
2f3139cda1
unify node resetting & deleting into delete/recreate (#450) 2021-01-22 22:04:44 +00:00
bmc-msft
c0a4b0dba4
remove workaround for an issue addressed in latest mypy (#455) 2021-01-22 14:00:35 -05:00
Joe Ranweiler
3c76baa3bb
Use goblin for more direct ELF handling (#453)
- Switch to using `goblin` for both ELF and PE parsing
- Refactor block entry point recovery, with better documentation
- Fix a broken example binary

Co-authored-by: bmc-msft <41130664+bmc-msft@users.noreply.github.com>
2021-01-21 15:00:19 -08:00
bmc-msft
4f941d5ff2
add the task_id to the integration failure logs (#446) 2021-01-21 13:35:54 -05:00
bmc-msft
95bd4ac308
add unit test to verify functionality of nested list expansion (#448) 2021-01-21 12:05:17 -05:00
bmc-msft
e6dec041b2
move to using machine_id rather than node_id (#451)
Handle unifying onto machine_id for NodeMessage.
2021-01-21 16:22:16 +00:00
bmc-msft
7e6415b15f
ensure consistency in doc generation by always sorting json keys (#449) 2021-01-21 15:28:11 +00:00
bmc-msft
b499b9b17d
always set SignalR events at the end of handlers (#445)
Addresses an issue where events meant for SignalR do not get sent to the service when an App Service instance spins down before a timer event fires.
2021-01-21 10:31:02 +00:00
bmc-msft
f3d81566e3
add expanding {tools_dir} to supervisor_options and supervisor_env (#444)
Fixes #443
2021-01-20 20:34:25 +00:00