Commit Graph

2095 Commits

Author SHA1 Message Date
b90ee03fd9 tasks must use pools not VMs (#1105)
using config.vm depricated prior to 1.0.0
2021-07-23 14:10:51 +00:00
55366e751a allow pools & scalesets set to shutdown to halt (#1104)
Currently, if a pool or scaleset is set to `shutdown`, it cannot be set to `halt`.

While moving from `halt` to `shutdown` would cause issues, moving from `shutdown` to `halt` is fine.
2021-07-23 13:14:47 +00:00
5be9c4dcee relay SignalR integrations through a storage queue (#1100)
The SignalR integration from Azure Functions does not have automatic retry.  When the SignalR instance has issues, all other APIs fail.

To make the service resilient to SignalR outages, this bounces SignalR events through an Azure Storage queue.

NOTE: This PR does not remove the integration from all of the functions.  That is intended to be done as a follow-on PR.
2021-07-22 18:10:20 +00:00
ee3d0871f2 handle azure-mgmt expired auth tokens by clearing the client cache and retrying (#1099)
In order to reduce how frequently the IMS is hit from the service, the service caches the azure-mgmt clients between API calls.  While the management APIs should have some amount of authentication expiration redundancy built in, not all of them do.

This is seen with `ClientAuthenticationError`, most often with the nested exception record of `ExpiredAuthenticationToken`.

This wraps all of the compute layer functionality with a wrapper that checks if there has been an exception, and retries the request.
2021-07-22 18:01:02 +00:00
3289644d2b automatically clean PII after retention period (#1051)
This PR removes PII from Jobs, Tasks, and Repros after 18 months.  

This PR also removes notifications tied to a container that has not been used in a task for 18 months.  This is done due to notifications having arbitrarily complex mechanisms for storing PII (typically the "assignee").
2021-07-22 17:12:08 +00:00
e359ed8c8f release 2.28.0 (#1101) 2.28.0 2021-07-21 18:25:43 -04:00
d50c5e0d09 enable the supervisor to handle longer service outages (#1098) 2021-07-21 17:03:06 -04:00
75d71209b8 verify {input} is used by target_env or target_options in coverage task (#1097) 2021-07-21 16:41:56 -04:00
3269dbb1aa delete secret on object delete (#1085) 2021-07-21 16:04:27 -04:00
041d3cc204 ignore table exists errors from integration testing at the azure functions layer (#1096) 2021-07-21 13:29:13 -04:00
198d765cb4 Add generic_analysis example that performs coverage analysis (#1072)
This adds an example script and tool that enables LLVM source-based coverage using the `generic_analysis` task.

This provides:
1. sample python script that launches the template and then the analysis task
1. sample `analysis_exe` wrapper script that launches the LLVM coverage tools
1. sample libfuzzer target for the example
1. walk through submitting the jobs and inspecting the results
2021-07-21 17:12:24 +00:00
f151591322 update python dependencies (#1086) 2021-07-21 12:07:52 -04:00
065272191e Replace notifications by default (#1084) 2021-07-20 18:39:31 -04:00
2691d91d8f add error context to generator tasks (#1094) 2021-07-20 18:39:13 -04:00
694df5de5a identify ASAN format string alerts (#1093) 2021-07-20 18:37:59 -04:00
da50ad9711 add coverage to local commands (#1091) 2021-07-20 17:55:04 -04:00
1bd4f33d83 include asan_text in log message as intended (#1092) 2021-07-20 13:48:28 -04:00
9c30183036 address upcoming clippy warnings (#1089) 2021-07-20 13:19:50 -04:00
77aaa5b88c use enum for local commands (#1090) 2021-07-20 12:14:12 -04:00
26654b925b address non-utf8 errors in testing (#1088) 2021-07-20 11:47:17 -04:00
e065b571f7 add format string bug to example integration test (#1087) 2021-07-20 05:00:45 -04:00
152dd190b7 Add more information to the logs of transient error (#1082) 2021-07-16 17:52:06 -04:00
3305732160 increase libfuzzer integration test -runs (#1081) 2021-07-16 17:29:37 -04:00
853287dc23 release 2.27.0 (#1077) 2.27.0 2021-07-16 13:52:52 -04:00
58e57621d1 add a short sleep in the supervisor's event loop to reduce CPU usage (#1080) 2021-07-16 13:32:26 -04:00
517aa54a59 Remove Stream impl for DirectoryMonitor (#1078) 2021-07-16 13:10:53 -04:00
db66a1d3c5 check change event type before container name (#1075) 2021-07-15 19:57:41 -04:00
f769364335 sync batch processed results after every 10 inputs (#1076) 2021-07-15 19:27:48 -04:00
94b2be1a3e release 2.26.1 (#1070) 2.26.1 2021-07-15 13:46:32 -04:00
59fea7e3b7 extend requests read timeout (#1068) 2021-07-14 15:33:28 -04:00
c724f2dd99 fix autoscale pool size check (#1067) 2021-07-14 15:09:04 -04:00
1121ebc6e8 fix issue specifying target_options for libfuzzer targets (#1066) 2021-07-14 14:38:35 -04:00
32d886a2cd release 2.26.0 (#1065) 2.26.0 2021-07-14 10:02:48 -04:00
808f4e4dd6 attempt to log panic stack trace to the supervisor failure log (#1062) 2021-07-13 19:23:25 -04:00
d2ec92afd1 provide more context on upload failure (#1063) 2021-07-13 18:48:13 -04:00
36d682aadf make invalid AZCOPY environment variables more clear (#1061) 2021-07-13 15:07:57 -04:00
39beb1591c use managed identity reader access for scaleset configs (#1060) 2021-07-13 13:20:50 -04:00
7a7ded6b7e force upgrade custom script extensions (#1059) 2021-07-13 12:08:07 -04:00
3092a28007 release 2.25.1 (#1057) 2.25.1 2021-07-09 14:41:18 -04:00
2b32779017 Update onefuzz supervisor service restart behavior (#1055) 2021-07-09 13:56:58 -04:00
89b7d13125 Fix get_dead_nodes query (#1054) 2021-07-09 13:33:42 -04:00
48978ce881 release 2.25.0 (#1053) 2.25.0 2021-07-08 19:13:29 -04:00
0bb75344e3 update 3rd-party rust dependencies (#1052) 2021-07-08 11:25:15 -04:00
826ef8dd22 Pool shrink queue (#1050) 2021-07-08 10:23:54 -04:00
45d468f2ce set pool_id on node creation (#1049) 2021-07-07 17:58:24 -04:00
5ba04d0cd2 use pydantic built-in size validators (#1048) 2021-07-07 17:07:05 -04:00
52f83b5b26 add EventScalesetResizeScheduled (#1047) 2021-07-07 14:15:26 -04:00
7b2679a1ce make ShrinkQueue not scaleset specific (#1046) 2021-07-07 13:27:49 -04:00
edecbbacf6 Adding timestamp to notification model to query from storage table. (#1043) 2021-07-07 12:41:45 -04:00
15063908b0 update azure-cli to 2.26.0 (#1045) 2021-07-07 12:07:34 -04:00