Commit Graph

947 Commits

Author SHA1 Message Date
edecbbacf6 Adding timestamp to notification model to query from storage table. (#1043) 2021-07-07 12:41:45 -04:00
15063908b0 update azure-cli to 2.26.0 (#1045) 2021-07-07 12:07:34 -04:00
636d267585 update azure-cli to 2.25.0 (#922) 2021-07-07 13:28:06 +00:00
aa4ed2893e update 3rd-party Rust dependencies (#1040) 2021-07-06 11:57:01 -04:00
51ddaaa747 only remove authenticated users access to ssh keys if they have permissions (#1038) 2021-07-02 13:45:34 -04:00
314fb18ca8 save the EventType enum name for status top (#1037) 2021-07-02 12:17:28 -04:00
a43f693761 Provide minimized stack lines (#993)
Using the function name only method for minimization results in a
function with two different malloc related bugs to get bucketed
together.

This provides a minimized stack without function address of stack depth.

For entries without source information, such as:
`# 34 0xf19113f in ChromeMain+0x13f (C:\\clusterfuzz\\bot\\builds\\chrome-test-builds_media_win32-release\\revisions\\asan-win32-release-335593\\chrome_child.dll+0x113f)`
users will see this:
`ChromeMain+0x13f`

For entries with source information, such as:
`# 20 0x58bd3bb in v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) ../../v8/src/builtins/builtins-api.cc:137:5`
users will see this:
`v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) builtins-api.cc:137:5`
2021-07-01 21:14:31 +00:00
9261e1f883 update from google/clusterfuzz c9c3623e9307dfb058f4328d9f85f7988468e75a (#1036) 2021-07-01 11:58:54 -04:00
cc60b2e946 release 2.24.0 (#1035) 2.24.0 2021-06-30 19:05:56 -04:00
80387bd8ca Use sync pointers for writing ModuleCache off-thread (#1033) 2021-06-30 18:10:03 -04:00
19089514b4 Don't panic on out-of-bounds bit set (#1034) 2021-06-30 17:47:40 -04:00
29dda54b83 instance wide configuration (#1010)
TODO:
* [x] add setting initial set of admins during deployment
2021-06-30 21:13:58 +00:00
1e90ed6092 Allow notifications to be retried when an error occurs (#1026) 2021-06-30 14:05:25 -04:00
a1502b7d58 move to using rustls (#1029) 2021-06-30 12:27:43 -04:00
59ad1f9b9b save the agents in a per-$(uname) directory (#1030) 2021-06-30 11:33:05 -04:00
2f71689d48 initial support for running the agent locally on OSX (#1016) 2021-06-29 16:20:23 -04:00
b5388d360c add min_length specifications to Github Issue Template configs (#1019) 2021-06-25 18:04:25 +00:00
bb7419186f bump github actions cache key IDs (#1024) 2021-06-25 13:32:48 -04:00
6a0c8e656b release 2.23.1 (#1025) 2.23.1 2021-06-25 12:19:19 -04:00
883c93aaf4 ensure VM IDs are unique before calling Azure reimage/delete APIs (#1023) 2021-06-25 11:54:52 -04:00
880039a617 release 2.23.0 (#1020) 2.23.0 2021-06-24 16:25:08 -04:00
5f72cc20b8 Fix issue when deserializing GithubIssueTemplate with hidden secret (#1008) 2021-06-24 13:50:54 -04:00
4c9a9f2948 address issues from latest clippy (#1018) 2021-06-24 12:13:16 -04:00
27b434e996 fix debug report (#1011)
The debug report created by the command `onefuzz debug  notification job <job id>`  is causing a crash in the regression task 
```
error running task: libfuzzer regression

Caused by:
    0: handling crash reports
    1: unable to parse crash report: fake-crash-sample.json
    2: unable to parse report: task_unique_reports_2/fake-crash-sample.json - "{\"input_url\": null, \"input_blob\": {\"account\": \"fuzz27ee6imdmr5gy\", \"container\": \"oft-crashes-cecbd958a1f257688f9768edaaf6c94d\", \"name\": \"fake-crash-sample\"}, \"executable\": \"fuzz.exe\", \"crash_type\": \"fake crash report\", \"crash_site\": \"fake crash site\", \"call_stack\": [\"#0 fake\", \"#1 call\", \"#2 stack\"], \"call_stack_sha256\": \"0000000000000000000000000000000000000000000000000000000000000000\", \"input_sha256\": \"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\", \"asan_log\": \"fake asan log\", \"task_id\": \"b1107de0-c3cb-43ff-ab68-5accc579f4d4\", \"job_id\": \"afa45e3e-9a75-4a47-8d59-ef3154599fc7\", \"scariness_score\": null, \"scariness_description\": null, \"minimized_stack\": null, \"minimized_stack_sha256\": null, \"minimized_stack_function_names\": null, \"minimized_stack_function_names_sha256\": null}"
```
2021-06-24 00:57:58 +00:00
10d2e3e366 update azure-keyvault-secrets to 4.3.0 (#1012) 2021-06-23 18:27:32 -04:00
5f8e423265 remove nodes from db upon reimage (#1005)
The flag `Node.reimage_queued` is intended to stop nodes from reimaging repeatedly.  

In #970, in order to work around Azure API failures, this flag was cycled if the node was already set to cleanup.  Unfortunately, reimaging can sometimes take a significant amount of time, causing this change to get nodes multiple times.

Instead of using `reimage_queued` as a flag, this PR deletes the node from the storage table upon reimage.  When the node registers OR the next time through `Scaleset.cleanup_nodes`, the Node will be recreated automatically, whichever comes first.
2021-06-23 22:25:15 +00:00
7c549e7fbb use the refreshed token when retrying after auth failure (#1017) 2021-06-23 18:03:51 -04:00
50652c2e48 mark tasks as failed when the node is being reimaged due to heartbeat issues (#1015) 2021-06-23 16:39:47 -04:00
97cba8eda1 update third-party Rust dependencies (#1009) 2021-06-22 11:51:38 -04:00
778f09efaf normalize log retention (#1007) 2021-06-22 07:59:17 -04:00
f450e937a8 update rust dependencies (#1004) 2021-06-18 12:58:13 -04:00
c8862c7f18 add cooloff period for rapidly exiting libFuzzer targets (#1002) 2021-06-18 11:40:36 -04:00
888c0459ad release 2.22.0 (#1001) 2.22.0 2021-06-17 17:54:47 -04:00
cb895d09c7 fix GithubIssueTemplate deserialization (#990) 2021-06-17 13:02:28 -04:00
2ff9c47e0c Explicitly track debuggee thread state to detect thread exit/debug event races (#1000)
- Define an enum to track the debugger's current understanding of debuggee thread state
- Update our private suspend/resume methods to update and return the current state
- Detect thread exit/debug event races in suspend/resume calls
2021-06-16 14:45:43 -07:00
a58df95573 reduce log retention duration to 30 days (#997) 2021-06-16 17:12:53 -04:00
5aba12716a fix SyncedDir.remote_url issue with relative paths (#996) 2021-06-16 14:13:06 -04:00
5efc4f959c add context to errors in syncdir (#995) 2021-06-16 12:49:24 -04:00
c71ce580bd updates from upstream libclusterfuzz (#992) 2021-06-15 16:32:13 -04:00
0d709ed5f8 fix markdown link (#989) 2021-06-14 16:40:04 -04:00
93cfedbd2e fix scaleset identity name in the documentation (#987) 2021-06-14 16:10:14 -04:00
ef5eb12abe set azcopy retry notice to debug rather than warn (#986) 2021-06-14 15:38:41 -04:00
b9950c5526 update log messages to ease debugging (#988) 2021-06-14 15:18:03 -04:00
4472d584ac handle serialization of secrets sent from the CLI (#985)
This normalizes the SecretData serialization from the client to address #981.

When serializing objects sent to the service with secrets, we would turn it into a SecretData

We use SecretData to convert this:
`{"auth": {"user": "A", "personal_access_token": "B"}}`
to this:
`"auth": { "secret": { "url": "https://KEYVAULT-URL" }}`

Currently, in the case we have a SecretData we've not yet saved, the serialized form looks like this:
`{"auth": { "secret": {"user": "A", "personal_access_token": "B"}}}`

This PR simplifies the client side serialization to this:
`{"auth": {"user": "A", "personal_access_token": "B"}}`
2021-06-12 14:39:14 +00:00
bcdae2d5cb Check scaleset size for missing nodes (#984) 2021-06-11 18:47:21 -04:00
c0b3a409e4 add monkeypatch to hotfix pydantic Union issues (#982)
Until Pydantic supports discriminated or "smart" unions, we need to work around the coercion issue impacting unions in our models.

This reuses the "smart union" implementation from https://github.com/samuelcolvin/pydantic/pull/2092
2021-06-11 22:00:06 +00:00
a0bce1a538 fix markdown link (#983) 2021-06-11 08:09:28 -04:00
75cc11af3a release 2.21.0 (#980) 2.21.0 2021-06-10 15:12:21 -04:00
1a059f2c40 Add context to coverage tracing errors (#979) 2021-06-10 12:41:32 -04:00
b39f0964c9 Retry coverage recording (#978) 2021-06-10 10:40:32 -04:00