Commit Graph

1141 Commits

Author SHA1 Message Date
a43f693761 Provide minimized stack lines (#993)
Using the function name only method for minimization results in a
function with two different malloc related bugs to get bucketed
together.

This provides a minimized stack without function address of stack depth.

For entries without source information, such as:
`# 34 0xf19113f in ChromeMain+0x13f (C:\\clusterfuzz\\bot\\builds\\chrome-test-builds_media_win32-release\\revisions\\asan-win32-release-335593\\chrome_child.dll+0x113f)`
users will see this:
`ChromeMain+0x13f`

For entries with source information, such as:
`# 20 0x58bd3bb in v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) ../../v8/src/builtins/builtins-api.cc:137:5`
users will see this:
`v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) builtins-api.cc:137:5`
2021-07-01 21:14:31 +00:00
9261e1f883 update from google/clusterfuzz c9c3623e9307dfb058f4328d9f85f7988468e75a (#1036) 2021-07-01 11:58:54 -04:00
cc60b2e946 release 2.24.0 (#1035) 2.24.0 2021-06-30 19:05:56 -04:00
80387bd8ca Use sync pointers for writing ModuleCache off-thread (#1033) 2021-06-30 18:10:03 -04:00
19089514b4 Don't panic on out-of-bounds bit set (#1034) 2021-06-30 17:47:40 -04:00
29dda54b83 instance wide configuration (#1010)
TODO:
* [x] add setting initial set of admins during deployment
2021-06-30 21:13:58 +00:00
1e90ed6092 Allow notifications to be retried when an error occurs (#1026) 2021-06-30 14:05:25 -04:00
a1502b7d58 move to using rustls (#1029) 2021-06-30 12:27:43 -04:00
59ad1f9b9b save the agents in a per-$(uname) directory (#1030) 2021-06-30 11:33:05 -04:00
2f71689d48 initial support for running the agent locally on OSX (#1016) 2021-06-29 16:20:23 -04:00
b5388d360c add min_length specifications to Github Issue Template configs (#1019) 2021-06-25 18:04:25 +00:00
bb7419186f bump github actions cache key IDs (#1024) 2021-06-25 13:32:48 -04:00
6a0c8e656b release 2.23.1 (#1025) 2.23.1 2021-06-25 12:19:19 -04:00
883c93aaf4 ensure VM IDs are unique before calling Azure reimage/delete APIs (#1023) 2021-06-25 11:54:52 -04:00
880039a617 release 2.23.0 (#1020) 2.23.0 2021-06-24 16:25:08 -04:00
5f72cc20b8 Fix issue when deserializing GithubIssueTemplate with hidden secret (#1008) 2021-06-24 13:50:54 -04:00
4c9a9f2948 address issues from latest clippy (#1018) 2021-06-24 12:13:16 -04:00
27b434e996 fix debug report (#1011)
The debug report created by the command `onefuzz debug  notification job <job id>`  is causing a crash in the regression task 
```
error running task: libfuzzer regression

Caused by:
    0: handling crash reports
    1: unable to parse crash report: fake-crash-sample.json
    2: unable to parse report: task_unique_reports_2/fake-crash-sample.json - "{\"input_url\": null, \"input_blob\": {\"account\": \"fuzz27ee6imdmr5gy\", \"container\": \"oft-crashes-cecbd958a1f257688f9768edaaf6c94d\", \"name\": \"fake-crash-sample\"}, \"executable\": \"fuzz.exe\", \"crash_type\": \"fake crash report\", \"crash_site\": \"fake crash site\", \"call_stack\": [\"#0 fake\", \"#1 call\", \"#2 stack\"], \"call_stack_sha256\": \"0000000000000000000000000000000000000000000000000000000000000000\", \"input_sha256\": \"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\", \"asan_log\": \"fake asan log\", \"task_id\": \"b1107de0-c3cb-43ff-ab68-5accc579f4d4\", \"job_id\": \"afa45e3e-9a75-4a47-8d59-ef3154599fc7\", \"scariness_score\": null, \"scariness_description\": null, \"minimized_stack\": null, \"minimized_stack_sha256\": null, \"minimized_stack_function_names\": null, \"minimized_stack_function_names_sha256\": null}"
```
2021-06-24 00:57:58 +00:00
10d2e3e366 update azure-keyvault-secrets to 4.3.0 (#1012) 2021-06-23 18:27:32 -04:00
5f8e423265 remove nodes from db upon reimage (#1005)
The flag `Node.reimage_queued` is intended to stop nodes from reimaging repeatedly.  

In #970, in order to work around Azure API failures, this flag was cycled if the node was already set to cleanup.  Unfortunately, reimaging can sometimes take a significant amount of time, causing this change to get nodes multiple times.

Instead of using `reimage_queued` as a flag, this PR deletes the node from the storage table upon reimage.  When the node registers OR the next time through `Scaleset.cleanup_nodes`, the Node will be recreated automatically, whichever comes first.
2021-06-23 22:25:15 +00:00
7c549e7fbb use the refreshed token when retrying after auth failure (#1017) 2021-06-23 18:03:51 -04:00
50652c2e48 mark tasks as failed when the node is being reimaged due to heartbeat issues (#1015) 2021-06-23 16:39:47 -04:00
97cba8eda1 update third-party Rust dependencies (#1009) 2021-06-22 11:51:38 -04:00
778f09efaf normalize log retention (#1007) 2021-06-22 07:59:17 -04:00
f450e937a8 update rust dependencies (#1004) 2021-06-18 12:58:13 -04:00
c8862c7f18 add cooloff period for rapidly exiting libFuzzer targets (#1002) 2021-06-18 11:40:36 -04:00
888c0459ad release 2.22.0 (#1001) 2.22.0 2021-06-17 17:54:47 -04:00
cb895d09c7 fix GithubIssueTemplate deserialization (#990) 2021-06-17 13:02:28 -04:00
2ff9c47e0c Explicitly track debuggee thread state to detect thread exit/debug event races (#1000)
- Define an enum to track the debugger's current understanding of debuggee thread state
- Update our private suspend/resume methods to update and return the current state
- Detect thread exit/debug event races in suspend/resume calls
2021-06-16 14:45:43 -07:00
a58df95573 reduce log retention duration to 30 days (#997) 2021-06-16 17:12:53 -04:00
5aba12716a fix SyncedDir.remote_url issue with relative paths (#996) 2021-06-16 14:13:06 -04:00
5efc4f959c add context to errors in syncdir (#995) 2021-06-16 12:49:24 -04:00
c71ce580bd updates from upstream libclusterfuzz (#992) 2021-06-15 16:32:13 -04:00
0d709ed5f8 fix markdown link (#989) 2021-06-14 16:40:04 -04:00
93cfedbd2e fix scaleset identity name in the documentation (#987) 2021-06-14 16:10:14 -04:00
ef5eb12abe set azcopy retry notice to debug rather than warn (#986) 2021-06-14 15:38:41 -04:00
b9950c5526 update log messages to ease debugging (#988) 2021-06-14 15:18:03 -04:00
4472d584ac handle serialization of secrets sent from the CLI (#985)
This normalizes the SecretData serialization from the client to address #981.

When serializing objects sent to the service with secrets, we would turn it into a SecretData

We use SecretData to convert this:
`{"auth": {"user": "A", "personal_access_token": "B"}}`
to this:
`"auth": { "secret": { "url": "https://KEYVAULT-URL" }}`

Currently, in the case we have a SecretData we've not yet saved, the serialized form looks like this:
`{"auth": { "secret": {"user": "A", "personal_access_token": "B"}}}`

This PR simplifies the client side serialization to this:
`{"auth": {"user": "A", "personal_access_token": "B"}}`
2021-06-12 14:39:14 +00:00
bcdae2d5cb Check scaleset size for missing nodes (#984) 2021-06-11 18:47:21 -04:00
c0b3a409e4 add monkeypatch to hotfix pydantic Union issues (#982)
Until Pydantic supports discriminated or "smart" unions, we need to work around the coercion issue impacting unions in our models.

This reuses the "smart union" implementation from https://github.com/samuelcolvin/pydantic/pull/2092
2021-06-11 22:00:06 +00:00
a0bce1a538 fix markdown link (#983) 2021-06-11 08:09:28 -04:00
75cc11af3a release 2.21.0 (#980) 2.21.0 2021-06-10 15:12:21 -04:00
1a059f2c40 Add context to coverage tracing errors (#979) 2021-06-10 12:41:32 -04:00
b39f0964c9 Retry coverage recording (#978) 2021-06-10 10:40:32 -04:00
3ec32160cb fix service authentication refresh errors (#976) 2021-06-09 19:11:18 -04:00
2be1edd9dc handle reimaging failures by resetting reimage_queued (#970)
In a previous commit, reimage_queued was added to prevent reimaging a node while it is reimaging.  However, this means reimaging failures due to Azure issues don't finish reimaging.

This will reset the this flag allowing the node to reimage in the following cleanup cycle.
2021-06-09 18:58:56 +00:00
4f6e4dbbec add types-requests (#973) 2021-06-09 13:01:17 -04:00
da931b3a5c address issues raised from latest mypy (#972) 2021-06-09 12:04:24 -04:00
3794bb0c68 move to wiremock for testing (#971) 2021-06-09 12:04:10 -04:00
78d5a58965 allow updates to a scaleset in the resize state (#969) 2021-06-09 13:10:43 +00:00