Commit Graph

114 Commits

Author SHA1 Message Date
f151591322 update python dependencies (#1086) 2021-07-21 12:07:52 -04:00
065272191e Replace notifications by default (#1084) 2021-07-20 18:39:31 -04:00
59fea7e3b7 extend requests read timeout (#1068) 2021-07-14 15:33:28 -04:00
1121ebc6e8 fix issue specifying target_options for libfuzzer targets (#1066) 2021-07-14 14:38:35 -04:00
36d682aadf make invalid AZCOPY environment variables more clear (#1061) 2021-07-13 15:07:57 -04:00
15063908b0 update azure-cli to 2.26.0 (#1045) 2021-07-07 12:07:34 -04:00
636d267585 update azure-cli to 2.25.0 (#922) 2021-07-07 13:28:06 +00:00
314fb18ca8 save the EventType enum name for status top (#1037) 2021-07-02 12:17:28 -04:00
29dda54b83 instance wide configuration (#1010)
TODO:
* [x] add setting initial set of admins during deployment
2021-06-30 21:13:58 +00:00
27b434e996 fix debug report (#1011)
The debug report created by the command `onefuzz debug  notification job <job id>`  is causing a crash in the regression task 
```
error running task: libfuzzer regression

Caused by:
    0: handling crash reports
    1: unable to parse crash report: fake-crash-sample.json
    2: unable to parse report: task_unique_reports_2/fake-crash-sample.json - "{\"input_url\": null, \"input_blob\": {\"account\": \"fuzz27ee6imdmr5gy\", \"container\": \"oft-crashes-cecbd958a1f257688f9768edaaf6c94d\", \"name\": \"fake-crash-sample\"}, \"executable\": \"fuzz.exe\", \"crash_type\": \"fake crash report\", \"crash_site\": \"fake crash site\", \"call_stack\": [\"#0 fake\", \"#1 call\", \"#2 stack\"], \"call_stack_sha256\": \"0000000000000000000000000000000000000000000000000000000000000000\", \"input_sha256\": \"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\", \"asan_log\": \"fake asan log\", \"task_id\": \"b1107de0-c3cb-43ff-ab68-5accc579f4d4\", \"job_id\": \"afa45e3e-9a75-4a47-8d59-ef3154599fc7\", \"scariness_score\": null, \"scariness_description\": null, \"minimized_stack\": null, \"minimized_stack_sha256\": null, \"minimized_stack_function_names\": null, \"minimized_stack_function_names_sha256\": null}"
```
2021-06-24 00:57:58 +00:00
cb895d09c7 fix GithubIssueTemplate deserialization (#990) 2021-06-17 13:02:28 -04:00
4472d584ac handle serialization of secrets sent from the CLI (#985)
This normalizes the SecretData serialization from the client to address #981.

When serializing objects sent to the service with secrets, we would turn it into a SecretData

We use SecretData to convert this:
`{"auth": {"user": "A", "personal_access_token": "B"}}`
to this:
`"auth": { "secret": { "url": "https://KEYVAULT-URL" }}`

Currently, in the case we have a SecretData we've not yet saved, the serialized form looks like this:
`{"auth": { "secret": {"user": "A", "personal_access_token": "B"}}}`

This PR simplifies the client side serialization to this:
`{"auth": {"user": "A", "personal_access_token": "B"}}`
2021-06-12 14:39:14 +00:00
da931b3a5c address issues raised from latest mypy (#972) 2021-06-09 12:04:24 -04:00
2d377d1423 add onefuzz debug job rerun JOB_ID command (#960)
This allows the specification of an alternate pool or duration when rerunning an existing job.
2021-06-07 20:53:34 +00:00
2c72bd590f Add generic coverage task (#763)
**Todo:**
- [x] Finalize format for coverage file(s)
- [x] Add service support
- [x] Integration test
- [x] Merge #926 
- [x] Merge #929
2021-06-03 23:36:00 +00:00
c46abbcec3 stabilize onefuzz jobs containers download (#953) 2021-06-02 13:29:20 -04:00
1822acf943 add onefuzz jobs containers delete JOB_ID (#949)
Addresses #943
2021-06-02 17:04:51 +00:00
a92c84d42a work around issue with discriminated typed unions (#939)
We're experiencing a bug where Unions of sub-models are getting downcast, which causes a loss of information.  

As an example, EventScalesetCreated was getting downcast to EventScalesetDeleted.  I have not figured out why, nor can I replicate it locally to minimize the bug send upstream, but I was able to reliably replicate it on the service.

While working through this issue, I noticed that deserialization of SignalR events was frequently wrong, leaving things like tasks as "init" in `status top`.

Both of these issues are Unions of models with a type field, so it's likely these are related.
2021-06-02 16:40:58 +00:00
d2084f70b1 Rename helper method for setup-relative files (#926)
We are going to use this for more than just target exes, and in fact, it applies to any file that must occur in the setup directory/container.
2021-05-26 18:49:52 +00:00
2f1fc61f01 always show message details in status top (#933) 2021-05-26 14:22:24 -04:00
7952f16032 basic list proxy functionality (#905) 2021-05-24 13:24:22 -04:00
db4c03ea56 Catch argparse type errors (#911)
Catch argparse argument type errors and display them in a simplified fashion.

As an example, this is how the output is rendered.
```
❯ onefuzz template libfuzzer basic --target_env  ASAN_OPTIONS=allocator_may_return_null=1 a a a linux
unable to parse arguments: unable to parse value as a key=value pair: 'a'   
❯
```

Note, this builds upon #910.
2021-05-24 16:04:59 +00:00
e7197f1407 explain key=value argument parsing failures (#910) 2021-05-21 13:44:34 -04:00
776e8fa909 Add setup arguments to enable specifying alt tenants on the CLI (#900)
This enables specifying the endpoint configuration for alternate tenants purely on the command line.

Previously, on a single tenant you could use the following:
```
onefuzz --endpoint https://INSTANCE.azurewebsites.net info get
```

For multi-tenant installs, we need to expose more than just endpoint.

This enables:
```
onefuzz --endpoint https://INSTANCE.azurewebsites.net --client_id CLIENT_ID  --authority https://login.microsoftonline.com/common --tenant_domain TENANT_DOMAIN info get
```
2021-05-19 07:38:34 +00:00
f7e5bc7293 use azcopy for upload_dir (#878)
fixes #869
2021-05-17 14:06:02 +00:00
358d85ef82 fix cli client registration (#825)
- The CLIClient approle was not assigned to the cli registration created byt he registration.py script
2021-04-26 20:35:34 +00:00
cf3d904940 address formatting from black 21.4b0 (#831) 2021-04-26 12:35:16 -04:00
b5db1bd3fe update azure-cli to 2.21.1 (#821) 2021-04-26 08:45:26 -04:00
d9eda00f12 disable expect_crash_on_failure by default in more places (#807)
dotnet, as well as the _create_tasks helper still used the old default for requiring crash files on libFuzzer exiting non-zero
2021-04-15 20:19:10 +00:00
bf5f3d0ebb always use unix file endings when building libfuzzer-qemu wrappers (#782) 2021-04-09 09:56:56 -04:00
624a7f77e8 set ephemeral disks off by default (#761) 2021-04-01 16:26:17 -04:00
3096f99e86 enable using ephemeral disks by default (#461) 2021-03-30 18:48:44 -04:00
3eb7c8643b set expect_crash_on_failure default to False on libFuzzer tasks (#748) 2021-03-30 21:51:15 +00:00
c06d439c92 update azure-cli, azure-cli-core, and msal (#721) 2021-03-23 20:22:54 -04:00
516b1e000e expose minimized_stack_depth functionality in the CLI/API (#715) 2021-03-23 10:09:34 -04:00
71f487506f update pydantic to 1.8.1 (#698) 2021-03-19 18:02:03 -04:00
14b295f337 add privacy statement to CLI (#695) 2021-03-19 12:52:37 -04:00
6e60a8cf10 add regression testing tasks (#664) 2021-03-18 15:37:19 -04:00
44c830f600 save login details on successful login (#665)
If we login successfully, save the login data immediately.  That way if users run a second command  before the first one finishes, they only have to login once.
2021-03-12 00:36:12 +00:00
54e2cb2bf1 update signalrcore (#640) 2021-03-08 12:22:55 -05:00
fdac6b02a8 re-pin pyjwt version due to conflicts with azure-cli-core (#630) 2021-03-03 22:18:45 -05:00
7fc725d012 add non-x86_64 architecture libfuzzer target support using qemu-user (#600) 2021-03-03 19:06:50 -05:00
4ccc84a7de remove pyjwt from dependency list (#627) 2021-03-03 12:47:04 -05:00
2bec9db828 update azure-cli and azure-cli-core (#626) 2021-03-03 10:50:30 +00:00
78d6adf555 upgrade azure-storage-blob to 12.8.0 (#625)
Note, this makes use of the new feature, `ContainerClient.exists()` which models our existing mechanism for doing container existence checking.
2021-03-03 10:33:23 +00:00
bc6c8408c4 add onefuzz containers files download_dir (#598)
fixes #571
2021-02-26 15:27:51 +00:00
657af9722c coverage containers should be unique to the project/name/build/platform (#572) 2021-02-18 17:07:44 -05:00
ce47e4924a add status job commands (#550) 2021-02-16 13:47:57 -05:00
933fe6850c libfuzzer-dotnet integration (#535) 2021-02-11 17:30:24 -05:00
18d9daf909 Enable waiting for a job to start for managed templates (#532)
Provide `--wait_for_running` for managed templates.
2021-02-11 08:34:28 +00:00