74 Commits

Author SHA1 Message Date
Joe Ranweiler
7f7d76fa7f
Use consistent case for project name in docs (#1307) 2021-09-29 17:01:58 -07:00
Joe Ranweiler
fb29d8946a
Remove references to playground instance (#1295) 2021-09-29 14:33:47 -07:00
bmc-msft
22b2d62e29
enable configurable virtual network ranges (#1268) 2021-09-27 18:01:32 +00:00
Noah McGregor Harper
599c400fa0
Custom Extension Instance Configuration (#1184) 2021-09-24 12:27:39 -04:00
bmc-msft
1b2962547c
fix container documentation to align with how coverage also maps to build & platform now (#1277) 2021-09-23 23:16:55 +00:00
bmc-msft
21e969c1eb
add support for expanding machine_id (#1216) 2021-08-31 18:11:30 +00:00
bmc-msft
2a2844ae7a
enable configuring proxy VM sku (#1128) 2021-08-23 16:04:59 +00:00
bmc-msft
2fcb499888
Merge pull request from GHSA-q5vh-6whw-x745
* verify aad tenants, primarily needed in multi-tenant deployments

* add logging and fix trailing slash for issuer

* handle call_if* not supporting additional argument callbacks

* add logging

* include new datatype in webhook docs

* fix pytypes unit tests

Co-authored-by: Brian Caswell <bmc@shmoo.com>
2021-08-13 14:50:54 -04:00
bmc-msft
338b541a94
expose coverage as an optional directory that gets synced to supervisor tasks (#1123)
Addresses #1122
2021-08-06 19:13:23 +00:00
bmc-msft
198d765cb4
Add generic_analysis example that performs coverage analysis (#1072)
This adds an example script and tool that enables LLVM source-based coverage using the `generic_analysis` task.

This provides:
1. sample python script that launches the template and then the analysis task
1. sample `analysis_exe` wrapper script that launches the LLVM coverage tools
1. sample libfuzzer target for the example
1. walk through submitting the jobs and inspecting the results
2021-07-21 17:12:24 +00:00
bmc-msft
5ba04d0cd2
use pydantic built-in size validators (#1048) 2021-07-07 17:07:05 -04:00
bmc-msft
52f83b5b26
add EventScalesetResizeScheduled (#1047) 2021-07-07 14:15:26 -04:00
bmc-msft
a43f693761
Provide minimized stack lines (#993)
Using the function name only method for minimization results in a
function with two different malloc related bugs to get bucketed
together.

This provides a minimized stack without function address of stack depth.

For entries without source information, such as:
`# 34 0xf19113f in ChromeMain+0x13f (C:\\clusterfuzz\\bot\\builds\\chrome-test-builds_media_win32-release\\revisions\\asan-win32-release-335593\\chrome_child.dll+0x113f)`
users will see this:
`ChromeMain+0x13f`

For entries with source information, such as:
`# 20 0x58bd3bb in v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) ../../v8/src/builtins/builtins-api.cc:137:5`
users will see this:
`v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) builtins-api.cc:137:5`
2021-07-01 21:14:31 +00:00
bmc-msft
29dda54b83
instance wide configuration (#1010)
TODO:
* [x] add setting initial set of admins during deployment
2021-06-30 21:13:58 +00:00
bmc-msft
0d709ed5f8
fix markdown link (#989) 2021-06-14 16:40:04 -04:00
Cheick Keita
93cfedbd2e
fix scaleset identity name in the documentation (#987) 2021-06-14 16:10:14 -04:00
Joe Ranweiler
2c72bd590f
Add generic coverage task (#763)
**Todo:**
- [x] Finalize format for coverage file(s)
- [x] Add service support
- [x] Integration test
- [x] Merge #926 
- [x] Merge #929
2021-06-03 23:36:00 +00:00
bmc-msft
a4bb670fb2
add proxy_state_updated events (#908) 2021-05-21 12:47:54 -04:00
nharper285
2f81c44f01
Refactoring proxy lifetime to only shutdown when proxy is out-of-date. (#839)
## Summary of the Pull Request

_What is this about?_
We'd like to refactor the proxy lifecycle to only delete when the proxy is out-of-date - i.e. when the proxy is older than 7 days or a mismatched version. I've changed two files, proxy.py and timer_daily\init.py to check for the version and timestamp before stopping a live proxy. 

## PR Checklist
* [ ] Applies to work item: #xxx
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/onefuzz) and sign the CLI.
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

## Info on Pull Request

_What does this include?_
Changes to two files: 
proxy.py: 
- get_or_create() edited to check if timestamp is >7 days.
- Created is_outdated() to check version and timestamp for out-of-date proxy. 
timer_daily/init.py
- Proxy check now includes is_outdated() before determining if a proxy should be shutdown. 

## Validation Steps Performed
Deploying test instance to determine if proxy lives past a single day.
2021-05-20 14:33:29 +00:00
bmc-msft
2b67c7b02f
add setup_dir to generator task (#901)
fixes #848
2021-05-19 16:10:13 +00:00
Cheick Keita
3c0f6f56a7
Azure AD entities description (#896)
Added a description of the configuration of azure AD entities

closes #875
2021-05-17 12:44:45 +00:00
bmc-msft
cb5e786bcd
add event for scaleset state updates (#882)
This moves all scaleset state updates through `Scaleset.set_state` and adds a new event EventScalesetStateUpdated.
2021-05-13 21:23:02 +00:00
bmc-msft
b599aea5cd
allow extra fields in Events (#805) 2021-04-14 13:09:21 -04:00
bmc-msft
46b8bdccbc
add TaskConfig to crash_reported and regression_reported events (#793)
resolves #757 and #758
2021-04-13 10:24:12 +00:00
bmc-msft
3096f99e86
enable using ephemeral disks by default (#461) 2021-03-30 18:48:44 -04:00
bmc-msft
e80b658a1a
expose new minimized stack fields in the crash report model (#719) 2021-03-23 12:29:08 -04:00
bmc-msft
516b1e000e
expose minimized_stack_depth functionality in the CLI/API (#715) 2021-03-23 10:09:34 -04:00
bmc-msft
e38281face
rename salvo telemetry to "upcoming feature" (#693)
Until salvo is released, this should be "upcoming feature" to avoid questions.
2021-03-19 14:59:08 +00:00
bmc-msft
769b614367
add git-bisect howto (#691) 2021-03-19 14:26:20 +00:00
bmc-msft
6e60a8cf10
add regression testing tasks (#664) 2021-03-18 15:37:19 -04:00
bmc-msft
ff3e80fca5
Add salvo telemetry (#619)
Adds telemetry fields for Salvo.

This PR should not be merged until approved by CELA.

Note: This PR includes #586
2021-03-12 23:39:42 +00:00
bmc-msft
19a3805130
include version and software role to Microsoft collected telemetry (#586)
NOTE: This should not be merged until the privacy review has completed.

Ref: reltrack-4150
2021-03-12 14:50:15 +00:00
bmc-msft
07f7f6fa48
Add input file sha256 placeholder (#641) 2021-03-11 20:27:58 -05:00
Joe Ranweiler
67435e7883
Simplify module/symbol filter definition format, impl (#628)
Update the filter rule format and implementation to be simpler and user-predictable. In particular, we remove an accidental dependence of rule application on hash map iteration order.
2021-03-09 12:45:01 -08:00
bmc-msft
0a3812d8bc
Add job stopped task info (#648) 2021-03-09 10:06:06 -05:00
bmc-msft
157a14d003
fix markdown link error (#637) 2021-03-05 19:20:00 -05:00
bmc-msft
1c09caedc5
add howto guide to understand the libfuzzer_coverage task (#631) 2021-03-05 19:21:11 +00:00
bmc-msft
7fc725d012
add non-x86_64 architecture libfuzzer target support using qemu-user (#600) 2021-03-03 19:06:50 -05:00
nharper285
ec33531870
Add Threat Model documentation (#482)
This PR includes a [Threat Model](https://aka.ms/tmt) for OneFuzz.
2021-03-03 19:30:56 +00:00
bmc-msft
4489036d9f
add node & task heartbeat events (#621)
This adds node & task heartbeats and makes the event data available as a structured data in the logs.
2021-03-02 22:04:39 +00:00
bmc-msft
e3c73d7a10
Update command variable expansion (#561)
* Documents `crashes_account` and `crashes_container`
* Adds `reports_dir` and support for `unique_reports`, `reports`, and `no_repro` containers to the generic analysis task
* Adds `microsoft_telemetry_key` and `instance_telemetry_key` to generic supervisor, generator, and analysis tasks
2021-02-26 20:58:09 +00:00
bmc-msft
6a049db3a3
Renames application insights keys to be more clear (#587)
* renames `telemetry_key` to `microsoft_telemetry_key`
* renames `instrumentation_key` to `instance_telemetry_key`
* renames `can_share` to `can_share_with_microsoft`
* renames the `applicationinsights-rs` instances to `internal` and `microsoft` respective of the keys used during construction.

This clarifies the underlying use of Application Insights keys and uses struct tuple to ensure the keys are used correctly via rust's type checker.
2021-02-26 17:04:49 +00:00
bmc-msft
305c23a4d9
add instance information to webhooks (#577)
Fixes #574
2021-02-19 21:00:51 +00:00
bmc-msft
4992b494f1
add task config to all task events (#580) 2021-02-19 14:10:48 -05:00
bmc-msft
933fe6850c
libfuzzer-dotnet integration (#535) 2021-02-11 17:30:24 -05:00
bmc-msft
bdcab6eb08
handle tokens from x-ms-token-aad-id-token (#531) 2021-02-10 12:41:15 -05:00
bmc-msft
cdfdc2be84
split telemetry into it's own crate (#501)
Splits out telemetry crate such that it can be reused by other components (specifically the proxy-manager) easily.
2021-02-04 14:46:35 +00:00
bmc-msft
5acb59e5b9
Add task_id & job_id to variable expansion (#481)
Fixes #479 

Note, this is built on top of #480
2021-01-29 15:35:59 +00:00
bmc-msft
a46f7b4193
expose supervisor tasks that are fully self-contained fuzzing tasks in the service (#474)
Exposes the functionality added in #454 to the service & CLI.

Fixes #439
2021-01-29 00:01:59 +00:00
bmc-msft
07760bf967
Update expansion documentation to describe nested options as implemented (#475)
This expands the documentation to demonstrate how nested lists are handled in practice, as well as to provide examples for how to deal with the nested list expansion.
2021-01-28 17:12:48 +00:00