Commit Graph

53 Commits

Author SHA1 Message Date
aa54a15427 Add extra_output container, rename extra container (#3064)
## Summary of the Pull Request

- **Breaking** (but as far as I know this feature is not yet in use): rename the `extra_container` to `extra_setup_container`.
- **Add**: the `extra_output_container`, which pushes its outputs continually.
  - We may also want a type of container which both pushes & pulls? See discussion below.
- **Improved**: if `onefuzz-task` fails upon launch, we will log its output for diagnosis (might close #3113)

---

Some thoughts for the future:

We might want to redesign the containers so that we have something like the following which is passed to the agent, and the agent doesn't need to know the specifics of the containers supplied:

```jsonc
{
    // ...
    "containers": {
        "extra_setup_dir": {
            "mode": "pull",
            "container_name": "yyy",
        },
        "extra_output_dir": {
            "mode": "push",
            "continuous": true, // keep pushing while job is running
            "container_name": "xxx"
        }
    }
}
```

At the moment the agent needs to know what each container is for, for each task type. A more generic and flexible method might be simpler overall.
2023-06-15 02:48:27 +00:00
f4ab22d570 Checkout compiler-rt sparsely (#3101) 2023-05-15 10:52:01 -07:00
256f261f2c Parse .NET exception stacks in crash output (#2988) 2023-04-18 10:50:02 +12:00
3adb2fee36 Update integration test pool size (#2935)
* Upping VMSS count from 10 -> 20 on default tests
2023-03-23 16:26:46 -07:00
3d299ce51e fix extra container intilization (#2887)
* fix extra container intilization

* fix extra url download

* fix extra dir parameter to the agent

* rename extra to extra_dir
2023-02-27 13:57:32 -08:00
b84896802c Adding extra container to tasks (#2847)
* adding extra container to tasks

* setup expand

* build fix

* generate docs

* build fix

* build fix

* build fix

* format

* format

* build fix

* fix extra container references

* format

* Update "Needs Triage" label to the one we use. (#2845)

* Report extension errors (#2846)

Old failure message:
```
failed to launch extension
```

New failure message:

```
failed to launch extension(s): Errors for extension 'CustomScriptExtension':
:Error: ProvisioningState/failed/3 (Provisioning failed) - Failed to download all specified files. Exiting. Error Message: The remote server returned an error: (400) Bad Request.
```

* Sematically validate notification configs (#2850)

* Add new command

* Update remaining jinja templates and references to use scriban

* Add ado template validation

* Validate ado and github templates

* Remove unnecessary function

* Update src/ApiService/ApiService/OneFuzzTypes/Model.cs

Co-authored-by: Cheick Keita <kcheick@gmail.com>

---------

Co-authored-by: Cheick Keita <kcheick@gmail.com>

* adding extra container to integration tests

* adding doc

* update tests

* format

* build and clippy fix

* Update src/agent/onefuzz-task/src/tasks/report/generic.rs

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>

---------

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>
2023-02-23 19:08:01 +00:00
6cd53b1db6 Fix .NET integration tests (#2879) 2023-02-22 12:05:45 +13:00
4caaf8fc32 Remove old libfuzzer dotnet template (#2875)
### Context
The original `libfuzzer dotnet` job template was a proof of concept that demonstrated how the `libfuzzer_fuzz` task could be used to express fuzzing via the (pre SharpFuzz 2.0) `libfuzzer-dotnet` tool. It (and its associated integration test) used a harness that linked an older version of SharpFuzz, and which is incompatible with LibFuzzerDotnetLoader (which requires SharpFuzz 2.0 or greater).

### Changes
- Rename `libfuzzer dotnet_dll` job template to `libfuzzer dotnet`, making it the _only_ `libfuzzer-dotnet` template
- Remove integration tests and docs for the old proof-of-concept job type

### Notice

This is a breaking change.

Closes #2874.
2023-02-21 10:54:36 +13:00
d732028201 Add unmanaged nodes integration tests (#2780)
* Add docker file to the runtime tools

* fixes

* bug fixes

* more bug fixes and added doc

* don;t overwrite the RUST_LOG env var

* integration test for unmanaged nodes

* add unamanged parameters to launch()

* add ing object_id

* more bug fixes

* bug fixes

* chmod on the linux files in docker

* format

* cleanup merge

* added test_unmanaged command

* cleanup

* use a single image for the docker compose
remove images after the test

* docs and formatting

* format

* format

* format and bug fixes

* using windows server

* fix linux container
make the base image a paramter on windows
use the windows server base image on  windows server

* format

* bug fix

* more fixes

* allow reboot

* more fixes

* added more logging around the service principal creation

* format

* more logging

* change restart policy

* fix multi tenant domain

* more fixes

* exit instead of reboot when running inside docker

* remove comment

* build fix

* try_exist instead of exist

* save the docker logs

* bug_fix

* adding timeout

* fix timeout logic

* adding a build profile

* make all agent depend on the first one

* remove profile

* another fix

* restart agent 1

* Update docs/unmnaged-nodes.md

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>

---------

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>
2023-02-08 11:07:19 -08:00
7fc6fc3668 Reverting Enable/Disable App Function Changes - Force -net App Function To Only Run Agent Functions (#2660)
* Testing app function.

* Addding functions back.

* Fixing name.

* updating to string.

* Fixing bicep.

* Fixing function names.

* Update src/ApiService/ApiService/Functions/ReproVmss.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Adding route specifier.

* Update src/ApiService/ApiService/Functions/NodeAddSshKey.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Update src/ApiService/ApiService/Functions/WebhookLogs.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Update src/ApiService/ApiService/Functions/WebhookPing.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Fixing function names.

* Increasing pool_size

* Adding logging statements.

* Adding check for transient failure.

* Adding check to else.

* Fix var

* Adding retry logic to dbg_lnx

* FIxing retry.

* Maybe fixed integration test.

* Formatting.

* Adding try and fail logic.

* Changing error.

* Refactoring logic.

* Fixing logic.

* Logic.

* Removing bad comparison.

* Fixing.

* All ready.

* Switching back to call.

* Add.

* Fixing

* Adding quit back.

* Reversing cli changes.

* Removing bad file.

* Formatting.

Co-authored-by: George Pollard <gpollard@microsoft.com>
2022-12-02 11:14:13 -08:00
e0634a3365 Deploy update (#2650)
* Remove Python App Function Deployment Code and  Code.

* Updating yml and zip names.

* Fixing ci.yml.

* Typo.

* Format

* Trying to remove python service.

* Updating directories.

* Removing flag.

* Format.

* Fixng api-service-net ref.

* Re-add requirement.

* Fixing refs in bicep.

* Specifying version.

* Removing dotnet refs in integration tests.

* Updating role assignment naming convention.

* Adding ignore.

* Update src/deny.toml

Co-authored-by: George Pollard <porges@porg.es>

* Update version.

* Removing onefuzztypes dependency.

* Switch app service plan to windows.

* Update test version.

* Changing version.

* Returning version.

* Trying to add onefuzz types back.

* Force pipenv version.

* Fix.

* Syntax.

* Renaming.

* Trying different version.

* Removing build step.

* Fixing bicep parameter.

* Retrying run with older version.

* Trying pipenv with another version.

* Forcing pipenv version in tox.

* Adding pipenv fix and updating version.

Co-authored-by: George Pollard <porges@porg.es>
2022-11-23 17:48:30 -08:00
894dcc62be Update to .NET 7 (#2615)
Update service code to .NET 7.0.

Also update the version installed onto agent nodes via setup scripts to 7.0.100.
2022-11-18 10:39:45 +13:00
e50bc0b389 Update the libfuzzer-with-options test to use fuzzing_target_options (#2565)
This currently causes the coverage task to time out, ensure that the `runs` parameter is only passed to the fuzzing task and not the coverage task.
2022-10-26 00:03:28 +00:00
ee0cbd70a0 Enable dotnet fuzzing (#2273)
Add a new CLI job template, `libfuzzer dotnet_dll`, and supporting server-side definitions.
2022-10-19 14:17:27 -07:00
02869eba9e Fix #1285 (#2353) 2022-09-08 08:36:38 +12:00
bd4dfdc592 Update chrono, remove time to fix audit failures (#2285)
Closes #1366 and incidentally closes #1266.

Requires using a not-yet-merged modification to the latest version of `appinsights-rs` to remove the `time` feature from the `chrono` dependency (https://github.com/dmolokanov/appinsights-rs/pull/280). 

There are changes to use the new (tokio-based) version of `appinsights-rs`, e.g., made `set_appinsights_clients` async to ensure it is always called from an async context, since the constructor for appinsights now invokes `Tokio::spawn`.
2022-09-07 00:34:31 +00:00
163e324227 Specifying initial size for scaleset create in integration-tests. (#2307)
* Specifying initial size for scaleset create in integration-tests.

* Spell check.
2022-08-26 16:09:36 -07:00
1a63f195f3 porting the proxy state machine (#2286)
* porting the proxy state machine

* use async version

* is used

* rename base state to state

* fix auth
fix extension path

* ignore log info in check logs
2022-08-25 19:42:01 +00:00
a3f1d59f70 Set autocrlf by default, update affected files (#2261) 2022-08-17 13:07:20 +12:00
c653f2e8ef Add support for dotnet_endpoint/dotnet_functions to integration-test.py (#2192)
This will allow testing client-facing .NET Functions during integration tests (`check-pr`).
2022-07-26 14:44:59 +12:00
69b11f60c2 Add GoodBad C# example project, integration test (#2148) 2022-07-12 12:07:30 -07:00
9cbbf86e36 Add the setup state to the unavailable list (#1731)
* Add the setup state to the unavailable list

* Add the init state to the unavailable list

* Fix integration test

* Wrong import
2022-04-01 14:00:25 -04:00
ce36de72e7 Generate debuginfo for windows-libfuzzer-load-library test target (#1684) 2022-03-11 09:40:51 -08:00
e2bd878f59 Release 5.1.0 (#1681)
* Release 5.1.0

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Prevent deletion of the repro VM on failure for debugging.

Co-authored-by: Joe Ranweiler <joe@lemma.co>
2022-03-03 15:55:36 -08:00
50bac07d19 increase the polling period in functional tests (#1682)
* increase the polling period

* update timeout to 30s

* format
2022-03-01 18:10:02 -08:00
674444b7d7 Split integration tests into different steps (#1650)
Refactoring check-pr.py to extract the logic of downloading the binaries
refactoring integration-tets.py to split the logic of setup, launch, check_result and cleanup
2022-02-22 22:33:00 +00:00
ee4dfe922f Allow authority to be specified in check-pr (#1473)
* fix interactive login in check-pr

* specify tenant domain in check-pr

* bug fix

* rename tenant to authority
2022-02-02 16:06:42 -08:00
d358938895 Working in retry to test endpoint during integration test. (#1565) 2022-01-27 11:00:22 -08:00
1de2cc841d Check-pr creates and uses SP. (#1504)
* Check-pr creates and uses SP.

* flake8.

* flake8.

* Fixing var name.

* Fixing deploy.py

* Looking for client_secret

* Change to check_output.

* mypy fix.

* Fixing check-pr

* working version.

* lint

* Updating arg text.

* Removing redundant functionality.

* Changing register codepath and adding flag.

* Removing pycache file.

* Fixing unattended flag.

* Adding space.

* Fixing a few calls.

* Removing file.

* Removing python3.

* Removing old file.

* Adding wait into registration.py

* Formatting registration.py.

* Removing space.

* Adding retry logic to check-pr.

* Formatting.

* Retriggering.

* Retriggering.

* Calling sp_create and adding retry to authorize.

* Fixing syntax.

* Removing comments.

* Adding another retry.

* Retriggering.

* Retriggering.

* Retriggering.

* Trying to find error.

* Adding retry logic.

* Increasing sleep.

* Fixing formatting.

* Retriggering.

* Removing bad file.

* Trying out retry for logger.

* typevar issue?

* Re-adding.

* Retriggering.

* retriggering.

* Retriggering.

Co-authored-by: nharper285 <nharper285@gmail.com>
Co-authored-by: Cheick Keita <kcheick@gmail.com>
2021-12-17 09:44:04 -08:00
09bdad4979 work around issues between released version of cargo-fuzz and nightly (#1284)
See rust-fuzz/cargo-fuzz#276

Co-authored-by: Brian Caswell <bmc@shmoo.com>
2021-09-27 11:29:31 -04:00
9413377c90 handle fake crash reports generated by debugging tools in regression tasks (#1233) 2021-09-22 17:16:35 +00:00
041d3cc204 ignore table exists errors from integration testing at the azure functions layer (#1096) 2021-07-21 13:29:13 -04:00
e065b571f7 add format string bug to example integration test (#1087) 2021-07-20 05:00:45 -04:00
3305732160 increase libfuzzer integration test -runs (#1081) 2021-07-16 17:29:37 -04:00
1121ebc6e8 fix issue specifying target_options for libfuzzer targets (#1066) 2021-07-14 14:38:35 -04:00
b9950c5526 update log messages to ease debugging (#988) 2021-06-14 15:18:03 -04:00
cf3d904940 address formatting from black 21.4b0 (#831) 2021-04-26 12:35:16 -04:00
794400adf1 ignore benign error messages when checking integration test logs (#775) 2021-04-06 15:35:54 -04:00
ca12904684 add log checking to refactored integration check (#700)
In practice, Application Insights can take up to 3 minutes before something sent to it is available via KQL.

This PR logs a start and stop marker such that the integration tests only search for logs during the integration tests. This reduces the complexity when using the integration tests during the development process.

Note: this migrated the new functionality from #356 into the latest integration test tools.
2021-04-02 21:49:19 +00:00
2417c37a60 add LoadLibrary integration test (#689) 2021-03-19 09:46:26 -04:00
6e60a8cf10 add regression testing tasks (#664) 2021-03-18 15:37:19 -04:00
f41f110af8 libfuzzer library integration tests (#681) 2021-03-17 16:18:10 -04:00
f6a426cc07 enable long-running integration tests (#654) 2021-03-10 17:03:15 -05:00
7fc725d012 add non-x86_64 architecture libfuzzer target support using qemu-user (#600) 2021-03-03 19:06:50 -05:00
cc5965ebbf add .gitignore to ignore libfuzzer-dotnet build artifacts (#564) 2021-02-19 09:32:26 +00:00
f64a0dcc05 lint integration-test.py (#549) 2021-02-16 12:22:45 -05:00
933fe6850c libfuzzer-dotnet integration (#535) 2021-02-11 17:30:24 -05:00
4f941d5ff2 add the task_id to the integration failure logs (#446) 2021-01-21 13:35:54 -05:00
af2ef9f4fa verify local debugger installs before launching repro (#436)
This only launches the repro test cases if you have the os-specific prerequisite debugger installed.
2021-01-18 23:30:53 +00:00
75d2ffd7f4 lint test utils (#395) 2021-01-05 08:50:52 -05:00