Commit Graph

2050 Commits

Author SHA1 Message Date
9000a234e9 Add crate to debug missing dynamic libraries on Windows (#1713)
Add a CLI tool and library code to debug missing dynamic library errors on Windows.

The implementation manually edits the registry global flags for an image file to temporarily enable loader snaps, runs the target under our custom debugger to collect the debug output strings, then parses them for informative loading errors. It does not depend on the presence of `gflags.exe`.

This detects both dynamic linking (and thus process startup) errors, as well as dynamic loading (`LoadLibrary`) errors. It can report multiple missing dynamically-linked libraries.
2022-03-16 16:11:26 -07:00
194e7d0e9e Move the event grid topic creation and subscription to the deployment template (#1591)
* move the event grid subscription to the template

* change the name of the new subscription to prevent deleting the wrong subscription

* refactoring

* mypy fix

* format

* format

* remove old event grid before arm deployment

* fix deply

* attempt to fix check-pr issue

* fix interactive login in check-pr

* move the event grid subscription to the tempalte

* change the name of the new subscription to prevent deleting the wrong subscription

* refactoring

* mypy fix

* format

* format

* remove old event grid before arm deployment

* using resource Id

* fix type

* fix location

* revert changes in registration.py

* build fix attempt

* build fix

* revert ci changes

* remove file

* address comment

* address PR comments

* naming

* fix deplyment
2022-03-15 10:48:42 -07:00
ce36de72e7 Generate debuginfo for windows-libfuzzer-load-library test target (#1684) 2022-03-11 09:40:51 -08:00
c1ffec32f4 Reduce log level in ASAN parsing (#1705) 2022-03-10 14:03:58 -08:00
99091e7035 Release 5.2.0 (#1701)
* 5.2.0

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: Joe Ranweiler <joe@lemma.co>
5.2.0
2022-03-10 11:30:20 -08:00
d4c92d497f Add type definition (#1703) 2022-03-09 23:52:06 +00:00
40b0e6685a Give function app resource group scoped contributor role (#1698)
* Give function app resource group scoped contributor role

* Reenable autoscaling

* We don't know what the minimum capacity for a sku is yet

* Lint
2022-03-09 13:07:22 -05:00
fa8589a3d6 Bump backoff from 0.3.0 to 0.4.0 in /src/agent, resolve dependabot block (#1589)
* Inital changes needed for backoff 0.4 to work

* Update backoff versions, fix BackoffError:Transient fields, other uses

* Format

* Removed redundant field name

* Improved backoff update changes

* Update backoff update

* Revert

* Changed to using Error::transient function
2022-03-08 16:07:07 -08:00
b22955de9b Upgrade regex crate (#1699)
* Bump to non vulnerable regex version

* Removed unnecessary cargo.toml change
2022-03-08 19:17:29 +00:00
4d1c1f5713 Abstract out node disposal (#1686)
* Abstract node disposal strategy

* Cleanup + lint

* Handle possibile scalesets being in resize state

* Setting the size is still exposed via CLI, we don't want to break that functionality

* PR comments
2022-03-08 13:30:34 -05:00
7c507ab7c7 Remove dependency on onefuzz deployment role to unblock (#1693) 5.1.0 2022-03-04 18:51:57 +00:00
e2bd878f59 Release 5.1.0 (#1681)
* Release 5.1.0

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Prevent deletion of the repro VM on failure for debugging.

Co-authored-by: Joe Ranweiler <joe@lemma.co>
2022-03-03 15:55:36 -08:00
c9f7dd51f7 Clippy fix fell through the cracks (#1690)
* Clippy and fmt

* Compare versions

* Move the version block up

* and_then is the function we're looking for here
2022-03-03 16:37:04 -05:00
cde7602553 Fixes clippy lint (#1687)
* Fix lint

* Cargo fmt

* more lints
2022-03-03 11:50:39 -05:00
50bac07d19 increase the polling period in functional tests (#1682)
* increase the polling period

* update timeout to 30s

* format
2022-03-01 18:10:02 -08:00
d260689233 Remove use of deprecated warn() method on logger object (#1641)
Remove use of deprecated `warn()` method on logger object.

Co-authored-by: Marc Greisen <marc@greisen.org>
2022-03-01 09:19:11 -08:00
a63ffc46f5 Update terminology.md (#1656)
replaced the typo: scalsets to scalesets

Co-authored-by: Marc Greisen <marc@greisen.org>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>
2022-03-01 06:42:54 -08:00
0a6b5898bc Add timeout to setup scripts (#1659)
Add a setup script-specific timeout of 59 minutes. This is just shorter than the service-side `NODE_EXPIRATION_TIME` which otherwise garbage collects nodes whose setup scripts are stuck or taking too long.

With this change, the high-level cause of the timeout is clear, instead of the closest error being something indirect, like "node reimaged during task execution".
2022-02-28 23:00:39 -08:00
1b019818b5 Fail fast if managed task workers are near-OOM (#1657)
- Add `onefuzz::memory::available_bytes()` to enable checking system-wide memory usage
- In managed task worker runs, heuristically check for imminent OOM conditions and try to exit early
2022-02-28 21:36:52 -08:00
7f932167fe Correct extraction path in README.md (#1643)
Co-authored-by: Marc Greisen <marc@greisen.org>
2022-02-28 13:35:54 -08:00
f918299df0 Force -runs=1 when invoking in repro mode (#1651)
Co-authored-by: Marc Greisen <marc@greisen.org>
2022-02-28 12:23:18 -08:00
16166e1c14 Create autoscale resources for scaleset (#1661)
* Initial progress to adding a auto scale resource

* auto scale API is ready

* When creating a scaleset, add an autoscale resource to it as well

* Auto scale is correctly linked with scaleset

* 🧹

* Lint

* Cleaned up
2022-02-28 17:28:31 +00:00
c6f65c0f0e Used fixed Windows image for integration test build (#1675) 2022-02-24 08:57:03 -08:00
40efa9ae1b Unescape blob name in BlobUrl (#1673)
* Unescape blob name in BlobUrl
2022-02-23 14:31:27 -08:00
674444b7d7 Split integration tests into different steps (#1650)
Refactoring check-pr.py to extract the logic of downloading the binaries
refactoring integration-tets.py to split the logic of setup, launch, check_result and cleanup
2022-02-22 22:33:00 +00:00
5d8516bd70 Enable scale in protection on VMSS instances (#1647)
* draft attempt at adding scaling protection

* Service can now control scaling protection policy on VM instances

* Improve logging a bit

* draft attempt at adding scaling protection

* Service can now control scaling protection policy on VM instances

* Improve logging a bit

* Error message was missing info

* Linter

* Don't schedule work if we can't protect the node

* Last of the linter changes
2022-02-14 14:56:55 +00:00
77dcd57b46 Add EventGrid compatible webhook format (#1640) 2022-02-11 16:39:19 -08:00
65fd48b31b Fix typo (#1625) 2022-02-10 13:15:48 -05:00
b7fc35ffea Update yanked block-buffer from 0.10.0 to 0.10.1 (#1648)
* Update yanked block-buffer from 0.10.0 to 0.10.1

* update yanked crossbeam-utils from 0.8.5 to 0.8.7

* update yanked crossbeam-utils from 0.8.5 to 0.8.7

Co-authored-by: stas <statis@microsoft.com>
2022-02-08 08:02:39 -08:00
e0f2e9ed5a fix cli build (#1638)
* fix cli build

* build fix
2022-02-03 23:54:30 +00:00
62731f3836 bump azure-cli-core and azure-cli to 2.32.0 (#1634) 2022-02-02 17:38:46 -08:00
ee4dfe922f Allow authority to be specified in check-pr (#1473)
* fix interactive login in check-pr

* specify tenant domain in check-pr

* bug fix

* rename tenant to authority
2022-02-02 16:06:42 -08:00
809db31186 fix node serialization (#1627)
* fix node serialization

* remove intermediate variable

* mypy fix
2022-01-31 18:05:28 +00:00
dab0dba34f Fix lint warning (#1631) 2022-01-31 15:59:20 +00:00
4e6f496e06 bump azure-mgmt-subscription to 3.0.0 (#1606) 2022-01-28 18:57:12 -08:00
a600769d69 Add test-input example (#1624) 2022-01-27 12:48:52 -08:00
d358938895 Working in retry to test endpoint during integration test. (#1565) 2022-01-27 11:00:22 -08:00
af684f1335 Release version 5.0.0 (#1620)
* Release version 5.0.0

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: Joe Ranweiler <joe@lemma.co>
5.0.0
2022-01-27 09:25:44 -08:00
b49d140ff0 Add fuzzer_target_options to libfuzzer basic template (#1610) 2022-01-26 20:31:13 -08:00
6100191aaf Fixing VMSS Re-Image 7-Day Timer (#1616)
* Fixing VMSS Re-Image 7-Day Timer

* Updating use of TimeStamp to created_at

* Renaming.

* Updating field name.

* Removing test chagne.

* Updating query to work if init_at entry does not exist yet.

* Changing timer for testing.

* Adding field comment.

* Formatting models.py

* Fixing where save is called.

* Adidng logging.

* Removing logging. Ready for merge.

* Update src/pytypes/onefuzztypes/models.py

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Formatting.

* Updating datetime.

* Testing after datetime change.

* Removing test.

Co-authored-by: nharper285 <nharper285@gmail.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>
2022-01-26 17:40:27 -08:00
f374801d35 Re-Image Functionality Now Includes 'Upgrade' Call (#1612)
* Re-Image Functionality Now Includes 'Upgrade' Call

* Fixing call.

* Fixing testing change.

* Shrinking timedelta even more.

* Update nodes.py

* Update scalesets.py

* Update nodes.py

* Update nodes.py

* Fixing merge.

* Revert "Fixing merge."

This reverts commit ab4d2a54c3.

* Adding comment and logging to new upgrade call.

* Removing old logging statement.

Co-authored-by: nharper285 <nharper285@gmail.com>
2022-01-26 10:42:03 -08:00
2ad37bdfb0 Remove in-agent tracking of child process perf stats (#1608) 2022-01-25 15:06:10 -08:00
464940d716 bumpt azure-mgmt-compute to 24.0.1 (#1599) 2022-01-22 08:25:06 -08:00
348398eb1e fix build break (#1594)
Co-authored-by: stas <statis@microsoft.com>
2022-01-21 02:35:09 +00:00
13e52308f2 Bump azure-identity to 1.7.1 (#1586) 2022-01-19 16:09:37 -08:00
6901cd15ff bump azure-functions to 1.8.0 (#1582) 2022-01-14 22:34:01 -08:00
f02f3f0ae2 Updating AAD Docs and UserAssignment Roel Description. (#1581)
* Updating doc and role description.

* Update src/deployment/deploy.py

Co-authored-by: Cheick Keita <kcheick@gmail.com>

* Update docs/AADEntitites.md

Co-authored-by: Cheick Keita <kcheick@gmail.com>

Co-authored-by: nharper285 <nharper285@gmail.com>
Co-authored-by: Cheick Keita <kcheick@gmail.com>
2022-01-14 10:02:14 -08:00
7c3beb4d6f Bump tokio from 1.14.0 to 1.15.0 in /src/proxy-manager (#1530)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.14.0...tokio-1.15.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>
2022-01-13 14:15:49 -08:00
5083dc6e03 bump azure-core to 1.21.1 (#1579) 2022-01-13 09:34:44 -08:00
9092276d96 bump jinja2 to 3.0.3 (#1577) 2022-01-12 18:30:25 -08:00