Commit Graph

187 Commits

Author SHA1 Message Date
ac9d072e1d fix linux proxy extensions provisioning failures (#2401)
* fix linux proxy extensions provisioning failures

* format

Co-authored-by: stas <statis@microsoft.com>
2022-09-14 23:52:35 +00:00
2fe73ab79c bug fix (#2392)
* bug fix

* rename anyNotStoppedJobs  to anyNotStoppedTasks

Co-authored-by: stas <statis@microsoft.com>
2022-09-14 10:07:15 -07:00
ca7b6be43b Refactor notification support (#2363)
* Add teams notifications

* .

* Fix compilation isues

* Checkpoint

* Added Ado

* Fix some TODOs

* Teams messages work! 🎉

* fmt

* Bug fix container url generator

* Some small ado changes

* 🧹

* PR comments

* Fix packages

* Get more detailed restore information to debug errors

* Maybe fixes this issue?

* Undo CI change
2022-09-14 15:07:52 +00:00
f375ee719e Two fixes to C# scheduling (#2390)
Two fixes to scheduling code:

- `GetPool` was not correct for the VM case (this code is possibly legacy and not used any more)
- `BuildWorkUnit` could fetch the same pool multiple times and then fail due to `BucketConfig` mismatch (on `TimeStamp`)
  - add a cache to the loop so that we only fetch each pool once
2022-09-14 02:06:01 +00:00
bb81f2ec51 Fix MarkDependantsFailed (#2389) 2022-09-14 01:21:13 +00:00
f7f91df622 CSharp Refactor - Instance Config Endpoint (#2347)
* CSharp Refactor - Instance Config Endpoint

* Finshing config update.

* Formatting.

* Formatting.

* formatting.

* Fixing encoding.

* Fixing config references.

* Fixing refs.

* Trying location.

* Trying ref to location.

* Passing nsg.

* Passing nsg.

* Setting nsg to not null.

* Fixing ok reference.

* Adding Instance Config Response.

* Setting required attribute.

* Adding route specifier.

* Formatting.

* Fixing route.

* Fixing optionals.

* Trying to set default

* Trying again.

* Setting require admins

* Removing optioanl.

* Testing with instancename.

* Updating instanceconfig model.

* Updating instance config response.

* Formatting.

* Removing AllowPoolManagement.

* Readding.

* Removing arg.

* Replacing with RequireAdminPrivs.

* Fix orm test.

* Setting requireadminprivs to true.

* Requiring admin privs.

* Fix formatting.

* fix test.

* Fixing.

* Changing error message.

* Changing.

* Reordering test args.

* Flipping.

* Fixing args.

* Fixing again.

* Removing false.

* Removing from constructor.

* Setting.

* Setting string to optional.

* Formatting.

* Adding default value.

* PUshing changes to OrmModelsTest

* Updating test to not pass null.

* George's suggestions.

* Removing entityconverter changes.

* Fixing import.
2022-09-13 08:55:40 -07:00
67e55910ac Fix TaskOperations.SearchStates (#2383) 2022-09-13 00:31:35 +00:00
bc33ae1d7a Fix Scaleset response Auth inclusion (#2382)
The `Auth` property is not meant to be returned upon Create/`POST`. Fix this, and make it easier to specify when `Auth` should be included or not.
2022-09-12 17:19:10 -07:00
ce1fc773a9 implement not-implemented: GetInputContainerQueues (#2380)
* implement not-implemented: GetInputContainerQueues

* named tuple

Co-authored-by: stas <statis@microsoft.com>
2022-09-12 15:17:21 -07:00
44f74f622a DoNotRunExtensionsOnOverprovisionedVms must be false if Overprovision is false (#2375) 2022-09-12 22:00:47 +00:00
9eaf92ea35 tests and bug fixes (#2379)
Co-authored-by: stas <statis@microsoft.com>
2022-09-12 14:45:45 -07:00
2a2c07ed35 Extend use of validated string types (#2357)
In the Python code there were more validated string types that haven't been properly ported for use in the C# code (such as Region). Add that type back in and improve some others:

- Use `Region` type to represent regions (implicitly convertible to/from the `AzureLocation` SDK type)
- Improve validation of `Container` type to match Azure specs and use it in more places
- Restore/fix validation of `PoolName` type which was previously removed (#2080) due to being too strict: now allows 1-64 ASCII alphanumeric/hyphen/dash
  - We want to restrict pool names so that we can use them as disambiguating prefixes for scaleset names (see #2189). Note that underscore is not actually permitted in scaleset names so we will probably end up mapping it to hyphen.

Note that once C#7 lands we will be able to simplify the usage of `ValidatedString` a lot (using static abstract methods).

----

Open questions:

For deserializing from "known-good" places such as table storage or from Azure SDK APIs, should we have an `T.UnsafeAssumeValid(string input)` method which does no validation, to protect us from breakage?
2022-09-12 10:06:51 +12:00
6dfa46e62f remove required field to match python behavior (#2367)
Co-authored-by: stas <statis@microsoft.com>
2022-09-08 23:10:12 +00:00
c562bbb0ad VMSS creation fixes (#2358) 2022-09-09 07:53:02 +12:00
d61f66a667 Fixing memory leaks (#2356)
* fix a memory leak

* fix another memory leak

* remove extra queries

* formatting

* do not throw when scale set cannot be updated

Co-authored-by: stas <statis@microsoft.com>
2022-09-07 17:03:38 -07:00
fe51e011d1 Use Graph SDK instead of manually constructing queries (#2324) 2022-09-07 01:12:36 +00:00
0a84b1466d Implement not implemented bits (#2341)
`ScalesetOperations` was missing:

* `ReimageNodes`
* `DeleteNodes`
2022-09-06 23:43:14 +00:00
b28008b519 Check if state transitions exist at test time (#2241)
Don't ignore missing state transitions in `StatefulOrm` static constructor test, instead error out. 

`Scaleset` was missing the `Resize` transition.
2022-09-06 22:52:31 +00:00
84b2cc9992 Rework use of storage clients (#2302)
- Reuse Storage clients by caching them against account name.
  - This follows recommendations of Storage documentation: https://devblogs.microsoft.com/azure-sdk/lifetime-management-and-thread-safety-guarantees-of-azure-sdk-net-clients/
- Use strongly-typed `ResourceIdentifier` when passing around Account IDs, to prevent mixing them up with Account Names. 
- Simplify & centralize SAS generation in one place.
2022-09-06 22:11:10 +00:00
dd3aa1ce0a Bug fixes to enable timer_workers running properly (#2343)
Co-authored-by: stas <statis@microsoft.com>
2022-09-06 09:13:33 -07:00
ee1ad0abd8 Functional tests (#2333)
* more tests

* revert compute, since "released" version has bug when creating scaleset extensions

* format

* found bug, i think...

* now should be fixed

Co-authored-by: stas <statis@microsoft.com>
2022-09-02 08:16:48 -07:00
c54db04083 Enforce that there no extra properties in request JSON, and that non-null properties are [Required] (#2328)
Closes #2314 via two fixes, one for additional properties and one for missing properties:

- Make all request types inherit from `BaseRequest` which has an `ExtensionData` property, and ensure that it is empty in `ParseRequest`.
- Add `[Required]` attribute to non-nullable properties that do not have defaults, and add a test that ensures we have this attribute where necessary.
2022-09-02 01:59:24 +00:00
52ba57bf0d Remove some response-only properties from the Task model (#2335) 2022-09-01 23:33:04 +00:00
f7e37355d0 Remove IDisposable from Creds (#2327) 2022-08-31 22:51:06 +00:00
366b671942 Cache InstanceConfig (#2329) 2022-08-31 22:30:38 +00:00
d711172ed7 Add scriban dependency and some template tests (#2330)
* Add scriban dependency and some template tests

* Resolve

* Missed force-evaluate

* fmt
2022-08-31 13:21:22 -04:00
8d23df328d bump nuget packages (#2321)
Co-authored-by: stas <statis@microsoft.com>
2022-08-30 13:58:07 -07:00
ea4d589abf Functional tests (#2317)
* more functional tests

* more tests

* bug fixes

* OneFuzz results do not map one to one with datastructures. So add support for multiple datastructures to be returned per API

* more tests

* format fixes

Co-authored-by: stas <statis@microsoft.com>
2022-08-30 12:26:06 -07:00
63c54193a9 Enable AppInsights dependency tracking (#2315) 2022-08-30 16:35:35 +12:00
ef434db201 Fix ServiceConfiguration.OneFuzzVersion (#2316) 2022-08-30 00:08:56 +00:00
5a8aa33af1 functional tests and bug fixes (#2301)
Co-authored-by: stas <statis@microsoft.com>
2022-08-29 09:33:15 -07:00
ae6df1e22f Create tables on startup (#2309) 2022-08-29 15:12:19 +12:00
d213eac44e Implement negotiate function (#2183) 2022-08-26 02:59:31 +00:00
c7e7fa1d27 Port autoscaling to C# (#2296)
Co-authored-by: stas <statis@microsoft.com>
2022-08-25 13:13:24 -07:00
1a63f195f3 porting the proxy state machine (#2286)
* porting the proxy state machine

* use async version

* is used

* rename base state to state

* fix auth
fix extension path

* ignore log info in check logs
2022-08-25 19:42:01 +00:00
c3c9e0d942 fix function name (#2282) 2022-08-25 03:27:36 +00:00
0133b3e42f Include serialization options when sending event message (#2290) 2022-08-24 19:42:34 +00:00
39e22b060a Only save the job once when creating it (#2289)
* Convert exception to warning

* don't save job before creating the log container

* same as the python code

* test fix

* format

* format

* don't schedule the task if the log container is missing
2022-08-24 10:19:18 -07:00
21a4231912 Fix NullReferenceException in CreateQueue (#2283) 2022-08-22 23:21:56 +00:00
dc3ab3d95a Clone the JsonSerilizerOption instead of modifying it (#2280) 2022-08-22 11:01:34 -07:00
6cb7e7e24a Make EntityConverter singleton (#2267)
* Make the EntityConverter a singleton

* Make the EntityConverter a singleton

* format

* fix tests
2022-08-19 18:57:19 +00:00
ac54c576ae Provide a default for settings (#2269) 2022-08-18 15:23:47 -04:00
e74671bcc4 fix pool comparison in the scheduler (#2260)
* fix pool comparison in the scheduler

* format
2022-08-18 01:49:09 +00:00
f134c0f122 Enabling csharp webhook and proxy heartbeat functions (#2263)
* Enabling csharp webhook and proxy heartbeat trigger

* Adding deploy exemption.

* Fixing queue table.

* Adding function to bicep.

* Renaming function.
2022-08-17 16:29:47 -07:00
24c4219949 Port auto scale scaleset shutdown bug fix to csharp (#2262)
* Checkpoint

* Merge main

* Implement the bug fix + some missing functionality in C#

* Test fix
2022-08-17 19:33:56 +00:00
d1bfaefd0d Implement scaleset function for C# (#2191)
Implements #2194
2022-08-17 15:00:57 +12:00
a3f1d59f70 Set autocrlf by default, update affected files (#2261) 2022-08-17 13:07:20 +12:00
2382ce5ca5 migrate webhooks (#2254)
* migrate webhooks

* webhook_logs and webhook_ping

* fix saving uri to db

* fix tests

* fix deploy.py
2022-08-16 12:20:16 -07:00
8738ef0044 timer workers fix up (#2257)
* timer workers fix up

* remove unused includes

* revert breaking changes

* update autoscale definition

Co-authored-by: stas <statis@microsoft.com>
2022-08-15 20:20:46 -07:00
90d3ec987b Port TimerWorkers to C# (#2235)
* Port TimerWorkers to C#

* test logs

* ..

* Some VMSS extensions have settings as null

* fix for

Cannot set DoNotRunExtensionsOnOverprovisionedVMs to true on Virtual Machine Scale Set which does not have overprovision enabled.

Co-authored-by: stas <statis@microsoft.com>
2022-08-15 16:54:48 -07:00