This makes sure debug_keep_node is reset and the rest of the reimage processing occurs regardless of reimage_requested and delete_requested being set.
Without this, nodes that are marked `debug_keep_node` do not get reimaged/deleted.
Currently, if a pool or scaleset is set to `shutdown`, it cannot be set to `halt`.
While moving from `halt` to `shutdown` would cause issues, moving from `shutdown` to `halt` is fine.
The flag `Node.reimage_queued` is intended to stop nodes from reimaging repeatedly.
In #970, in order to work around Azure API failures, this flag was cycled if the node was already set to cleanup. Unfortunately, reimaging can sometimes take a significant amount of time, causing this change to get nodes multiple times.
Instead of using `reimage_queued` as a flag, this PR deletes the node from the storage table upon reimage. When the node registers OR the next time through `Scaleset.cleanup_nodes`, the Node will be recreated automatically, whichever comes first.
In a previous commit, reimage_queued was added to prevent reimaging a node while it is reimaging. However, this means reimaging failures due to Azure issues don't finish reimaging.
This will reset the this flag allowing the node to reimage in the following cleanup cycle.
If a user manually deletes a scaleset managed by OneFuzz, then `get_vmss_size` returns None.
When this happens, `Scaleset.shutdown` generates an exception from the `logging.info` call on line 573.
This PR handles this edge condition.
As is, when multiple tasks are running on a single node, if any one of them stops, the node gets reimaged.
This changes the behavior such that when a node with multiple tasks has one task stop, the other tasks will continue.