mirror of
https://github.com/nasa/trick.git
synced 2025-03-27 22:28:35 +00:00
* Provide MonteCarloGenerate capability Intermediate commit, this squash represents all of Isaac Reaves' work during his Fall 2022 Pathways internship tour [skip ci] * TrickOps: Add phase, [min-max] range, and overhaul YAML verification * Add new "phase:" mechanism to TrickOps Runs and Builds to support project-specific constraints on build and run ordering - phase defaults to zero if not specified and must be between -1000 and 1000 if given. - jobs can now optionally be requested by their phase or phase range - See trickops/README.md for details * Add [min-max] notation capability to run: entries and compare: entries - [min-max] ranges provide definition of a set of runs using a common numbering scheme in the YAML file, greatly reducing YAML file size for monte-carlo and other zero-padded run numbering use cases - See trickops/README.md for details * YAML parsing changes - Overhaul the logic which verifies YAML files for the expected TrickOps format. This is now done in TrickWorkflowYamlVerifier and provides much more robust error checking than previous approach - .yaml_requirements.yml now provides the required types, ranges, and default values as applicable to expected entries in YAML files - valgrind: is now an sub-option to run: entries, not its own section Users should now list their runs normallly and define their flags in in that run's valgrind: subsection - parallel_safety is now a per-sim parameter and not global. Users should move their global config to the sim layer - self.config_errors is now a list of errors. Users should now check for empty list when using instead of True/False * Robustify the get_koviz_report_jobs unit test to work whether koviz exists on PATH or not * Adjust trickops.py to use the new phase and range features - Make it more configurable on the command-line via argparse - Move SIM_mc_generation tests into test_sims.yml [skip ci] * Code review and cleanup from PR #1389 Documentation: * Adjust documentation to fit suggested symlinked approach. Also cleaned up duplicate images and old documentation. * Moved the verification section out of markdown and into a PDF since it heavily leverages formatting not available in markdown. * Clarify a couple points on the Darwin Trick install guide * Update wiki to clarify that data recording strings is not supported MCG Code: * Replace MonteCarloVariableRandomNormal::is_near_equal with new Trick::dbl_is_near from trick team MCG Testing: * Reduce the set of SIM_mc_generation comparisons. After discussion the trick team, we are choosing to remove all comparisons to verif_data/ which contain random-generated numbers since these tests cannot pass across all supported trick platforms. * Fix the wrong rule on exlcuding -Werror for Darwin builds of SIM_mc_generation * Remove data recording of strings in SIM_mc_generation Trickops: * Replace build_command with build_args per discussion w/ Trick team Since we only support arguments to trick-CP, replace the build_command yaml entry with build_args * Disable var server connection by default in SingleRun if TrickWorkflow.quiet is True * Guard against multiple Job starts * Remove SimulationJob inheritance layer since old monte-carlo wasn't and never will be supported by TrickOps * Ignore IOError raise from variable_server that looks like "The remote endpoint has closed the connection". This appears to occur when SingleRun jobs attempt to connect to the var server for a sim that terminates very early [skip ci] * Adjust phasing of old/new MCG initialize functions * Clarify failure message in generate_dispersions if new/old MC are both used. * Adjust the phasing order of MCG intialize method to be before legacy MC initialized. Without this, monte-carlo dry run completes with success before the check in generate_dispersions() can run * Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation since gcc 8+ warns about SWIG generated content in top.cpp * Introduce MonteCarloGenerationHelper python class This new class provides an easy-to-use interface for MCG sim-module users: 1. Run generation 2. Getting an sbatch array job suitable for SLURM 3. Getting a list of SingleRun() instances for generated runs, to be executed locally if desired --------- Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
125 lines
4.8 KiB
C++
125 lines
4.8 KiB
C++
/*******************************TRICK HEADER******************************
|
|
PURPOSE: (Provides the front-end interface to the monte-carlo model)
|
|
|
|
LIBRARY DEPENDENCY:
|
|
((../src/mc_master.cc))
|
|
|
|
PROGRAMMERS:
|
|
(((Gary Turner) (OSR) (October 2019) (Antares) (Initial)))
|
|
(((Isaac Reaves) (NASA) (November 2022) (Integration into Trick Core)))
|
|
**********************************************************************/
|
|
#ifndef CML_MONTE_CARLO_MASTER_HH
|
|
#define CML_MONTE_CARLO_MASTER_HH
|
|
|
|
#include <string>
|
|
#include <list>
|
|
#include <utility> // std::pair
|
|
|
|
#include "mc_variable.hh"
|
|
#include "mc_variable_file.hh"
|
|
#include "mc_python_code.hh"
|
|
#include "mc_variable_random.hh"
|
|
#include "mc_variable_fixed.hh"
|
|
#include "mc_variable_semi_fixed.hh"
|
|
|
|
/*****************************************************************************
|
|
MonteCarloMaster
|
|
Purpose:()
|
|
*****************************************************************************/
|
|
class MonteCarloMaster
|
|
{
|
|
public:
|
|
bool active; /* (--)
|
|
The main active flag determining whether an input file should be
|
|
processed for a monte-carlo run. This flag is used to manage the
|
|
configuration of the scenario, including things like which variables
|
|
to log.*/
|
|
bool generate_dispersions; /* (--)
|
|
This flag controls whether the variables should be loaded and
|
|
dispersed. It has no effect if the active flag is off.
|
|
False: Configure the run for MC; this
|
|
configuration typically uses a previously-generated
|
|
monte_input.py file; it does not read in MC variables and
|
|
does not generate new monte_input files.
|
|
True: Use this execution of the sim to read in the MC variables and
|
|
generating the monte-input files. After doing so, the
|
|
execution will terminate.
|
|
The sim can then be re-run using one of the new monte_input files.
|
|
Default: true. This is set to false in the monte_input.py files to
|
|
allow the base input file to be processed from within the
|
|
monte-input file without regenerting the monte-input files.*/
|
|
std::string run_name; /* (--)
|
|
The name of the scenario, used in generating the "MONTE_<run_name>"
|
|
directory, which contains all of the runs.*/
|
|
std::string monte_dir; /* (--)
|
|
The name of the MONTE<RUN...> directory relative to the SIM directory
|
|
where runs will be generated*/
|
|
std::string input_file_name; /* (--)
|
|
The name of the original file used as the main input file.
|
|
Default: input.py.*/
|
|
bool generate_meta_data; /* (--)
|
|
Flag indicating whether to generate the meta-data output file.*/
|
|
bool generate_summary; /* (--)
|
|
Flag indicating whether to generate the dispersion summary file. */
|
|
int minimum_padding; /* (--)
|
|
The minimum width of the run-number field, e.g. RUN_1 vs RUN_00001;
|
|
The run-number field will be sized to the minimum of this value or the
|
|
width necessary to accommodate the highest number.
|
|
Defaults to 0. */
|
|
size_t monte_run_number; /* (--)
|
|
A unique identifying number for each run.*/
|
|
|
|
protected:
|
|
bool input_files_prepared; /* (--)
|
|
Internal flag indicating that the input files have been generated and
|
|
waiting for execution. Effectively blocks further modifications to
|
|
variables after this flag has been set.*/
|
|
|
|
std::string location; /* (--)
|
|
The location in the main sim by which this instance may be addressed
|
|
in an input file. */
|
|
|
|
std::list<MonteCarloVariable *> variables; /* (--)
|
|
A STL-list of pointers to instances of all the base-class
|
|
MonteVarVariable instances. Note that this has to be a list of pointers
|
|
rather than instances because the actual instances are polymorphic;
|
|
making a copy would restrict them to be actual MonteCarloVariable
|
|
instances and we need the polymorphic capabilities for generating the
|
|
monte_input command.*/
|
|
|
|
unsigned int num_runs; /* (--)
|
|
The number of runs to execute for this scenario.*/
|
|
|
|
private:
|
|
std::list< std::pair< std::string,
|
|
MonteCarloVariableFile *> > file_list; /* (--)
|
|
A list of filenames being read as part of the MonteVarFile variables
|
|
being managed by this class. */
|
|
|
|
|
|
public:
|
|
MonteCarloMaster(std::string location);
|
|
virtual ~MonteCarloMaster(){};
|
|
|
|
void activate( std::string run_name);
|
|
bool prepare_input_files();
|
|
void add_variable( MonteCarloVariable & variable);
|
|
MonteCarloVariable * find_variable( std::string var_name);
|
|
void remove_variable( std::string var_name);
|
|
void set_num_runs( unsigned int num_runs);
|
|
void execute();
|
|
void collate_meta_data();
|
|
|
|
private:
|
|
static bool seed_sort( std::pair< unsigned int, std::string> left,
|
|
std::pair< unsigned int, std::string> right)
|
|
{
|
|
return left.first < right.first;
|
|
}
|
|
|
|
// and undefined:
|
|
MonteCarloMaster (const MonteCarloMaster&);
|
|
MonteCarloMaster& operator = (const MonteCarloMaster&);
|
|
};
|
|
#endif
|