trick/include/trick/mc_master.hh
ddj116 9099792947
Integrate MonteCarloGenerate capability from EG CML and associated TrickOps enhancements (#1415)
* Provide MonteCarloGenerate capability

Intermediate commit, this squash represents all of Isaac Reaves' work
during his Fall 2022 Pathways internship tour

[skip ci]

* TrickOps: Add phase, [min-max] range, and overhaul YAML verification

* Add new "phase:" mechanism to TrickOps Runs and Builds to support
  project-specific constraints on build and run ordering
  - phase defaults to zero if not specified and must be between -1000
    and 1000 if given.
  - jobs can now optionally be requested by their phase or phase range
  - See trickops/README.md for details
* Add [min-max] notation capability to run: entries and compare: entries
  - [min-max] ranges provide definition of a set of runs using a common
    numbering scheme in the YAML file, greatly reducing YAML file size
    for monte-carlo and other zero-padded run numbering use cases
  - See trickops/README.md for details
* YAML parsing changes
  - Overhaul the logic which verifies YAML files for the expected
    TrickOps format. This is now done in TrickWorkflowYamlVerifier and
    provides much more robust error checking than previous approach
  - .yaml_requirements.yml now provides the required types, ranges, and
    default values as applicable to expected entries in YAML files
  - valgrind: is now an sub-option to run: entries, not its own section
    Users should now list their runs normallly and define their flags in
    in that run's valgrind: subsection
  - parallel_safety is now a per-sim parameter and not global. Users
    should move their global config to the sim layer
  - self.config_errors is now a list of errors. Users should now
    check for empty list when using instead of True/False
* Robustify the get_koviz_report_jobs unit test to work whether koviz
  exists on PATH or not
* Adjust trickops.py to use the new phase and range features
   - Make it more configurable on the command-line via argparse
   - Move SIM_mc_generation tests into test_sims.yml

[skip ci]

* Code review and cleanup from PR #1389

Documentation:

* Adjust documentation to fit suggested symlinked approach. Also
  cleaned up duplicate images and old documentation.
* Moved the verification section out of markdown and into a PDF since it
  heavily leverages formatting not available in markdown.
* Clarify a couple points on the Darwin Trick install guide
* Update wiki to clarify that data recording strings is not supported

MCG Code:

* Replace MonteCarloVariableRandomNormal::is_near_equal with new
  Trick::dbl_is_near from trick team

MCG Testing:

* Reduce the set of SIM_mc_generation comparisons. After discussion
  the trick team, we are choosing to remove all comparisons to
  verif_data/ which contain random-generated numbers since
  these tests cannot pass across all supported trick platforms.
* Fix the wrong rule on exlcuding -Werror for Darwin builds
  of SIM_mc_generation
* Remove data recording of strings in SIM_mc_generation

Trickops:

* Replace build_command with build_args per discussion w/ Trick team
  Since we only support arguments to trick-CP, replace the build_command
  yaml entry with build_args
* Disable var server connection by default in SingleRun if TrickWorkflow.quiet
  is True
* Guard against multiple Job starts
* Remove SimulationJob inheritance layer since old monte-carlo wasn't
  and never will be supported by TrickOps
* Ignore IOError raise from variable_server that looks like "The remote
  endpoint has closed the connection". This appears to occur when
  SingleRun jobs attempt to connect to the var server for a sim that
  terminates very early

[skip ci]

* Adjust phasing of old/new MCG initialize functions

* Clarify failure message in generate_dispersions if new/old MC are both
  used.
* Adjust the phasing order of MCG intialize method to be before
  legacy MC initialized. Without this, monte-carlo dry run completes with
  success before the check in generate_dispersions() can run
* Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation
  since gcc 8+ warns about SWIG generated content in top.cpp

* Introduce MonteCarloGenerationHelper python class

This new class provides an easy-to-use interface for MCG sim-module
users:

1. Run generation
2. Getting an sbatch array job suitable for SLURM
3. Getting a list of SingleRun() instances for generated runs, to be
   executed locally if desired

---------

Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-03-06 09:25:50 -06:00

125 lines
4.8 KiB
C++

/*******************************TRICK HEADER******************************
PURPOSE: (Provides the front-end interface to the monte-carlo model)
LIBRARY DEPENDENCY:
((../src/mc_master.cc))
PROGRAMMERS:
(((Gary Turner) (OSR) (October 2019) (Antares) (Initial)))
(((Isaac Reaves) (NASA) (November 2022) (Integration into Trick Core)))
**********************************************************************/
#ifndef CML_MONTE_CARLO_MASTER_HH
#define CML_MONTE_CARLO_MASTER_HH
#include <string>
#include <list>
#include <utility> // std::pair
#include "mc_variable.hh"
#include "mc_variable_file.hh"
#include "mc_python_code.hh"
#include "mc_variable_random.hh"
#include "mc_variable_fixed.hh"
#include "mc_variable_semi_fixed.hh"
/*****************************************************************************
MonteCarloMaster
Purpose:()
*****************************************************************************/
class MonteCarloMaster
{
public:
bool active; /* (--)
The main active flag determining whether an input file should be
processed for a monte-carlo run. This flag is used to manage the
configuration of the scenario, including things like which variables
to log.*/
bool generate_dispersions; /* (--)
This flag controls whether the variables should be loaded and
dispersed. It has no effect if the active flag is off.
False: Configure the run for MC; this
configuration typically uses a previously-generated
monte_input.py file; it does not read in MC variables and
does not generate new monte_input files.
True: Use this execution of the sim to read in the MC variables and
generating the monte-input files. After doing so, the
execution will terminate.
The sim can then be re-run using one of the new monte_input files.
Default: true. This is set to false in the monte_input.py files to
allow the base input file to be processed from within the
monte-input file without regenerting the monte-input files.*/
std::string run_name; /* (--)
The name of the scenario, used in generating the "MONTE_<run_name>"
directory, which contains all of the runs.*/
std::string monte_dir; /* (--)
The name of the MONTE<RUN...> directory relative to the SIM directory
where runs will be generated*/
std::string input_file_name; /* (--)
The name of the original file used as the main input file.
Default: input.py.*/
bool generate_meta_data; /* (--)
Flag indicating whether to generate the meta-data output file.*/
bool generate_summary; /* (--)
Flag indicating whether to generate the dispersion summary file. */
int minimum_padding; /* (--)
The minimum width of the run-number field, e.g. RUN_1 vs RUN_00001;
The run-number field will be sized to the minimum of this value or the
width necessary to accommodate the highest number.
Defaults to 0. */
size_t monte_run_number; /* (--)
A unique identifying number for each run.*/
protected:
bool input_files_prepared; /* (--)
Internal flag indicating that the input files have been generated and
waiting for execution. Effectively blocks further modifications to
variables after this flag has been set.*/
std::string location; /* (--)
The location in the main sim by which this instance may be addressed
in an input file. */
std::list<MonteCarloVariable *> variables; /* (--)
A STL-list of pointers to instances of all the base-class
MonteVarVariable instances. Note that this has to be a list of pointers
rather than instances because the actual instances are polymorphic;
making a copy would restrict them to be actual MonteCarloVariable
instances and we need the polymorphic capabilities for generating the
monte_input command.*/
unsigned int num_runs; /* (--)
The number of runs to execute for this scenario.*/
private:
std::list< std::pair< std::string,
MonteCarloVariableFile *> > file_list; /* (--)
A list of filenames being read as part of the MonteVarFile variables
being managed by this class. */
public:
MonteCarloMaster(std::string location);
virtual ~MonteCarloMaster(){};
void activate( std::string run_name);
bool prepare_input_files();
void add_variable( MonteCarloVariable & variable);
MonteCarloVariable * find_variable( std::string var_name);
void remove_variable( std::string var_name);
void set_num_runs( unsigned int num_runs);
void execute();
void collate_meta_data();
private:
static bool seed_sort( std::pair< unsigned int, std::string> left,
std::pair< unsigned int, std::string> right)
{
return left.first < right.first;
}
// and undefined:
MonteCarloMaster (const MonteCarloMaster&);
MonteCarloMaster& operator = (const MonteCarloMaster&);
};
#endif