* Provide MonteCarloGenerate capability
Intermediate commit, this squash represents all of Isaac Reaves' work
during his Fall 2022 Pathways internship tour
[skip ci]
* TrickOps: Add phase, [min-max] range, and overhaul YAML verification
* Add new "phase:" mechanism to TrickOps Runs and Builds to support
project-specific constraints on build and run ordering
- phase defaults to zero if not specified and must be between -1000
and 1000 if given.
- jobs can now optionally be requested by their phase or phase range
- See trickops/README.md for details
* Add [min-max] notation capability to run: entries and compare: entries
- [min-max] ranges provide definition of a set of runs using a common
numbering scheme in the YAML file, greatly reducing YAML file size
for monte-carlo and other zero-padded run numbering use cases
- See trickops/README.md for details
* YAML parsing changes
- Overhaul the logic which verifies YAML files for the expected
TrickOps format. This is now done in TrickWorkflowYamlVerifier and
provides much more robust error checking than previous approach
- .yaml_requirements.yml now provides the required types, ranges, and
default values as applicable to expected entries in YAML files
- valgrind: is now an sub-option to run: entries, not its own section
Users should now list their runs normallly and define their flags in
in that run's valgrind: subsection
- parallel_safety is now a per-sim parameter and not global. Users
should move their global config to the sim layer
- self.config_errors is now a list of errors. Users should now
check for empty list when using instead of True/False
* Robustify the get_koviz_report_jobs unit test to work whether koviz
exists on PATH or not
* Adjust trickops.py to use the new phase and range features
- Make it more configurable on the command-line via argparse
- Move SIM_mc_generation tests into test_sims.yml
[skip ci]
* Code review and cleanup from PR #1389
Documentation:
* Adjust documentation to fit suggested symlinked approach. Also
cleaned up duplicate images and old documentation.
* Moved the verification section out of markdown and into a PDF since it
heavily leverages formatting not available in markdown.
* Clarify a couple points on the Darwin Trick install guide
* Update wiki to clarify that data recording strings is not supported
MCG Code:
* Replace MonteCarloVariableRandomNormal::is_near_equal with new
Trick::dbl_is_near from trick team
MCG Testing:
* Reduce the set of SIM_mc_generation comparisons. After discussion
the trick team, we are choosing to remove all comparisons to
verif_data/ which contain random-generated numbers since
these tests cannot pass across all supported trick platforms.
* Fix the wrong rule on exlcuding -Werror for Darwin builds
of SIM_mc_generation
* Remove data recording of strings in SIM_mc_generation
Trickops:
* Replace build_command with build_args per discussion w/ Trick team
Since we only support arguments to trick-CP, replace the build_command
yaml entry with build_args
* Disable var server connection by default in SingleRun if TrickWorkflow.quiet
is True
* Guard against multiple Job starts
* Remove SimulationJob inheritance layer since old monte-carlo wasn't
and never will be supported by TrickOps
* Ignore IOError raise from variable_server that looks like "The remote
endpoint has closed the connection". This appears to occur when
SingleRun jobs attempt to connect to the var server for a sim that
terminates very early
[skip ci]
* Adjust phasing of old/new MCG initialize functions
* Clarify failure message in generate_dispersions if new/old MC are both
used.
* Adjust the phasing order of MCG intialize method to be before
legacy MC initialized. Without this, monte-carlo dry run completes with
success before the check in generate_dispersions() can run
* Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation
since gcc 8+ warns about SWIG generated content in top.cpp
* Introduce MonteCarloGenerationHelper python class
This new class provides an easy-to-use interface for MCG sim-module
users:
1. Run generation
2. Getting an sbatch array job suitable for SLURM
3. Getting a list of SingleRun() instances for generated runs, to be
executed locally if desired
---------
Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
* Replace sprintf with snprintf in all of Trick source. #1384
* Don't add -Werror for MacOS because of deprecated sprintf warnings which we cant get rid of because SWIG. #1384
* Fixed an unbalanced parenthesis in S_overrides.mk. #1384
* Merging changes from cmake branch to master
* Fixing includes for renamed header files
* still need build rule
* Adding warning for swig code for gcc8+
* Adding CMakeLists.txt for data products
* Cmake merge #901
Making adjustments to get cmake working on the Mac (Mojave)
* Cmake merge #901
Changing string append to list append
Removed checks on dead units conversion messenger. Units conversion
messages cannot be suppressed. Added deprecated messages to the functions
that modified the message printouts.
Created a new message class that will copy incoming messages to a
buffer area. The buffer area is pre allocated during simulation
initialization. On a separate thread the buffer area is written
to std::cout.
Found more places where symbolic links cause havoc. Also added back a #define
that was removed in parameter_types.h and added back the messaging routines to
the list swig processes.
Fixed the configure script to allow the user to specify using a python3
interpreter. Had to change the the way to load the swig generated python modules.
Also had to change code dealing with python strings, oct, long, and
boolean operations.
refs #258
Copied the units_conv.c from 15 that includes the new units to master.
Swept through the code removing includes to Unit.hh and UCF.h where they
are no longer needed. Remade makefile dependencies.
refs #231
Changed the include for udunits to #include <udunits2.h> to work
on all platforms. Added the include directory /usr/include/udunits2
for Linux.
refs #231
Changed the handling of how we handle doubles and ints allowing
more mathematical operations like multiplication, division, and
some limited exponentiation.
refs #231
Initial checkin of using udunits for measurement units conversions. Replaced
our conversion code in ICG and swig code to use udunits.
Also added code to convert unit symbols that we used in Trick that are different
names in udunits.
Added a sim_object that will eventually allow users to specify a custom
set of files to read units information.
refs #231
For addition and subtraction, we created a new routine that
only scales second units to the first. We apply it to all
conversions when adding and subtracting, but it only changes
the temperature results, because that is the only units that
has a bias factor. The comparision conversion routines were not
changed.
refs #202
Taking in the latest er7_utils directory and adding it into Trick in the
same location it was as an external repository. Made one change to
the files_to_ICG.hh file in the repository to remove the CheckpointHelper
header files. Those go in the Trick files_to_ICG.hh file.
refs #180
Wrote a python routine called get_value_by_ref_name that takes
a variable name string as an argument. The string is passed to
ref_attributes to find the address and returns a REF2 structure. The REF2
structure is passed to ref_to_value and the type and value of the variable
is returned in a structure. We call vval_<type> to extract the raw value
from the structure and return that to python space.
Made a couple of other bug fixes in the FieldVisior and cpp input file
example found while investigating this issue.
refs #123
Well sims compiled, but I forgot to test if Trick itself
would compile. It didn't. Added TRICK_SYSTEM_CFLAGS and
TRICK_SYSTEM_CXXFLAGS where needed.
refs #90
Moved the header files and the swig .i files to include/trick/swig. Adjusted the
source files and the convert_swig script to find the headers in their new
location.
refs #67
Moved the no_arch swig files to share/trick/swig and adjusted the makefiles to look for the files in the new location. Changed the install rule to only copy the files we need for our installation.
refs #63
Reorganized. Created a new top level include directory that will hold all of Trick's header files. Moved all of the Trick headers to this directory. Created a libexec directory that holds all of the executables that users don't need to execute directly. Changed all of the executables remaining in bin to start with "trick-". In the sim_services directories changed all source files to find the Trick headers in their new location. Since all of the include files are gone in sim_services, removed the src directories as well, moving all of the source files up a level. Moved the makefiles, docs, man, and other architecture independent files into a top level share directory. Renamed lib_${TRICK_HOST_CPU} to lib64 or lib depending on the platform we're currently on.
refs #63
Changed all header file once include variables to follow the same naming
convention and not start with any underscores. Also deleted old
incorrect copyright notices. Also removed $Id: tags from all files.
Fixes#14. Fixes#22.
