* Repurposed -OO so all sim run outputs including S_sie.resource are saved to the specified folder. The files are placed in the sub-dir either RUN_xxx or DP_Product of the specified folder.
* Updated for repurposed -OO.
* Added quoting code markdown for such as in order to show <name> as is.
* Added quoting code for missing items.
* Updated to raise an error when the --read-only-sim flag is used without the -O or -OO. Also made updates to the related document accordingly.
* Fixed the error message to be more clear.
* Proposed fix for deadlock on shutdown
* Terminate C style comment.
* Added needed libs for applicable tests and updated the logic for when allowing/disallowing connections.
* Need to load additional trick libs for applicable tests for Linux.
---------
Co-authored-by: Hong Chen <hong.chen-1@nasa.gov>
* Implemented use of the python GIL API to replace the less portable mutex solution.
* Replaced nullptr with NULL
---------
Co-authored-by: Thomas Brain <thomas.a.brain@nasa.gov>
* Added additional resource usage info for sim run.
* Fixed so voluntary context switches output uses corresponding ru_nvcsw instead of ru_nivcsw due to copy and paste error.
* Removed added resource usage data for page faults and block operations due to inconsistency on different OS.
* Updated to have voluntary and involuntary usage data for both initilization and run for sim shutdown run summary.
* Repurpose the "Sim/Real Time" field on the sim control panel #1626
Added calculations in the realtime sync monitor job to calculate the
average sim/realtime ratio for the past 100 frames. This is saved to
a new variable, actual_run_ratio. Changed the sim control panel to
display the actual_run_ratio value.
* Repurpose the "Sim/Real Time" field on the sim control panel #1626
how does this compile locally but not in CI? Didn't include cmath.
* Updated to log applicable time in seconds instead of tics, remove scale attribute in dp product XMLs, and reset clock before realtime clock sync to avoid logging epoch time for frame logging.
* Removed "_seconds" from applicable time variable names.
* Add type, dispersion, min_value, max_value and other relevant internal
members of MonteCarloVariable* classes to the output of
MonteCarlo_Meta_data_output. Motivation is for users wanting to post-process
dispersion parameters used during generation of runs
* Protect against invalid memory access when length of values is zero in
MonteCarloVariableRandomStringSet::generate_assignment(). Add a new
verif sim warning case to cover these new lines
* Update new verif data for SIM_mc_generation to support these changes
Closes#1574
Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
* Added system CPU time and initialization system CPU time and added user wording for existing CPU time to distinguish it from system CPU time used for trick run summary.
* Updated the order of a couple of times of shutdown messages printed on screen and sim/cpu time ratio to both user and system cpu time.
* Minor change for lining up the shutdown message.
* Added input file SHA1 info being printed on screen when running a trick sim with -d option (basically verifying the input)
* Added additional checks to exclude trace info printout for file in /opt or .trick/ or not a real file such as <xxx> as the filename.
* In DPV_textbuffer.cpp, close the opened file on read failure, so we don't have a resource leak. #1561
* Disable 2 tests in VariableServerSessionThread_test.cc: exit_if_handle_message_fails, and exit_if_write_fails.
* Possible speedup for requesting sie file #1555
When creating the sie file there is a marker string that Trick searches
for where it writes the run time allocated memory. The loop that was
searching for the string was iterating one character at a time and reading
from file each time. This was slow. Changed the loop to read 1Mb into
memory at a time and search for the string. Handled the case where the
marker string could straddle the 1Mb boundary by copying a small portion
of the previous part of the file for the next search. For my one test
point sie file generation dropped from 90+ seconds to 3.5 seconds.
* Possible speedup for requesting sie file #1555
Math was wrong on some offset values. Fixed them now.
* Possible speedup for requesting sie file #1555
The mac won't allow me to declare a "char buff[1000001];". It compiles
but throws an exception when run. Changed it to "char *buff = new char[1000001];"
* Added SIM RAM usage info to trick run summary.
* Fixed to use #if for determing which OS.
* Removed the system print out statement when MonitorHealthStatusTask is finished.
* Changed "M" to "MB" for RAM usage info on run summery.
* Fixed SIGABRT for checkpointing change variables
* Expanded the Data Recording Checkpoint test sim
- Added a case for checkpointing change variables
* Updated the test case for checkpointing change vars
- Changed the header, input, and log files to get better data representation
* Expanded the Checkpoint test sim
- Added a test case covering when the only tracked variable is the change var.
* Variable Server byteswapping crashes the sim #1513
The code to byteswap a variable server buffer has a comment saying there is a bug. The comment is
correct. The original code would swap the parameter and anything else that followed that parameter
in the structure. Crashes everywhere. Created a new routine that byteswaps a single parameter.
Strangely we didn't have such a routine until now. Did some testing of doubles, floats, ints,
shorts, and chars and all were swapped correctly.
* Variable Server byteswapping crashes the sim #1513
enabling binary byteswap test.
* Add unittests to cover uncovered code in MemoryManager ADEF_ and REF_ parsers.
* Remove the second production of the 'user_defined_type' target
because it is superfluous, and impossible to reach. This is
because the NAME token is defined as pattern : '[_a-zA-Z][_a-zA-Z0-9:]*'.
This pattern will always match a string of colon separated names.
Note that the NAME itoken is used for colon separated type-name as well as
colon separated variable names.
* Add a test to MM_declare_var_unittest that covers the HEX token in adef_parser.l.
* Add a unittest for MemoryManager_JSON_Intf.cpp.
* Tweak MM_JSON_Intf.cc to get better code-coverage.
* Fix formatting, grammar, and syntax
Additionally, a sentence that referenced something that isn't used anywhere was removed.
* Promote headers, Format stuff
Many headers were far too deep. In some cases, the highest header was h4. To correct this, the offending headers were promoted once or twice as appropriate. Minor formatting changes were made, too.
* Fixed an incorrect conversion constant.
* Update library dependency explanation
The old explanation was confusing and seemingly self contradicting in places. This new explanation aims to fix that.
Co-authored-by: Matthew Elmer <m.elmer@mailbox.org>
* Provide MonteCarloGenerate capability
Intermediate commit, this squash represents all of Isaac Reaves' work
during his Fall 2022 Pathways internship tour
[skip ci]
* TrickOps: Add phase, [min-max] range, and overhaul YAML verification
* Add new "phase:" mechanism to TrickOps Runs and Builds to support
project-specific constraints on build and run ordering
- phase defaults to zero if not specified and must be between -1000
and 1000 if given.
- jobs can now optionally be requested by their phase or phase range
- See trickops/README.md for details
* Add [min-max] notation capability to run: entries and compare: entries
- [min-max] ranges provide definition of a set of runs using a common
numbering scheme in the YAML file, greatly reducing YAML file size
for monte-carlo and other zero-padded run numbering use cases
- See trickops/README.md for details
* YAML parsing changes
- Overhaul the logic which verifies YAML files for the expected
TrickOps format. This is now done in TrickWorkflowYamlVerifier and
provides much more robust error checking than previous approach
- .yaml_requirements.yml now provides the required types, ranges, and
default values as applicable to expected entries in YAML files
- valgrind: is now an sub-option to run: entries, not its own section
Users should now list their runs normallly and define their flags in
in that run's valgrind: subsection
- parallel_safety is now a per-sim parameter and not global. Users
should move their global config to the sim layer
- self.config_errors is now a list of errors. Users should now
check for empty list when using instead of True/False
* Robustify the get_koviz_report_jobs unit test to work whether koviz
exists on PATH or not
* Adjust trickops.py to use the new phase and range features
- Make it more configurable on the command-line via argparse
- Move SIM_mc_generation tests into test_sims.yml
[skip ci]
* Code review and cleanup from PR #1389
Documentation:
* Adjust documentation to fit suggested symlinked approach. Also
cleaned up duplicate images and old documentation.
* Moved the verification section out of markdown and into a PDF since it
heavily leverages formatting not available in markdown.
* Clarify a couple points on the Darwin Trick install guide
* Update wiki to clarify that data recording strings is not supported
MCG Code:
* Replace MonteCarloVariableRandomNormal::is_near_equal with new
Trick::dbl_is_near from trick team
MCG Testing:
* Reduce the set of SIM_mc_generation comparisons. After discussion
the trick team, we are choosing to remove all comparisons to
verif_data/ which contain random-generated numbers since
these tests cannot pass across all supported trick platforms.
* Fix the wrong rule on exlcuding -Werror for Darwin builds
of SIM_mc_generation
* Remove data recording of strings in SIM_mc_generation
Trickops:
* Replace build_command with build_args per discussion w/ Trick team
Since we only support arguments to trick-CP, replace the build_command
yaml entry with build_args
* Disable var server connection by default in SingleRun if TrickWorkflow.quiet
is True
* Guard against multiple Job starts
* Remove SimulationJob inheritance layer since old monte-carlo wasn't
and never will be supported by TrickOps
* Ignore IOError raise from variable_server that looks like "The remote
endpoint has closed the connection". This appears to occur when
SingleRun jobs attempt to connect to the var server for a sim that
terminates very early
[skip ci]
* Adjust phasing of old/new MCG initialize functions
* Clarify failure message in generate_dispersions if new/old MC are both
used.
* Adjust the phasing order of MCG intialize method to be before
legacy MC initialized. Without this, monte-carlo dry run completes with
success before the check in generate_dispersions() can run
* Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation
since gcc 8+ warns about SWIG generated content in top.cpp
* Introduce MonteCarloGenerationHelper python class
This new class provides an easy-to-use interface for MCG sim-module
users:
1. Run generation
2. Getting an sbatch array job suitable for SLURM
3. Getting a list of SingleRun() instances for generated runs, to be
executed locally if desired
---------
Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
* Add tests for alternate ways to open VS port
* Don't connect to varserv when quiet=true in trickops
* Add print to try to help debug hanging trickops test
* Handle multicast connect failures gracefully
* Multicast is disabled by default on mac
* Forgot an important return value
* Take away retries
* Fix issue with restart test
* Revert trickops debugging changes
* Remove debugging accidentally left in [no ci]
* whoops
* Allow retries
* Update trickops.py
* sim test adjustments
* Add docs [no ci]
* wording [no ci]
* Cleanup
* Remove large messages, test that one in unit tests
