Commit Graph

215 Commits

Author SHA1 Message Date
Hong Chen
e8508ea575
Add calls for getting the total number of data recording groups and the data recording group pointer by its id number ()
* Added new function calls to data_record_utilities for getting the total number of data recording groups and the data recording group pointer by its id number per customer request.

* Deleted unnecessary variable.

* Added a couple of more unit tests and fixed the return number to 0 instead of NULL for getting total number of drgs.
2024-11-12 10:42:46 -06:00
Pherring04
9e974e6a9b
Addressed phantom frame log restart jobs ()
* Frame Log DRG now manage their own jobs (no longer add a phantom restart job)

* Actually removed frame log drg restart jobs this time

* Refactored to avoid duplicate code

* Refactor

* Fixed DR constructors

* Fixed some syntax stuff centos was throwing a tantrum about

* More centos whining

* Centos...

* Centos...

* Clean up
2024-10-29 11:05:33 -05:00
Hong Chen
077064f225
Fixed one warning message at shutdown after checkpoint load and kept the current multicast group after checkpoint load. ()
* Added a check before calling MM delete_var in ExternalApplication destructor;
Made sure that the MulticastGroup is not initialized before initializing it in VariableServerListenThread.cpp;

* Updated to call multicast group initialization to be consistent for the unit test.

* Removed unnecessary command c str pointer.

* Removed unnecessary command c str pointer.
2024-09-26 11:17:57 -05:00
Hong Chen
b46ba50fd0
Added optional remote shell config file for the remote startup command. ()
* Added optional remote sheel config file for the remote startup command.

* Updated to cd to the sim directory after sourcing a config file in case the config file changed pwd.
2024-09-17 10:59:52 -05:00
Thomas Brain
26f6a02e44
Added "target_integ_time" to Integrator class. It can be useful to know the integration time. ()
Co-authored-by: Thomas Brain <thomas.a.brain@nasa.gov>
2024-08-13 10:51:24 -05:00
Hong Chen
a1d151c4c3
Updated not to always pass disown flag for swig pointer conversion instead base on "own" value of the input object. () 2024-05-23 10:37:27 -05:00
Pherring04
5219c1d279
message () 2024-05-23 10:29:53 -05:00
jmpenn
5065d96a15
Pre-increment (rather than post-increment) STL iterators in for loops… ()
* Pre-increment (rather than post-increment) STL iterators in for loops. 

* Fix a goof. 
2024-04-18 11:41:35 -05:00
jmpenn
04822023de
In MemoryManager change delete_var(const char*) to delete_var(std::string) () 2024-04-04 10:33:18 -05:00
jmpenn
c4e60d9f9a
Proposed fix for deadlock on shutdown ()
* Proposed fix for deadlock on shutdown

* Terminate C style comment.

* Added needed libs for applicable tests and updated the logic for when allowing/disallowing connections.

* Need to load additional trick libs for applicable tests for Linux.

---------

Co-authored-by: Hong Chen <hong.chen-1@nasa.gov>
2024-03-28 15:15:19 -05:00
Thomas Brain
48924d585a
python3 GIL locking solution ()
* Implemented use of the python GIL API to replace the less portable mutex solution.

* Replaced nullptr with NULL

---------

Co-authored-by: Thomas Brain <thomas.a.brain@nasa.gov>
2024-03-27 14:20:58 -05:00
Hong Chen
d4f92cc501
Update frame_time attribute in FrameDataRecordGroup from unitless to s. () 2024-03-21 09:52:30 -05:00
Hong Chen
2ac342cfc5
More rusage info added to the end of the run summary output enhancement ()
* Added additional resource usage info for sim run.

* Fixed so voluntary context switches output uses corresponding ru_nvcsw instead of ru_nivcsw due to copy and paste error.

* Removed added resource usage data for page faults and block operations due to inconsistency on different OS.

* Updated to have voluntary and involuntary usage data for both initilization and run for sim shutdown run summary.
2024-02-29 14:52:41 -06:00
jmpenn
bfb6419637
Fix the alloc_type of allocations performed by declare_var. () 2024-01-24 14:06:04 -06:00
Alex Lin
7547d36ab4
Repurpose the "Sim/Real Time" field on the sim control panel ()
* Repurpose the "Sim/Real Time" field on the sim control panel 

Added calculations in the realtime sync monitor job to calculate the
average sim/realtime ratio for the past 100 frames.  This is saved to
a new variable, actual_run_ratio.  Changed the sim control panel to
display the actual_run_ratio value.

* Repurpose the "Sim/Real Time" field on the sim control panel 

how does this compile locally but not in CI?  Didn't include cmath.
2024-01-23 10:46:55 -06:00
Hong Chen
6d86a0fd97
Updated to log applicable time in seconds instead of tics for frame logging ()
* Updated to log applicable time in seconds instead of tics, remove scale attribute in dp product XMLs, and reset clock before realtime clock sync to avoid logging epoch time for frame logging.

* Removed "_seconds" from applicable time variable names.
2024-01-18 10:17:54 -06:00
ddj116
483cacfafd
MonteCarloGenerate: Provide more metadata on dispersions ()
* Add type, dispersion, min_value, max_value and other relevant internal
  members of MonteCarloVariable* classes to the output of
  MonteCarlo_Meta_data_output. Motivation is for users wanting to post-process
  dispersion parameters used during generation of runs
* Protect against invalid memory access when length of values is zero in
  MonteCarloVariableRandomStringSet::generate_assignment(). Add a new
  verif sim warning case to cover these new lines
* Update new verif data for SIM_mc_generation to support these changes

Closes 

Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-12-01 10:09:17 -06:00
Hong Chen
260b5fa558
Changed not to inherit from binary_function for c++11 or above. () 2023-11-16 10:55:13 -06:00
Hong Chen
2c27b65e14
Use PyUnicode_GET_LENGTH when possible otherwise use PyUnicode_GET_SIZE for Python 3 ()
* Use PyUnicode_GET_LENGTH for Python 3.3+ and PyUnicode_GET_SIZE before Python3.3 as GET_SIZE is deprecated in 3.3 and removed in 3.12 while GET_LENGTH is new in 3.3.

* Fixed correct python version for when PyUnicode_GET_LENGTH was new in a comment.
2023-11-16 10:27:50 -06:00
Hong Chen
9076a3e88f
1592 send hs run stats improvement ()
* Added system CPU time and initialization system CPU time and added user wording for existing CPU time to distinguish it from system CPU time used for trick run summary.

* Updated the order of a couple of times of shutdown messages printed on screen and sim/cpu time ratio to both user and system cpu time.

* Minor change for lining up the shutdown message.
2023-10-31 10:20:17 -05:00
jmpenn
cc6fbc4ff2
Fix and unittest lqueue functions in trick_adt. () 2023-10-26 10:26:40 -05:00
jmpenn
9185faf322
Add va_end() to close corresponding va_start(). () 2023-09-21 10:15:32 -05:00
Jacqueline Deans
d3cc021cca Move VariableServerSession logs into subdirectory; make separate toggle for session logs 2023-06-26 12:23:58 -05:00
Deans
a4d49850f3 Add per-session variable server logs 2023-06-26 12:23:58 -05:00
Deans
99ee88a686 Add more Variable Server unit and integration tests, clean up and clarify naming 2023-06-26 12:23:58 -05:00
Jacqueline Deans
0788dcfa9b Replace TrickComm with new connection_handler library 2023-06-26 12:23:58 -05:00
Jacqueline Deans
c2e42f4ef4 Refactor and test Variable Server.
- Split VariableServerThread into VariableServerSession and VariableReference classes
- Use C++ streams for data handling
- Unit tests
2023-06-26 12:23:58 -05:00
Alex Lin
ad42376d5e
Variable Server byteswapping crashes the sim ()
* Variable Server byteswapping crashes the sim 

The code to byteswap a variable server buffer has a comment saying there is a bug. The comment is
correct.  The original code would swap the parameter and anything else that followed that parameter
in the structure.  Crashes everywhere.  Created a new routine that byteswaps a single parameter.
Strangely we didn't have such a routine until now.  Did some testing of doubles, floats, ints,
shorts, and chars and all were swapped correctly.

* Variable Server byteswapping crashes the sim 

enabling binary byteswap test.
2023-06-01 13:20:14 -05:00
Jacqueline Deans
d76de7e1ee
Allow for read only S_sie.resource () 2023-04-17 17:23:48 -05:00
ddj116
9099792947
Integrate MonteCarloGenerate capability from EG CML and associated TrickOps enhancements ()
* Provide MonteCarloGenerate capability

Intermediate commit, this squash represents all of Isaac Reaves' work
during his Fall 2022 Pathways internship tour

[skip ci]

* TrickOps: Add phase, [min-max] range, and overhaul YAML verification

* Add new "phase:" mechanism to TrickOps Runs and Builds to support
  project-specific constraints on build and run ordering
  - phase defaults to zero if not specified and must be between -1000
    and 1000 if given.
  - jobs can now optionally be requested by their phase or phase range
  - See trickops/README.md for details
* Add [min-max] notation capability to run: entries and compare: entries
  - [min-max] ranges provide definition of a set of runs using a common
    numbering scheme in the YAML file, greatly reducing YAML file size
    for monte-carlo and other zero-padded run numbering use cases
  - See trickops/README.md for details
* YAML parsing changes
  - Overhaul the logic which verifies YAML files for the expected
    TrickOps format. This is now done in TrickWorkflowYamlVerifier and
    provides much more robust error checking than previous approach
  - .yaml_requirements.yml now provides the required types, ranges, and
    default values as applicable to expected entries in YAML files
  - valgrind: is now an sub-option to run: entries, not its own section
    Users should now list their runs normallly and define their flags in
    in that run's valgrind: subsection
  - parallel_safety is now a per-sim parameter and not global. Users
    should move their global config to the sim layer
  - self.config_errors is now a list of errors. Users should now
    check for empty list when using instead of True/False
* Robustify the get_koviz_report_jobs unit test to work whether koviz
  exists on PATH or not
* Adjust trickops.py to use the new phase and range features
   - Make it more configurable on the command-line via argparse
   - Move SIM_mc_generation tests into test_sims.yml

[skip ci]

* Code review and cleanup from PR 

Documentation:

* Adjust documentation to fit suggested symlinked approach. Also
  cleaned up duplicate images and old documentation.
* Moved the verification section out of markdown and into a PDF since it
  heavily leverages formatting not available in markdown.
* Clarify a couple points on the Darwin Trick install guide
* Update wiki to clarify that data recording strings is not supported

MCG Code:

* Replace MonteCarloVariableRandomNormal::is_near_equal with new
  Trick::dbl_is_near from trick team

MCG Testing:

* Reduce the set of SIM_mc_generation comparisons. After discussion
  the trick team, we are choosing to remove all comparisons to
  verif_data/ which contain random-generated numbers since
  these tests cannot pass across all supported trick platforms.
* Fix the wrong rule on exlcuding -Werror for Darwin builds
  of SIM_mc_generation
* Remove data recording of strings in SIM_mc_generation

Trickops:

* Replace build_command with build_args per discussion w/ Trick team
  Since we only support arguments to trick-CP, replace the build_command
  yaml entry with build_args
* Disable var server connection by default in SingleRun if TrickWorkflow.quiet
  is True
* Guard against multiple Job starts
* Remove SimulationJob inheritance layer since old monte-carlo wasn't
  and never will be supported by TrickOps
* Ignore IOError raise from variable_server that looks like "The remote
  endpoint has closed the connection". This appears to occur when
  SingleRun jobs attempt to connect to the var server for a sim that
  terminates very early

[skip ci]

* Adjust phasing of old/new MCG initialize functions

* Clarify failure message in generate_dispersions if new/old MC are both
  used.
* Adjust the phasing order of MCG intialize method to be before
  legacy MC initialized. Without this, monte-carlo dry run completes with
  success before the check in generate_dispersions() can run
* Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation
  since gcc 8+ warns about SWIG generated content in top.cpp

* Introduce MonteCarloGenerationHelper python class

This new class provides an easy-to-use interface for MCG sim-module
users:

1. Run generation
2. Getting an sbatch array job suitable for SLURM
3. Getting a list of SingleRun() instances for generated runs, to be
   executed locally if desired

---------

Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-03-06 09:25:50 -06:00
Jacqueline Deans
5f6a5de070
Expand SIM_test_varserver ()
* Add tests for alternate ways to open VS port

* Don't connect to varserv when quiet=true in trickops

* Add print to try to help debug hanging trickops test

* Handle multicast connect failures gracefully

* Multicast is disabled by default on mac

* Forgot an important return value

* Take away retries

* Fix issue with restart test

* Revert trickops debugging changes

* Remove debugging accidentally left in [no ci]

* whoops

* Allow retries

* Update trickops.py

* sim test adjustments

* Add docs [no ci]

* wording [no ci]

* Cleanup

* Remove large messages, test that one in unit tests
2023-02-24 14:44:55 -06:00
Jacqueline Deans
a8c9572186
Correct check for unsupported types in data recording () 2023-02-16 16:27:47 -06:00
jmpenn
9478b6a442
Fix mm_mutex locking and unlocking fiasco in clear_all_vars. () 2023-02-10 14:06:27 -06:00
Jacqueline Deans
9c7becb4ff
Handle thread shutdown gracefully and test ()
---------

Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-02-08 17:58:59 -06:00
jmpenn
afbc78c5e7
Initial commit of compareFloatingPoint utils. ()
* Initial commit of compareFloatingPoint utils.

* Update .gitignore to properly ignore unittest executables.
2023-02-01 15:54:31 -06:00
Jacqueline Deans
5ff52a01c5
Add option to terminate sim on python event error () 2023-01-20 15:46:10 -06:00
Jacqueline Deans
97cf685b6a
Parser and test for var_binary () 2023-01-11 09:27:30 -06:00
Cody Martin
3870dc73b6
Improve tutorial documentation () 2022-12-02 14:55:40 -06:00
jmpenn
2a03ff5cf4
Replace sprintf with snprintf in all of Trick source. ()
* Replace sprintf with snprintf in all of Trick source. 

* Don't add -Werror for MacOS because of deprecated sprintf warnings which we cant get rid of because SWIG. 

* Fixed an unbalanced parenthesis in S_overrides.mk. 
2022-11-15 15:00:05 -06:00
Jacqueline Deans
9d305fec09
Add a way to disable sim objects ()
* Add set_sim_object_jobs_onoff, get_sim_object_on_off, and change set_sim_object_onoff to preserve the on/off status of jobs
2022-11-15 09:16:08 -06:00
Scott Fennell
d0511e8fb9
1372 use temporary job pointer instead of modifying curr_job in exec_set_time_tic_value
* add integration test sim SIM_exec_set_time_tic_value
* use temporary job pointer instead of modifying curr_job in exec_set_time_tic_value
closes 
2022-10-28 17:19:54 -05:00
Jacqueline Deans
2519c914a5
Pair checkpointing fix ()
* Fix pair checkpointing bug, add tests
* Documentation update
2022-10-07 17:00:09 -05:00
Jacqueline Deans
3d2e8e52ca
Set checkpointing fix ()
* Fix set checkpointing, add tests
* Documentation update
2022-10-07 14:24:21 -05:00
Jacqueline Deans
f19ba7df78
Test and Document STL Checkpointing ()
* Updates SIM_stl to include checkpoint writing and checkpoint restore, as well as adding more data structures to test
* Thoroughly tests supported STL types with MM_stl_checkpoint and MM_stl_restore
* Adds an option to enable or disable STL restore in accessible interfaces and changes default to true
* Updates documentation on STL checkpointing to clearly state limitations and known bugs
2022-10-07 08:37:09 -05:00
John M. Penn
8eea9e9ab9 Make MemoryManager::reset_memory and class destructors coordinate the deletion of objects. 2022-09-26 16:54:41 -05:00
Jacqueline Deans
9c8799610f
Variable Server - var_send_once and integration test ()
* Adds the send_once command and message type, which allows a user to request a variable to be sent immediately and only once (intended to replace the var_add, var_send, var_clear idiom that is commonly used for this purpose)
* Minor refactoring of variable server internals to reduce repeated code
* Adds SIM_test_varserv to integration test to test basic variable server functionality
* Changes graphics client for SIM_billiards to use var_send_once as an example of intended use
* Add documentation for var_send_once in docs and tutorial
* Set exit_code_enabled in trick unit tests to be true by default
* Patch for failing bookworm build
2022-08-18 10:47:07 -05:00
Jacqueline Deans
77dae99fc8
Race condition patch ()
* Add patch for an uncommon bug where some data recording threads hang forever
2022-08-10 13:10:18 -05:00
John M. Penn
d94d5fc863 Remove destroy flag from MemoryManager::delete_var(). 2022-07-28 15:00:55 -05:00
Scott Fennell
2dd12fcbb6
parse websocket msg len for correct data ()
closes 
2022-07-27 08:40:40 -05:00
Scott Fennell
878b895588 change get_user_tag_ref to const because its the right thing to do 2022-05-17 12:08:13 -05:00