Commit Graph

89 Commits

Author SHA1 Message Date
Deans
99ee88a686 Add more Variable Server unit and integration tests, clean up and clarify naming 2023-06-26 12:23:58 -05:00
Jacqueline Deans
0788dcfa9b Replace TrickComm with new connection_handler library 2023-06-26 12:23:58 -05:00
Jacqueline Deans
c2e42f4ef4 Refactor and test Variable Server.
- Split VariableServerThread into VariableServerSession and VariableReference classes
- Use C++ streams for data handling
- Unit tests
2023-06-26 12:23:58 -05:00
Mrockwell2
f64a16ecaa
Fixed an error with checkpointing change variables (#1518)
* Fixed SIGABRT for checkpointing change variables

* Expanded the Data Recording Checkpoint test sim
  - Added a case for checkpointing change variables

* Updated the test case for checkpointing change vars
  - Changed the header, input, and log files to get better data representation

* Expanded the Checkpoint test sim
  - Added a test case covering when the only tracked variable is the change var.
2023-06-07 15:17:24 -05:00
Alex Lin
ad42376d5e
Variable Server byteswapping crashes the sim #1513 (#1514)
* Variable Server byteswapping crashes the sim #1513

The code to byteswap a variable server buffer has a comment saying there is a bug. The comment is
correct.  The original code would swap the parameter and anything else that followed that parameter
in the structure.  Crashes everywhere.  Created a new routine that byteswaps a single parameter.
Strangely we didn't have such a routine until now.  Did some testing of doubles, floats, ints,
shorts, and chars and all were swapped correctly.

* Variable Server byteswapping crashes the sim #1513

enabling binary byteswap test.
2023-06-01 13:20:14 -05:00
Jacqueline Deans
d76de7e1ee
Allow for read only S_sie.resource (#1420) 2023-04-17 17:23:48 -05:00
Jacqueline Deans
7d2d5d870c
Fix 32 bit build (#1487) 2023-04-17 15:32:59 -05:00
Jacqueline Deans
fc8eb338f1
Add test for a multicast variable server session (#1480) 2023-04-04 11:02:32 -05:00
Jacqueline Deans
02a1f95121
Allow integ_loop, logging, data_record to be reordered (#1472) 2023-03-10 10:20:33 -06:00
Jacqueline Deans
19016b4817
job_class_order no longer accidentally disables some system job classes (#1465) 2023-03-07 08:43:39 -06:00
ddj116
9099792947
Integrate MonteCarloGenerate capability from EG CML and associated TrickOps enhancements (#1415)
* Provide MonteCarloGenerate capability

Intermediate commit, this squash represents all of Isaac Reaves' work
during his Fall 2022 Pathways internship tour

[skip ci]

* TrickOps: Add phase, [min-max] range, and overhaul YAML verification

* Add new "phase:" mechanism to TrickOps Runs and Builds to support
  project-specific constraints on build and run ordering
  - phase defaults to zero if not specified and must be between -1000
    and 1000 if given.
  - jobs can now optionally be requested by their phase or phase range
  - See trickops/README.md for details
* Add [min-max] notation capability to run: entries and compare: entries
  - [min-max] ranges provide definition of a set of runs using a common
    numbering scheme in the YAML file, greatly reducing YAML file size
    for monte-carlo and other zero-padded run numbering use cases
  - See trickops/README.md for details
* YAML parsing changes
  - Overhaul the logic which verifies YAML files for the expected
    TrickOps format. This is now done in TrickWorkflowYamlVerifier and
    provides much more robust error checking than previous approach
  - .yaml_requirements.yml now provides the required types, ranges, and
    default values as applicable to expected entries in YAML files
  - valgrind: is now an sub-option to run: entries, not its own section
    Users should now list their runs normallly and define their flags in
    in that run's valgrind: subsection
  - parallel_safety is now a per-sim parameter and not global. Users
    should move their global config to the sim layer
  - self.config_errors is now a list of errors. Users should now
    check for empty list when using instead of True/False
* Robustify the get_koviz_report_jobs unit test to work whether koviz
  exists on PATH or not
* Adjust trickops.py to use the new phase and range features
   - Make it more configurable on the command-line via argparse
   - Move SIM_mc_generation tests into test_sims.yml

[skip ci]

* Code review and cleanup from PR #1389

Documentation:

* Adjust documentation to fit suggested symlinked approach. Also
  cleaned up duplicate images and old documentation.
* Moved the verification section out of markdown and into a PDF since it
  heavily leverages formatting not available in markdown.
* Clarify a couple points on the Darwin Trick install guide
* Update wiki to clarify that data recording strings is not supported

MCG Code:

* Replace MonteCarloVariableRandomNormal::is_near_equal with new
  Trick::dbl_is_near from trick team

MCG Testing:

* Reduce the set of SIM_mc_generation comparisons. After discussion
  the trick team, we are choosing to remove all comparisons to
  verif_data/ which contain random-generated numbers since
  these tests cannot pass across all supported trick platforms.
* Fix the wrong rule on exlcuding -Werror for Darwin builds
  of SIM_mc_generation
* Remove data recording of strings in SIM_mc_generation

Trickops:

* Replace build_command with build_args per discussion w/ Trick team
  Since we only support arguments to trick-CP, replace the build_command
  yaml entry with build_args
* Disable var server connection by default in SingleRun if TrickWorkflow.quiet
  is True
* Guard against multiple Job starts
* Remove SimulationJob inheritance layer since old monte-carlo wasn't
  and never will be supported by TrickOps
* Ignore IOError raise from variable_server that looks like "The remote
  endpoint has closed the connection". This appears to occur when
  SingleRun jobs attempt to connect to the var server for a sim that
  terminates very early

[skip ci]

* Adjust phasing of old/new MCG initialize functions

* Clarify failure message in generate_dispersions if new/old MC are both
  used.
* Adjust the phasing order of MCG intialize method to be before
  legacy MC initialized. Without this, monte-carlo dry run completes with
  success before the check in generate_dispersions() can run
* Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation
  since gcc 8+ warns about SWIG generated content in top.cpp

* Introduce MonteCarloGenerationHelper python class

This new class provides an easy-to-use interface for MCG sim-module
users:

1. Run generation
2. Getting an sbatch array job suitable for SLURM
3. Getting a list of SingleRun() instances for generated runs, to be
   executed locally if desired

---------

Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-03-06 09:25:50 -06:00
Jacqueline Deans
5f6a5de070
Expand SIM_test_varserver (#1459)
* Add tests for alternate ways to open VS port

* Don't connect to varserv when quiet=true in trickops

* Add print to try to help debug hanging trickops test

* Handle multicast connect failures gracefully

* Multicast is disabled by default on mac

* Forgot an important return value

* Take away retries

* Fix issue with restart test

* Revert trickops debugging changes

* Remove debugging accidentally left in [no ci]

* whoops

* Allow retries

* Update trickops.py

* sim test adjustments

* Add docs [no ci]

* wording [no ci]

* Cleanup

* Remove large messages, test that one in unit tests
2023-02-24 14:44:55 -06:00
Jacqueline Deans
a8c9572186
Correct check for unsupported types in data recording (#1458) 2023-02-16 16:27:47 -06:00
Jacqueline Deans
9c7becb4ff
Handle thread shutdown gracefully and test (#1448)
---------

Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-02-08 17:58:59 -06:00
Jacqueline Deans
930dfbc683
Add check and warning for unsupported types in data recording (#1441) 2023-02-07 16:41:53 -06:00
Jacqueline Deans
8be3e30f7a
Fix Mac pipeline (#1454) 2023-02-07 15:35:22 -06:00
Jacqueline Deans
fb25ea70d2
Fix undefined behavior in vs_format_ascii (#1446) 2023-02-07 13:11:51 -06:00
Jacqueline Deans
8821bac18a
Handle empty PYTHON_MODULE in header comment (#1432) 2023-01-23 08:50:13 -06:00
Jacqueline Deans
5ff52a01c5
Add option to terminate sim on python event error (#1434) 2023-01-20 15:46:10 -06:00
Jacqueline Deans
807161b19c
Fix 32 bit pipeline (#1430) 2023-01-16 15:54:23 -06:00
Jacqueline Deans
77fa8d78b0
Time tic precision fix and test (#1428) 2023-01-16 08:29:41 -06:00
Jacqueline Deans
97cf685b6a
Parser and test for var_binary (#1425) 2023-01-11 09:27:30 -06:00
Jacqueline Deans
a8daf4514c
Improvements to SIM_test_varserv (#1414)
* Add freeze test to SIM_test_varserv; make mode test more comprehensive
2022-12-15 12:53:39 -06:00
Jacqueline Deans
48f6f76ef3
Fix various issues in scheduled CI jobs (#1419)
- Update S_overrides.mk for SIM_test_varserv so it links correctly in 32 bit (why did only that one have a problem? the world will never know)
- Stop testing Debian Bookworm on python2.7 since support was removed
- Fix some TrickOps unit tests
2022-12-14 17:04:09 -06:00
Jacqueline Deans
90b5cdcb3c
Variable Server Integration Test (#1406)
* Added restart to trickops script; updated varserver integration test
2022-12-05 15:24:20 -06:00
Jacqueline Deans
1b394e9386
Add test sim for interaction between data recording and checkpointing (#1398)
* Add test sim and documentation for interaction between data recording and checkpointing
* Dump failing logs from first phase jobs
2022-11-21 11:57:19 -06:00
Jacqueline Deans
d9e7ac250d
Refactored sim tests to run through TrickOps (#1390) 2022-11-17 15:43:29 -06:00
jmpenn
2a03ff5cf4
Replace sprintf with snprintf in all of Trick source. #1384 (#1392)
* Replace sprintf with snprintf in all of Trick source. #1384

* Don't add -Werror for MacOS because of deprecated sprintf warnings which we cant get rid of because SWIG. #1384

* Fixed an unbalanced parenthesis in S_overrides.mk. #1384
2022-11-15 15:00:05 -06:00
Scott Fennell
d0511e8fb9
1372 use temporary job pointer instead of modifying curr_job in exec_set_time_tic_value
* add integration test sim SIM_exec_set_time_tic_value
* use temporary job pointer instead of modifying curr_job in exec_set_time_tic_value
closes #1372
2022-10-28 17:19:54 -05:00
Jacqueline Deans
2519c914a5
Pair checkpointing fix (#1356)
* Fix pair checkpointing bug, add tests
* Documentation update
2022-10-07 17:00:09 -05:00
Jacqueline Deans
3d2e8e52ca
Set checkpointing fix (#1357)
* Fix set checkpointing, add tests
* Documentation update
2022-10-07 14:24:21 -05:00
Jacqueline Deans
f19ba7df78
Test and Document STL Checkpointing (#1355)
* Updates SIM_stl to include checkpoint writing and checkpoint restore, as well as adding more data structures to test
* Thoroughly tests supported STL types with MM_stl_checkpoint and MM_stl_restore
* Adds an option to enable or disable STL restore in accessible interfaces and changes default to true
* Updates documentation on STL checkpointing to clearly state limitations and known bugs
2022-10-07 08:37:09 -05:00
Jacqueline Deans
2deb7cd180
Sit out SIM_test_varserv
SIM_test_varserv fails intermittently for no apparent reason; take it out of CI until it can be debugged.
2022-08-24 13:34:03 -05:00
Jacqueline Deans
9c8799610f
Variable Server - var_send_once and integration test (#1330)
* Adds the send_once command and message type, which allows a user to request a variable to be sent immediately and only once (intended to replace the var_add, var_send, var_clear idiom that is commonly used for this purpose)
* Minor refactoring of variable server internals to reduce repeated code
* Adds SIM_test_varserv to integration test to test basic variable server functionality
* Changes graphics client for SIM_billiards to use var_send_once as an example of intended use
* Add documentation for var_send_once in docs and tutorial
* Set exit_code_enabled in trick unit tests to be true by default
* Patch for failing bookworm build
2022-08-18 10:47:07 -05:00
Scott Fennell
7c58425f9a
#1258 add sim compile test for anonymous enum (#1259) 2022-04-28 12:22:21 -05:00
Penn, John M 047828115
e487e5609f Initial excision of DMTCP from Trick. #28 2021-08-27 19:38:56 -05:00
Derek Bankieris
2c06ae6eaf Rename SIM_*/trick to SIM_*/.trick
Hide the non-zipped Python modules to indicate to users that changing
them will have no effect on the sim.

Refs #1144
2021-05-19 17:33:13 -05:00
Derek Bankieris
8d314fa1b1 Zip SWIG-generated python modules
Closes #1144
2021-05-19 17:23:33 -05:00
Scott Fennell
9589c1062c
in ICG: Add FileSkipped preprocessor callback to FindTrickICG (#1125)
* #608 add implementation of FileSkipped callback to FindTrickICG to add include chains for headers that have already been preprocessed

* #608 add test SIM for FindTrickICG offsets SIM_test_icg_file_skipped
2021-04-19 19:34:17 -05:00
Scott Fennell
c0791b46d9
icg default constructor is deleted (#1109)
* check if default constructor is deleted #1094

* unit test icg deleted default constructor #1094

* Update centos 7 swig version to 3.0 to support deleted function in SIM_deleted_default_constructor test

Co-authored-by: Fennell, Scott P 263712616 <scott.p.fennell@nasa.gov>

closes #1094
2021-02-05 19:32:36 -06:00
Scott Fennell
f04dcd7567
Small changes to test builds to be compatible with latest macOS and tools (#1078)
closes #1077 

Co-authored-by: Fennell, Scott P 263712616 <scott.p.fennell@nasa.gov>
2020-11-16 22:56:56 -06:00
Scott Fennell
b724f502b3
fix 32 bit test (#953)
* fix tests for 32 bit mode

* fix issues with mongoose build

* Update docker.yml
2020-02-14 10:23:19 -06:00
Scott Fennell
4ac0e75d10 Fix python 3 syntax test/SIM_events 2020-01-09 14:28:20 -06:00
Derek Bankieris
5a7ec20606 Support default member initializer for templates
Closes #918
2019-12-11 14:19:10 -06:00
Scott Fennell
ff771f7bd6
Centos8 compat (#908)
make print statements in tests python3 compatible
2019-11-18 10:10:33 -06:00
Scott Fennell
0d5aa45c20
Python3 exec open (#907)
* update .gitignore

* #906 change all python execfile() to exec(open().read())
2019-11-14 13:35:57 -06:00
Derek Bankieris
1f33406c95 Don't trace TRICK_EXT_LIB_DIRS lib deps in CP
Refs #871
2019-10-07 14:12:06 -05:00
Scott Fennell
82bae3b0d0 #836 remove all S_default.dat stuff 2019-07-09 15:58:26 -05:00
Derek Bankieris
67decaed24 Add support for -isystem
Closes #407
2019-06-20 10:12:33 -05:00
Derek Bankieris
6c9326f760 Add makefile function for target-specific variable
Closes #672
2019-06-19 15:57:26 -05:00