2015-02-26 15:02:31 +00:00
% m o d u l e s i m _ s e r v i c e s
2016-04-01 22:05:24 +00:00
% {
# i n c l u d e < c s t d def >
Integrate MonteCarloGenerate capability from EG CML and associated TrickOps enhancements (#1415)
* Provide MonteCarloGenerate capability
Intermediate commit, this squash represents all of Isaac Reaves' work
during his Fall 2022 Pathways internship tour
[skip ci]
* TrickOps: Add phase, [min-max] range, and overhaul YAML verification
* Add new "phase:" mechanism to TrickOps Runs and Builds to support
project-specific constraints on build and run ordering
- phase defaults to zero if not specified and must be between -1000
and 1000 if given.
- jobs can now optionally be requested by their phase or phase range
- See trickops/README.md for details
* Add [min-max] notation capability to run: entries and compare: entries
- [min-max] ranges provide definition of a set of runs using a common
numbering scheme in the YAML file, greatly reducing YAML file size
for monte-carlo and other zero-padded run numbering use cases
- See trickops/README.md for details
* YAML parsing changes
- Overhaul the logic which verifies YAML files for the expected
TrickOps format. This is now done in TrickWorkflowYamlVerifier and
provides much more robust error checking than previous approach
- .yaml_requirements.yml now provides the required types, ranges, and
default values as applicable to expected entries in YAML files
- valgrind: is now an sub-option to run: entries, not its own section
Users should now list their runs normallly and define their flags in
in that run's valgrind: subsection
- parallel_safety is now a per-sim parameter and not global. Users
should move their global config to the sim layer
- self.config_errors is now a list of errors. Users should now
check for empty list when using instead of True/False
* Robustify the get_koviz_report_jobs unit test to work whether koviz
exists on PATH or not
* Adjust trickops.py to use the new phase and range features
- Make it more configurable on the command-line via argparse
- Move SIM_mc_generation tests into test_sims.yml
[skip ci]
* Code review and cleanup from PR #1389
Documentation:
* Adjust documentation to fit suggested symlinked approach. Also
cleaned up duplicate images and old documentation.
* Moved the verification section out of markdown and into a PDF since it
heavily leverages formatting not available in markdown.
* Clarify a couple points on the Darwin Trick install guide
* Update wiki to clarify that data recording strings is not supported
MCG Code:
* Replace MonteCarloVariableRandomNormal::is_near_equal with new
Trick::dbl_is_near from trick team
MCG Testing:
* Reduce the set of SIM_mc_generation comparisons. After discussion
the trick team, we are choosing to remove all comparisons to
verif_data/ which contain random-generated numbers since
these tests cannot pass across all supported trick platforms.
* Fix the wrong rule on exlcuding -Werror for Darwin builds
of SIM_mc_generation
* Remove data recording of strings in SIM_mc_generation
Trickops:
* Replace build_command with build_args per discussion w/ Trick team
Since we only support arguments to trick-CP, replace the build_command
yaml entry with build_args
* Disable var server connection by default in SingleRun if TrickWorkflow.quiet
is True
* Guard against multiple Job starts
* Remove SimulationJob inheritance layer since old monte-carlo wasn't
and never will be supported by TrickOps
* Ignore IOError raise from variable_server that looks like "The remote
endpoint has closed the connection". This appears to occur when
SingleRun jobs attempt to connect to the var server for a sim that
terminates very early
[skip ci]
* Adjust phasing of old/new MCG initialize functions
* Clarify failure message in generate_dispersions if new/old MC are both
used.
* Adjust the phasing order of MCG intialize method to be before
legacy MC initialized. Without this, monte-carlo dry run completes with
success before the check in generate_dispersions() can run
* Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation
since gcc 8+ warns about SWIG generated content in top.cpp
* Introduce MonteCarloGenerationHelper python class
This new class provides an easy-to-use interface for MCG sim-module
users:
1. Run generation
2. Getting an sbatch array job suitable for SLURM
3. Getting a list of SingleRun() instances for generated runs, to be
executed locally if desired
---------
Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-03-06 15:25:50 +00:00
# i n c l u d e < random >
2016-04-01 22:05:24 +00:00
% }
2015-06-17 19:07:52 +00:00
# i n c l u d e "trick/swig/trick_swig.i"
# i n c l u d e "trick/swig/swig_class_typedef.i"
# i n c l u d e "trick/swig/cast_as.i"
2016-05-09 13:36:43 +00:00
# i n c l u d e "units_attach.i"
# i n c l u d e "extra_functions.i"
2015-02-26 15:02:31 +00:00
// Special typemap for collectee in add_collect and delete_collect.
% t y p e m a p ( in ) void * c o l l e c t e e {
// VOID *
void * a r g p 2 ;
if ( S W I G _ I s O K ( S W I G _ C o n v e r t P t r ( $ input , &argp2,SWIG_TypeQuery("void *"), 0)) ) {
// We have an address coming in, we don't have to do any translation
$ 1 = r e i n t e r p r e t _ c a s t < $ 1 _ b a s e t y p e * > ( a r g p 2 ) ;
} else if ( S W I G _ I s O K ( S W I G _ C o n v e r t P t r ( $ input , &argp2,SWIG_TypeQuery("swig_ref *"), 0)) ) {
// Array to pointer assignment
s w i g _ r e f * t e m p _ s w i g _ r e f = r e i n t e r p r e t _ c a s t < s w i g _ r e f * > ( a r g p 2 ) ;
if ( t e m p _ s w i g _ r e f ! = NULL ) {
$ 1 = ( $ 1 _ b a s e t y p e * ) t e m p _ s w i g _ r e f - > r e f . a d d r e s s ;
}
// package the array address as a void
}
}
% i n l i n e % {
2015-06-17 19:07:52 +00:00
# i n c l u d e "trick/swig/swig_global_vars.hh"
2015-02-26 15:02:31 +00:00
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/reference_frame.h"
# i n c l u d e "trick/rand_generator.h"
# i n c l u d e "trick/wave_form.h"
# i n c l u d e "trick/integrator_c_intf.h"
2018-12-07 14:08:15 +00:00
# i n c l u d e "trick/units_conv.h"
2015-02-26 15:02:31 +00:00
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/GetTimeOfDayClock.hh"
# i n c l u d e "trick/clock_proto.h"
# i n c l u d e "trick/CommandLineArguments.hh"
# i n c l u d e "trick/command_line_protos.h"
# i n c l u d e "trick/DRAscii.hh"
# i n c l u d e "trick/DRBinary.hh"
2015-02-26 15:02:31 +00:00
# i f def H D F 5
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/DRHDF5.hh"
2015-02-26 15:02:31 +00:00
# e n d i f
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/DataRecordDispatcher.hh"
# i n c l u d e "trick/data_record_proto.h"
# i n c l u d e "trick/DebugPause.hh"
# i n c l u d e "trick/debug_pause_proto.h"
# i n c l u d e "trick/EchoJobs.hh"
# i n c l u d e "trick/echojobs_proto.h"
# i n c l u d e "trick/Environment.hh"
# i n c l u d e "trick/env_proto.h"
# i n c l u d e "trick/Executive.hh"
2018-10-30 18:49:55 +00:00
# i n c l u d e "trick/ExecutiveException.hh"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/exec_proto.h"
# i n c l u d e "trick/exec_proto.hh"
# i n c l u d e "trick/MalfunctionsTrickView.hh"
# i n c l u d e "trick/MonteMonitor.hh"
# i n c l u d e "trick/SimControlPanel.hh"
# i n c l u d e "trick/StripChart.hh"
# i n c l u d e "trick/TrickView.hh"
# i n c l u d e "trick/ExternalApplicationManager.hh"
# i n c l u d e "trick/external_application_c_intf.h"
# i n c l u d e "trick/FrameDataRecordGroup.hh"
# i n c l u d e "trick/FrameLog.hh"
# i n c l u d e "trick/framelog_proto.h"
# i n c l u d e "trick/IPPython.hh"
# i n c l u d e "trick/input_processor_proto.h"
# i n c l u d e "trick/MTV.hh"
# i n c l u d e "trick/EventManager.hh"
# i n c l u d e "trick/EventProcessor.hh"
# i n c l u d e "trick/EventManager_c_intf.hh"
# i n c l u d e "trick/JITInputFile.hh"
# i n c l u d e "trick/jit_input_file_proto.hh"
# i n c l u d e "trick/JSONVariableServer.hh"
# i n c l u d e "trick/IntegLoopScheduler.hh"
# i n c l u d e "trick/IntegLoopManager.hh"
# i n c l u d e "trick/IntegLoopSimObject.hh"
# i n c l u d e "trick/Integrator.hh"
# i n c l u d e "trick/regula_falsi.h"
2015-02-26 15:02:31 +00:00
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/ABM_Integrator.hh"
# i n c l u d e "trick/Euler_Cromer_Integrator.hh"
# i n c l u d e "trick/Euler_Integrator.hh"
# i n c l u d e "trick/MM4_Integrator.hh"
# i n c l u d e "trick/NL2_Integrator.hh"
# i n c l u d e "trick/RK2_Integrator.hh"
# i n c l u d e "trick/RK4_Integrator.hh"
# i n c l u d e "trick/RKF45_Integrator.hh"
# i n c l u d e "trick/RKF78_Integrator.hh"
# i n c l u d e "trick/RKG4_Integrator.hh"
2015-02-26 15:02:31 +00:00
2016-06-15 21:11:09 +00:00
# i n c l u d e "trick/SimTime.hh"
# i n c l u d e "trick/simtime_proto.h"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/MSSocket.hh"
# i n c l u d e "trick/MSSharedMem.hh"
# i n c l u d e "trick/Master.hh"
Integrate MonteCarloGenerate capability from EG CML and associated TrickOps enhancements (#1415)
* Provide MonteCarloGenerate capability
Intermediate commit, this squash represents all of Isaac Reaves' work
during his Fall 2022 Pathways internship tour
[skip ci]
* TrickOps: Add phase, [min-max] range, and overhaul YAML verification
* Add new "phase:" mechanism to TrickOps Runs and Builds to support
project-specific constraints on build and run ordering
- phase defaults to zero if not specified and must be between -1000
and 1000 if given.
- jobs can now optionally be requested by their phase or phase range
- See trickops/README.md for details
* Add [min-max] notation capability to run: entries and compare: entries
- [min-max] ranges provide definition of a set of runs using a common
numbering scheme in the YAML file, greatly reducing YAML file size
for monte-carlo and other zero-padded run numbering use cases
- See trickops/README.md for details
* YAML parsing changes
- Overhaul the logic which verifies YAML files for the expected
TrickOps format. This is now done in TrickWorkflowYamlVerifier and
provides much more robust error checking than previous approach
- .yaml_requirements.yml now provides the required types, ranges, and
default values as applicable to expected entries in YAML files
- valgrind: is now an sub-option to run: entries, not its own section
Users should now list their runs normallly and define their flags in
in that run's valgrind: subsection
- parallel_safety is now a per-sim parameter and not global. Users
should move their global config to the sim layer
- self.config_errors is now a list of errors. Users should now
check for empty list when using instead of True/False
* Robustify the get_koviz_report_jobs unit test to work whether koviz
exists on PATH or not
* Adjust trickops.py to use the new phase and range features
- Make it more configurable on the command-line via argparse
- Move SIM_mc_generation tests into test_sims.yml
[skip ci]
* Code review and cleanup from PR #1389
Documentation:
* Adjust documentation to fit suggested symlinked approach. Also
cleaned up duplicate images and old documentation.
* Moved the verification section out of markdown and into a PDF since it
heavily leverages formatting not available in markdown.
* Clarify a couple points on the Darwin Trick install guide
* Update wiki to clarify that data recording strings is not supported
MCG Code:
* Replace MonteCarloVariableRandomNormal::is_near_equal with new
Trick::dbl_is_near from trick team
MCG Testing:
* Reduce the set of SIM_mc_generation comparisons. After discussion
the trick team, we are choosing to remove all comparisons to
verif_data/ which contain random-generated numbers since
these tests cannot pass across all supported trick platforms.
* Fix the wrong rule on exlcuding -Werror for Darwin builds
of SIM_mc_generation
* Remove data recording of strings in SIM_mc_generation
Trickops:
* Replace build_command with build_args per discussion w/ Trick team
Since we only support arguments to trick-CP, replace the build_command
yaml entry with build_args
* Disable var server connection by default in SingleRun if TrickWorkflow.quiet
is True
* Guard against multiple Job starts
* Remove SimulationJob inheritance layer since old monte-carlo wasn't
and never will be supported by TrickOps
* Ignore IOError raise from variable_server that looks like "The remote
endpoint has closed the connection". This appears to occur when
SingleRun jobs attempt to connect to the var server for a sim that
terminates very early
[skip ci]
* Adjust phasing of old/new MCG initialize functions
* Clarify failure message in generate_dispersions if new/old MC are both
used.
* Adjust the phasing order of MCG intialize method to be before
legacy MC initialized. Without this, monte-carlo dry run completes with
success before the check in generate_dispersions() can run
* Add -Wno-stringop-truncation to S_override.mk for SIM_mc_generation
since gcc 8+ warns about SWIG generated content in top.cpp
* Introduce MonteCarloGenerationHelper python class
This new class provides an easy-to-use interface for MCG sim-module
users:
1. Run generation
2. Getting an sbatch array job suitable for SLURM
3. Getting a list of SingleRun() instances for generated runs, to be
executed locally if desired
---------
Co-authored-by: Dan Jordan <daniel.d.jordan@nasa.gov>
2023-03-06 15:25:50 +00:00
# i n c l u d e "trick/mc_master.hh"
# i n c l u d e "trick/mc_python_code.hh"
# i n c l u d e "trick/mc_variable_file.hh"
# i n c l u d e "trick/mc_variable_fixed.hh"
# i n c l u d e "trick/mc_variable.hh"
# i n c l u d e "trick/mc_variable_random_bool.hh"
# i n c l u d e "trick/mc_variable_random.hh"
# i n c l u d e "trick/mc_variable_random_normal.hh"
# i n c l u d e "trick/mc_variable_random_string.hh"
# i n c l u d e "trick/mc_variable_random_uniform.hh"
# i n c l u d e "trick/mc_variable_semi_fixed.hh"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/Slave.hh"
# i n c l u d e "trick/master_proto.h"
# i n c l u d e "trick/MemoryManager.hh"
# i n c l u d e "trick/memorymanager_c_intf.h"
# i n c l u d e "trick/CheckPointRestart.hh"
# i n c l u d e "trick/CheckPointRestart_c_intf.hh"
# i n c l u d e "trick/collect_proto.hh"
# i n c l u d e "trick/AttributesMap.hh"
# i n c l u d e "trick/sie_c_intf.h"
# i n c l u d e "trick/MessageCout.hh"
2018-01-25 21:59:19 +00:00
# i n c l u d e "trick/MessageThreadedCout.hh"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/MessageFile.hh"
2023-04-18 21:27:32 +00:00
# i n c l u d e "trick/MessageHSFile.hh"
# i n c l u d e "trick/MessageCustomFile.hh"
# i n c l u d e "trick/MessageCustomManager.hh"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/MessageLCout.hh"
# i n c l u d e "trick/MessagePublisher.hh"
# i n c l u d e "trick/MessageSubscriber.hh"
# i n c l u d e "trick/MessageTCDevice.hh"
2017-12-12 16:14:23 +00:00
# i n c l u d e "trick/Message_proto.hh"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/message_proto.h"
# i n c l u d e "trick/MonteCarlo.hh"
# i n c l u d e "trick/montecarlo_c_intf.h"
# i n c l u d e "trick/MonteVarCalculated.hh"
# i n c l u d e "trick/MonteVarFile.hh"
# i n c l u d e "trick/MonteVarFixed.hh"
# i n c l u d e "trick/MonteVarRandom.hh"
# i n c l u d e "trick/RealtimeSync.hh"
# i n c l u d e "trick/realtimesync_proto.h"
# i n c l u d e "trick/RtiExec.hh"
# i n c l u d e "trick/RtiStager.hh"
# i n c l u d e "trick/ITimer.hh"
2018-12-07 14:08:15 +00:00
# i n c l u d e "trick/Unit.hh"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/UnitTest.hh"
# i n c l u d e "trick/trick_tests.h"
# i n c l u d e "trick/VariableServer.hh"
# i n c l u d e "trick/variable_server_proto.h"
2015-09-18 14:03:24 +00:00
# i n c l u d e "trick/vval.h"
2015-06-01 15:28:29 +00:00
# i n c l u d e "trick/Flag.h"
2016-05-05 19:49:49 +00:00
# i n c l u d e "trick/UdUnits.hh"
2021-08-18 18:15:10 +00:00
# i n c l u d e "trick/MyCivetServer.hh"
2015-02-26 15:02:31 +00:00
2015-03-02 18:48:17 +00:00
# i f def U S E _ E R 7 _ U T I L S _ I N T E G R A T O R S
2015-02-26 15:02:31 +00:00
# i n c l u d e "er7_utils/integration/core/include/integrator_constructor_factory.hh"
# i n c l u d e "er7_utils/integration/core/include/integrable_object.hh"
# i n c l u d e "er7_utils/integration/core/include/base_integration_group.hh"
2019-11-19 15:01:16 +00:00
# i n c l u d e "er7_utils/trick/integration/include/trick_first_order_ode_integrator.hh"
# i n c l u d e "er7_utils/trick/integration/include/trick_second_order_ode_integrator.hh"
2015-03-02 18:48:17 +00:00
# e n d i f
2015-02-26 15:02:31 +00:00
% }
% p y t h o n c o d e % {
import o s
import l i n e c a c h e
import r e
c u r r _ d i r = o s . g e t c w d ( )
e x c l u d e _ d i r = o s . g e t c w d ( ) + "/trick"
def t r a c e i t ( frame , event , a r g ) :
if event = = "line" :
l i n e n o = frame . f _ l i n e n o
filename = frame . f _ c o d e . c o _ f i l e n a m e
2023-10-05 15:56:02 +00:00
if ( not filename . s t a r t s w i t h ( e x c l u d e _ d i r ) and not filename . s t a r t s w i t h ( "/usr" ) and not filename . s t a r t s w i t h ( "/opt" ) and not filename . s t a r t s w i t h ( "<" ) and not filename . s t a r t s w i t h ( ".trick/" ) ) :
2015-02-26 15:02:31 +00:00
if ( filename . e n d s w i t h ( ".pyc" ) or
filename . e n d s w i t h ( ".pyo" ) ) :
filename = filename [ : - 1 ]
line = l i n e c a c h e . g e t l i n e ( filename , l i n e n o )
2016-06-28 18:44:21 +00:00
p r i n t ( filename , ":" , l i n e n o , ": " , line . r s t r i p ( ) )
2015-02-26 15:02:31 +00:00
return t r a c e i t
% }
