ExternalVendorCode/trick
1
0
Fork 0
mirror of https://github.com/nasa/trick.git synced 2024-12-18 20:57:55 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add old design documentation under developer_docs

Browse Source
This commit is contained in:
Jacqueline Deans 2023-05-04 08:53:56 -05:00
parent 746311c06e
commit 2ef76cf42a
10 changed files with 1323 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

333
docs/developer_docs/DesConvertSwig.md Normal file
View File

@ -0,0 +1,333 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Convert Swig |
|------------------------------------------------------------------|
# convert_swig
The purpose of convert_swig is to create SWIG interface files for the given
C/C++ header file (usually S_source.hh) and each of the header files that it
(recursively) includes. SWIG (Simplified Wrapper and Interface Generator) is
an interface compiler that connects programs written in C and C++ with scripting
languagues such as Perl and Python.
In Trick the list of header files to be processed is usually produced by the script
make_swig_makefile.pm, as it's creating Makefile_swig. This list is stored in
the file ".S_library_swig". So, if .S_library_swig exists, we can just open and read it.
Otherwise we need to process S_source.hh to produce the list of header files.
Specifically, we want to generate SWIG interfaces for those header files that are:
1. actual dependencies of S_source.hh, GIVEN THE CURRENT environment and
1. not excluded from ICG processing ( by ICG_NO or ICG_EXCLUDE).
The header files that are actually included are the dependencies we care
about. Keep in mind that the preprocessor and the current ENVIRONMENT
may cause some headers to be conditionally included or excluded. We only
want to generate SWIG interfaces for headers that are ACTUALLY included.
Whereas the preprocessor can (using the gcc -MM option) generate a list
of dependencies that satisfy 1) (above), it can't handle that ICG exclusions.
And, whereas the function get_headers() can generate a list of dependences
which are flagged if they contain ICG_NO, it doesn't handle conditional includes.
So, the strategy that we employ is to generate and then find the
intersection of both lists. Then we eliminate those that are in 1)
$TRICK_HOME/trick_source, or 2) flagged as containing ICG_NO or 3) are
in ICG_EXCLUDE'd directories.
First, create a list headers using the GCC with the -MM option. GCC will
handle conditional inclusion.
Second, create a list where the files are flagged if they contain ICG_NO.
Then we generate the intersection of the two lists and then eliminate the dependencies that:
1. are in $TRICK_HOME/trick_source.
1. contain ICG_NO.
1. are in ICG_EXCLUDE'd directories.
to create the final list of header dependencies that we need to convert into SWIG interfaces.
Next we need to determine which of the files do not have up-to-date SWIG files.
For each header file in final dependency list, if the corresponding SWIG (.i) file
doesn't exist or the header file is newer than the existing SWIG file, then record
that a new SWIG file needs needs to be created. The global hash %out_of_date
represents a list of header files whose corresponding .i files need to be regenerated.
Finally, call process_file() to create SWIG interface files for each of the out_of_date headers.
## process_file
### Synopsis
This subroutine processes S_source.h and each of it's requisite header files to
generate the corresponding SWIG interfaces files.
### Parameters
<b>sim_ref</b> Is this parameter ever used?
<b>in_file</b> The name of input file, invariably "S_source.hh".
### Design
This function first reads the contents of <in_file> and each of the header files
that it (recursively) includes into a hash (%file_contents), keyed by the
corresponding filenames. It then converts the contents of the each header whose
corresponding SWIG interface file is out of date into a SWIG interface file.
#### Creating the %file_contents hash
To create the file_contents hash, we first run the input file through the
C/C++ preprocessor with the -dI option. This creates <b>one</b> file containing
the contents of all of the included header files delimited by "line markers".
The line markers indicate which header file each of content came from.
Preprocessor line markers are of the form: '#' <linenum> <filename> <flags>
They are described in Chapter 9, "Preprocessor Output" of the GCC document,
"Preprocessor Output - The C Preprocessor". Whether in_file is run through
the C or C++ preprocessor depends on its name suffix.
For each line in the ONE big file, check whether it's a linemarker or not.
if it's a linemarker ( telling us where the following content is from)
extract the header filename. This will be our current filecontents hash key.
If it's not a linemarker, then it must be content. So, append it to the string,
whose key is the current file name ($curr_name).
#### Creating SWIG interface files
The global hash <i>\%out_of_date</i> represents the list of header files whose
corresponding SWIG interface files are out of date. It is generated in the main
part of the convert_swig program.
For each of these out of date header files, we generate a SWIG interface file
from the contents stored in the <i>\%file_contents</i> hash.
First we remove the friend init_attr functions from the headers content.
They don't need to be wrapped.
Then, for each of the #includes in the out_of_date header file
create a corresponding %import directive.
Next, we generate a module name and path for the SWIG interface file. The module
name is generated from an md5 hash of the header file's full name.
Finally we open the SWIG interface file, and in it we:
1. write a %module directive that identifies the module.
1. write a #include directive to include trick_swig interface utilities.
1. write a #include directive to include the header file to which this
interface file corresponds and from which it was derived.
1. create a SWIG interface for each class declared in the corresponding
header file using the %trick_swig_class_typemap() macro. This macro is
defined in swig_class_typedef.i, included by trick_swig.i (see above).
1. Write the SWIG interface code (processed header file) and the header
file contents.
## process_contents
### Synopsis
Process header file contents for use in the corresponding SWIG interface file.
### Parameters
<b>contents_ref</b>
(IN) reference to header file contents that are to be converted to a SWIG interface.
<b>new_contents_ref</b>
(OUT) SWIG interface code, derived from the header file contents.
<b>curr_namespace</b>
(IN) current namespace.
<b>class_names_ref</b>
(OUT) reference to an array of class and/or struct names encountered when
processing the header file contents.
<b>template_typedefs_ref</b>
(OUT) Series of SWIG %template directives. %template directives create a type in
the target language that corresponds to a C++ template instance.
### Description
While there's header file content remaining to be processed, repeatedly make the
best match with the following available patterns:
Case of :<br>
<b>typedef</b> <i>existing-type-name new-type-name</i> <b>';'</b><br>
Concatenate the matched text to the SWIG interface text.
Case of :<br>
<b>typedef enum</b> <i>optional-name</i> <b>'{'</b> <i>bracketed-content</i> <b>'}'</b> <i>enum-name</i><b>';'</b><br>
Concatenate the matched text to the SWIG interface text.
Case of :<br>
<b>typedef</b> ( <b>struct</b> | <b>union</b> ) <i>name</i> <b>'{'</b><br>
Call process_typedef_struct() to process the matched text and the bracketed
content of the struct that follows in the header file contents and update the
SWIG interface text accordingly.
Case of :<br>
<b>template '<'</b> <i>template-parameters</i> <b>'>' class</b> <i>class-name</i><br>
then just concatenate the matched text to the SWIG interface text.
Case of:<br>
<b>namespace</b> <i>name</i><br>
then call <i>process_namespace()</i> to process the matched text and the
bracketed content that follows in the header file contents and update the the
SWIG interface text accordingly.
Case of:<br>
( <b>class</b> | <b>struct</b> ) <i>class-name</i> ( <b>'{'</b> | <b>':'</b> )<br>
Call <i>process_class()</i> to process the matched text and the bracketed
that follows in the header file contents and update the the SWIG interface text.
Default:
Match anything that doesn't match the other patterns and concatenate it to the
to the SWIG interface text. Note that (in Perl) <b>*?</b> in the regular
expression <b>(.*?)</b> is a non-greedy quantifier, so it gobbles up text only
until another match can be made.
## process_namespace
### Synopsis
Process namespaces found in a header file for use in the corresponding SWIG
interface file.
### Parameters
<b>namespace_string</b>
(IN) This is a string of the form: <b>namespace</b> <i>name</i>, that was
extracted from the header file contents. In the contents there should remain the bracketed
content to which this namespace applies.
<b>contents_ref</b>
(IN) This is a reference to the remainder of the header file (following the
above string) to be processed.
<b>new_contents_ref</b>
(OUT) The SWIG code generated so far.
<b>curr_namespace</b>
(IN) current namespace.
<b>class_names_ref</b>
(OUT) reference to an array of class and/or struct names encountered when
processing the header file contents.
<b>template_typedefs_ref</b>
(OUT) Series of SWIG %template directives. %template directives create a type
in the target language that corresponds to a C++ template instance.
### Description
Extract the name from the <b>namespace_string</b> and append it to the current namespace's name.
Add the <b>namespace_string</b> to the SWIG interface text.
Call extract_bracketed() to extract the contents of the namespace from the header text.
Call process_contents() to convert the extracted namespace contents to a SWIG interface.
Append whatever wasn't matched in process contents to the SWIG interface text.
## process_class
### Synopsis
Process classes declarations found in a header file for use in the corresponding
SWIG interface file.
### Parameters
<b>class_string</b>
(IN) This is a string of the form:<br>
( <b>class</b> | <b>struct</b> ) <i>class-name</i> ( <b>'{'</b> | <b>':'</b> )<br>
<b>contents_ref</b>
(IN) This is a reference to the remainder of the header file (following the
class_string) to be processed.
<b>new_contents_ref</b>
(OUT) The SWIG code generated so far.
<b>curr_namespace</b>
(IN) current namespace.
<b>class_names_ref</b>
(OUT) reference to an array of class and/or struct names encountered when
processing the header file contents.
<b>template_typedefs_ref</b>
(OUT) Series of SWIG %template directives. %template directives create a type
in the target language that corresponds to a C++ template instance.
### Description
process_class() processes class declarations with the following steps:
1. Extract the class_name from the class_string.
Add the <b>class_string</b> to the SWIG interface text.
1. Call <b>extract_bracketed()</b> to extract the class contents between '{' and '}'.
1. While there's class content text remaining to be processed,
repeatedly search for data members that match : <b>template_name '<'</b> <i>template-params</i> <b>'>' name ;</b>
For each match, create a SWIG %template directive to create an instantiation
of the specific templated type used by the data member. Add the
SWIG %template directive to the templated typedefs string that
Otherwise append whatever wasn't matched in process contents to
the SWIG interface text.
## process_typedef_struct
### Synopsis
Process a type definition of a struct or union to make it suitable as SWIG
interface code. Extract the struct (or union) name and bracketed contents from
the header file text (typedef_struct_string and contents_ref) . Record the
extracted names in the list referenced by class_names_ref, and then reconsistute
the type definition, via the new_contents_ref.
### Parameters
<b>typedef_struct_string</b>
(IN) This is a string of the form:<br>
1. <b>typedef struct</b> [ <i>optional-name</i> ] <b>"{"</b> OR<br>
1. <b>typedef union</b> [ <i>optional-name</i> ] <b>"{"</b><br>
<b>contents_ref</b>
(IN) This is a reference to the remainder of the header file (following the
above string) to be processed.
<b>new_contents_ref</b>
(OUT) The SWIG interface code generated so far.
<b>class_names_ref</b>
(OUT) reference to an array of class and/or struct names encountered when
processing the header file contents.
### Description
process_typedef_struct() processes a type definition of a struct or union
with the following steps:
1. Append the <b>typedef_struct_string</b> to the SWIG interface text
(via <b>new_contents_ref</b>).
1. Extract the optional-name from the typedef_struct_string.
1. Call extract_bracketed() to extract the struct contents from the header
text (via <b>contents_ref</b>), including the right bracket.
1. Extract the one or more typedef'ed names followed by a semi-colon, that
should still be in the header text.
1. Push the optional-name and the typedef'ed names into the class_names list
(via <b>class_names_ref</b>).
1. Append the bracketed struct contents and the one or more typedef'ed names
and the semi-colon that we just extracted to the SWIG interface text.

290
docs/developer_docs/DesDataRecording.md Normal file
View File

@ -0,0 +1,290 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Data Recording |
|------------------------------------------------------------------|
# Data Recording Design
The data recording design consists of 3 object types working together. At the bottom
level are the data recording groups. The data recording groups contain the user
specified variables to record, the frequency and the format to write the data in.
Each data recording group is associated with a buffering technique object. These objects
determine when the data recording groups write their data. Finally at the top is the
data recording dispatcher. The dispatcher holds the list of all the buffering techniques
and serves as a single calling point for the simulation executive to fire off all
data recording group activities.
## DataRecordGroup
The first objects are a set of DataRecordGroups. The DataRecordGroups are
responsible for copying simulation data into a temporary memory buffer
while the simulation is running. The DataRecordGroups also are responsible
for formatting the simulation data into a recording format.
A base DataRecordGroup object provides functions common to all recording
groups and provides an interface for the specific recording formats to follow.
The base DataRecordGroup derives from the SimObject class. This allows
new data recording groups to be added and removed from the scheduler as
other SimObjects.
The base class provides the following common functions:
1. Common initialization of a data record group.
1. Common copying of simulation data to memory buffers.
The base class provides function definitions for the specific formats to define:
1. Format specific initialization
1. Format specific write data from memory buffer to disk
1. Format specific shutdown of data recording files.
## DataRecordGroup Common Initialization
Initialization of a DataRecord group is handled in three phases.
The first is during the construction of the object. New data recording group
objects are typically created within the run input file read in during simulation
initialization.
@copydetails Trick::DataRecordGroup::DataRecordGroup(string)
Trick::DataRecordGroup::DataRecordGroup(string)
The second part of a DataRecordGroup initialization is supplied by the user.
The user typically sets these in the run input file read in during simulation
initialization. Typically the user will:
1. Add variables to be recorded.
1. Add variables to be added to the changes watch list.
1. Assign a different recording frequency.
1. Change the amount of memory data record is to allocate for memory buffers
The third part of initialization occurs at the end of simulation initialization.
During this phase all data recording groups finish their initialization. Format
specific initialization is also called during this phase.
@copydetails Trick::DataRecordGroup::init()
Trick::DataRecordGroup::init()
### DataRecordGroup Common Copy Simulation Data
This routine copies simulation data into a large buffer which will be written to
disk when commanded by the DataRecordDispatcher. Copying the simulation off into
a buffer allows the simulation to continue executing and data recording to use
a separate thread of execution to write the data to disk. The following algorithm
-# Return if data recording is not enabled.
-# If the recording frequency is to record on parameter changes
-# For each change parameter compare the current value against the previous frame value.
-# If the value has changed mark the change_detected flag true
-# Copy the current value to the previous frame value.
-# If the recording frequency is set to always or the change_detected flag is true
-# If the recording frequency is to record on parameter changes step
-# Copy to memory the previous value with the current time stamp
-# If the copy memory pointer is at the end of the buffer, set the pointer to the
beginning of the buffer
-# Copy to memory the current value with the current time stamp
### Data Recording Specific Formats Routines
#### ASCII Recording
@copydoc Trick::DRAscii
Trick::DRAscii
##### ASCII Initialization
@copydetails Trick::DRAscii::format_specific_init()
Trick::DRAscii::format_specific_init()
##### ASCII Write Data
@copydetails Trick::DRAscii::write_data()
Trick::DRAscii::write_data()
##### ASCII Shutdown
@copydetails Trick::DRAscii::shutdown()
Trick::DRAscii::shutdown()
#### Binary Recording
@copydoc Trick::DRBinary
Trick::DRBinary
##### Binary Initialization
@copydetails Trick::DRBinary::format_specific_init()
Trick::DRBinary::format_specific_init()
##### Binary Write Data
@copydetails Trick::DRBinary::write_data()
Trick::DRBinary::write_data()
##### Binary Shutdown
@copydetails Trick::DRBinary::shutdown()
Trick::DRBinary::shutdown()
#### HDF5 Recording
@copydoc Trick::DRHDF5
Trick::DRHDF5
##### HDF5 Initialization
@copydetails Trick::DRHDF5::format_specific_init()
Trick::DRHDF5::format_specific_init()
##### HDF5 Write Data
@copydetails Trick::DRHDF5::write_data()
Trick::DRHDF5::write_data()
##### HDF5 Shutdown
@copydetails Trick::DRHDF5::shutdown()
Trick::DRHDF5::shutdown()
## DataRecordBuffering
The second objects are a set of DataRecordBuffering objects. These objects
are responsible for writing the copied simulation data to disk. There are
three techniques to write data to disk.
A base DataRecordBuffering object provides a common interface for each of
the buffering techniques. The common object provides routines for. These
base routines may be overridden by the specific buffering techniques.
1. Processing simulation arguments
1. Initialization
1. Adding a data recording group
The base DataRecordBuffering provides hooks for:
1. Processing data at the end of each software frame
1. Processing data at shutdown.
An empty initialization routine is provided by the base object if the
specific buffering techniques do not require a specialized initialization routine.
The buffering classes do not write the data themselves, rather they call the data record
groups write_data routines to do the writing.
The 3 specific buffering techniques are
1. Write directly to disk immediately.
1. Use a thread to write data to disk asynchronously
1. Do not write data until simulation shutdown. Allow data copying to
overwrite data stored in the memory buffer.
### DataRecordBuffering Base Routines
#### DataRecordBuffering Process Simulation Arguments
@copydetails Trick::DataRecordBuffering::process_sim_args()
Trick::DataRecordBuffering::process_sim_args()
#### DataRecordBuffering Initialization
@copydetails Trick::DataRecordBuffering::init()
Trick::DataRecordBuffering::init()
#### DataRecordBuffering Adding a Recording Group
@copydetails Trick::DataRecordBuffering::add_group()
Trick::DataRecordBuffering::add_group()
### DataRecordDisk
This buffering technique calls all of the data record write_data routines in the
main simulation thread of execution.
#### DataRecordDisk End of Frame Processing
@copydetails Trick::DataRecordDisk::write_data()
Trick::DataRecordDisk::write_data()
#### DataRecordDisk Shutdown Processing
@copydetails Trick::DataRecordDisk::shutdown()
Trick::DataRecordDisk::shutdown()
### DataRecordRing
This buffering technique does not call the write_data routines during runtime. Only
at shutdown does the write_data routines get called. The last contents of the memory
buffer are then dumped to disk.
#### DataRecordRing End of Frame Processing
@copydetails Trick::DataRecordRing::write_data()
Trick::DataRecordRing::write_data()
#### DataRecordRing Shutdown Processing
@copydetails Trick::DataRecordRing::shutdown()
Trick::DataRecordRing::shutdown()
### DataRecordThreaded
This buffering technique uses a separate thread to write data to disk. The main
thread of execution attempts to signal the thread at the end of each frame. The
thread calls the group write_data routines when signaled.
#### DataRecordThreaded Writer
@copydetails Trick::DataRecordThreaded::drt_writer()
Trick::DataRecordThreaded::drt_writer()
#### DataRecordThreaded End of Frame Processing
@copydetails Trick::DataRecordThreaded::write_data()
Trick::DataRecordThreaded::write_data()
#### DataRecordThreaded Shutdown Processing
@copydetails Trick::DataRecordThreaded::shutdown()
Trick::DataRecordThreaded::shutdown()
## DataRecordDispatcher
The third object is the DataRecordDispatcher. The DataRecordDispatcher
calls each data recording buffering object's end of frame and shutdown
jobs. It serves as a single entry point for all data recording activities.
The dispatcher is intended to be the only data recording object called by
the %Trick scheduler. The dispatcher contains routines to
1. Process simulation arguments
1. Add data recording group
1. Initialization
1. End of Frame processing
1. Shutdown
### DataRecordDispatcher Process Simulation Arguments
@copydetails Trick::DataRecordDispatcher::process_sim_args()
Trick::DataRecordDispatcher::process_sim_args()
### DataRecordDispatcher Add Data Recording Group
When a data recording group is added to the DataRecordDispatcher the group is
tied to the requested buffering technique.
@copydetails Trick::DataRecordDispatcher::add_group()
Trick::DataRecordDispatcher::add_group()
### DataRecordDispatcher Initialization
@copydetails Trick::DataRecordDispatcher::init()
Trick::DataRecordDispatcher::init()
### DataRecordDispatcher End of Frame Processing
@copydetails Trick::DataRecordDispatcher::write_data()
Trick::DataRecordDispatcher::write_data()
### DataRecordDispatcher Shutdown
@copydetails Trick::DataRecordDispatcher::shutdown()
Trick::DataRecordDispatcher::shutdown()

110
docs/developer_docs/DesInterfaceCodeGenerator.md Normal file
View File

@ -0,0 +1,110 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → ICG |
|------------------------------------------------------------------|
# Interface Code Generator Design
The Interface Code Generator (ICG) creates class/struct layout information, called
attributes, used to convert string variable names to simulation memory
addresses. The attributes are used with several of Trick's services
including checkpointing, data recording, and the variable server.
ICG execution is divided into 2 parts.
1. Executing the Clang C++ frontend to parse all header files and extract
class/struct information.
1. Print class/struct information as attributes in input/output source code.
## Clang C++ Frontend
ICG uses the Clang frontend. Clang is a C/C++ frontend to the LLVM compiler.
Clang and LLVM provide libraries that allow developers to create their own tools
using the compiler. ICG calls the clang parser to process the header
files and converts the source text into a tree form called an abstract syntax tree
(AST). Clang completely handles all textual parsing. ICG provides classes to
provide input parameters to the compiler and to extract the information out of the
AST.
### Parser Setup
The frontend has some parameters that must be set in order for the parser to
execute.
#### Parsing Language
The S_source.hh file where parsing begins is written in C++. The frontend is set to
parse C++.
#### Target Architecture
The frontend requires a target architecture set. ICG does not pass the AST to the
next stage of compiler tools after the frontend, so it doesn't matter what the
target architecture is set to. Setting the target to the the default architecture
works.
### Preprocessor
The frontend includes a preprocessor. The preprocessor handles include and define
statements. ICG provides a pair of classes that interact with the clang preprocesor.
The HeaderSearchDirs class to provide the preprocessor with include search
directories and default defines. The CommentSaver class saves all comments parsed
by the preprocessor. These classes are used throughout the rest of ICG execution.
#### HeaderSearchDirs
The HeaderSearchDirs manages the header search directories used by the preprocessor
and provides an interface for querying the inclusion status of a directory. Since
this class is closely related to the preprocessor, it also handles define macros.
The HeaderSearchDirs class provides a list of include search directories to the
clang preprocessor. There are two types of include directories, system include
directories and user provided directories. Both types must be supplied to the
preprocessor.
The system include directories are retrieved by running the TRICK_CPPC compiler
in verbose preprocessor mode with no input code. When the compiler is run
in this mode, it prints the system include directories built into the compiler.
This class parses that output for the system include directories and feeds the
list into the preprocessor.
The user include directories are retrieved from the command line when executing
ICG. Typically ICG is called with the TRICK_CXXFLAGS environment variable
included as an argument. TRICK_CXXFLAGS may include -I<include_dir> strings.
These include directories are parsed from the command line and fed to the
preprocessor.
This class parses the TRICK_ICG_EXCLUDE environment variable and manages a list
of directories ICG should not write any output source code.
The HeaderSearchDirs class provides boolean responses to questions to whether a
path is in a user directory or is in an TRICK_ICG_EXCLUDE directory.
#### CommentSaver
The CommentSaver class saves comments found in user code and provides an iterface
for retrieving comments based on file name/line number. The class also provides
an interface to retrieving several fields from a Trick header comment including
the ICG and ICG_EXCLUDE_TYPES fields.
The clang preprocessor includes a comment handler interface. ICG uses
this interface to attach the CommentSaver class. Each comment handled by this
class is stored in a map indexed by file name and line number.
### Abstract Syntax Tree (AST) Traversal
Traversing the resulting AST is bulk of ICG's work. The clang compiler provides
an interface allowing ICG to attach handlers to process interesting events during
code parsing. In this case, ICG only attaches a handler when clang is finished
parsing the entire input file, called HandleTranslationUnit.
When HandleTranslationUnit is start parsing the AST searching for top level
declarations we want to process. From the top level recursively calls
tree parsers that specialize in extracting information about typedefs,
classes and structs, fields within each class/struct, and enumerations.
The following diagram shows the recursive class instantiation sequence as well
as the data returned to the caller.
## Writing Class/Struct Attributes

168
docs/developer_docs/DesMasterSlave.md Normal file
View File

@ -0,0 +1,168 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Master-Slave |
|------------------------------------------------------------------|
# Trick Master/Slave Synchronization Design
Master/Slave synchronization is an optional class when running simulations.
Master/Slave synchronization synchronizes a master simulation to one or more slave
simulations. While synchronized, the master controls the overall simulation
mode as well as the current simulation time for all simulations. The set of
simulations may change mode if one of the simulations in the set freezes or exits
unexpectedly.
Master/Slave syncrhonization uses the Trick::MSConnect class for communications.
The MSConnect class is an abstract base class that defines functions for
connecting the simulations together and passing the simulation mode and
simulation time between simulations. The Trick::MSSocket class derives from the
Trick::MSConnect class, and uses sockets as the transport medium for master/slave communications.
## Master
The master simulation starts slave simulations during initialization. At the
end of each software frame during Run mode and at the end of each freeze frame
during Freeze mode the master reads the mode of all connected slave simulations.
The master will determine the mode of the simulations for the next frame
according to the slave modes and the master mode. The new mode and the
current simulation time is sent to the slaves.
Master/Slave synchronization is divided into code based on the mode of the
simulation. The modes are Initialization, Run, Freeze, and Shutdown.
Master/Slave functions are designed to be called from the S_define file.
Listed below are the various modes of operation and the Master
responsibilities within each.
### Initialization -- Master
During Initialization slave connections are allocated and defined. Default
values to start each slave using ssh are set when a slave connection is
allocated. At the mimimum a slave must define a remote machine, the path
to the simulation executive, and a Trick::MSConnect class to use for
communications with the slave. Users may also define the remote shell to use,
a simulation executable name, a remote run directory, and additional user arguments
to pass to the slave. The allocation and definition of the slave
connections are typically done through Trick's input processor, but may also be
done in default_data or initialization jobs.
At the end of the scheduler initialization phase the master starts the slaves.
For each slave the master creates a slave startup command and executes the
slave startup command. This is how the slave startup command is created:
1. If the MSConnect class is not defined, return an error.
1. Begin the slave startup command with the slave remote shell.
1. Add the remote display environment variable to the startup command if
one was defined.
1. Add a cd command to the slave simulation path directory to the slave starup
command.
1. Add the S_main executable name to the slave startup command.
1. Add the Run Input File to the slave startup command.
1. Add additional user arguments to the slave startup command.
1. Execute the slave startup command.
The master sends its software frame to each slave.
### Run -- Master
During Run mode the master reads the simulation mode of each slave. The
master determines the new simulation mode and sends the new mode and the
current simulation time to the slaves.
1. For each slave
1. Read the simulation command of the slave.
1. If the master mode command is not ExitCmd take action if the slave
returns the following:
1. If the slave command was not readable, returned ErrorCmd, and:
1. Terminate the sim if sync_error_terminate == true.
1. Deactivate the slave if sync_error_terminate == false.
1. If the slave command is ExitCmd:
1. Terminate the sim if sync_error_terminate == true.
1. Deactivate the slave if sync_error_terminate == false.
1. If the slave command is FreezeCmd:
1. Freeze the master sim
After gathering the simulation command from all of the slaves, send the
command to all of the slaves.
1. For each slave
1. Write the simulation time to the slave.
1. Write the simulation mode command to the slave.
### Freeze -- Master
Upon entering freeze mode the master turns off sync_time_limit because while
in freeze, master/slave communications may not be regular and periodic.
At the end of each freeze frame call Trick::Master::end_of_frame() to perform the same
synchronization as in run mode.
When exiting freeze mode the master sets sync_time_limit back to the user
set value ready for running again.
### Shutdown -- Master
The master tells the slaves that it is shutting down.
1. For each slave
1. Write the simulation time to the slave.
1. Write the simulation mode command ExitCmd to the slave.
## Slave
At the end of each software frame during Run mode and at the end of each
freeze frame during Freeze mode slave simulations send the master the current
mode of the slave. The slaves wait for the new mode and simulation time from
the master.
### Initialization -- Slave
The Trick::Slave class is enabled by calling the MSConnect::process_sim_args()
routine. If the MSConnect::process_sim_args() finds command line arguments
specifying this simulation is a slave, then the Trick::Slave is enabled.
At the end of the scheduler initialization phase the slave connects to the
master.
1. Call Trick::MSConnect::connect() to connect the slave to the master.
1. Search for a RealtimeSync class. If one exists, turn the object off.
1. Search for a Master class. If one exists, turn the object off.
1. Read and set the slave's software frame from the master.
### Run -- Slave
In Run mode the slave writes its next executive command to the master. The
master will return the overall next simulation mode to the slave. The slave
will also read the current simulation time from the master.
1. Write the simulation command to the master.
1. Read the simulation time from the master.
1. if the simulation time read was not successful:
1. Terminate the sim if sync_error_terminate == true.
1. Disable master/slave communication and command the sim to freeze if sync_error_terminate == false.
1. If the simulation time read was successful set the slave simulation
time to the time of the master.
1. Read the simulation command from the master.
1. If the simulation command was not successful:
1. Terminate the sim if sync_error_terminate == true.
1. Disable master/slave communication and command the sim to freeze if sync_error_terminate == false.
1. If the simulation command read was successful set the slave simulation
command to the command of the master.
### Freeze -- Slave
Upon entering freeze mode the slave turns off sync_time_limit because while
in freeze, master/slave communications may not be regular and periodic.
At the end of each freeze frame call Trick::Slave::end_of_frame() to perform the same
synchronization as in run mode.
When exiting freeze mode the slave sets sync_time_limit back to the user
set value ready for running again.
### Shutdown -- Slave
If a slave is shutting down, inform the master that it is exiting.
[@anchor d_slave_shutdown_0 d_slave_shutdown_0]

85
docs/developer_docs/DesMonteCarlo.md Normal file
View File

@ -0,0 +1,85 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Monte Carlo |
|------------------------------------------------------------------|
# Monte Carlo and Optimization
Monte Carlo is the process of iteratively calling a simulation over a set of predetermined or auto-generated inputs.
Trick has designed its Monte Carlo capability to run distributed.
## Structure
In particular, Monte Carlo is designed after a "master/slave" model. The master is in charge of creating slaves and
tasking them to work. There may be any number of slaves distributed over a network. Master and slave communicate through
sockets. Theoretically, a master and slave need not have the same file system. Each slave is responsible for requesting
work, accomplishing work and reporting results. The work at hand is running a single simulation iteratively over an input space.
### The Master
The master is the command center of a Monte Carlo simulation. The master tasks slaves to run the simulation with a given set of inputs. The master will task slaves to run in parallel. The master is responsible for keeping the slaves as busy as possible. To keep things running smoothly, the master is designed to reassign work when a slave is either dead or running too slowly. the master is only in charge of tasking work. The master does not run the simulation itself. The master will continue issuing work to the slaves until it is satisfied all simulation runs are complete.
The master's life cycle consists of the following:
- Initialize
- While there are unresolved runs:
- Spawn any uninitialized slaves.
- Dispatch runs to ready slaves.
- Resolve run based on slave's exit status.
- Receive results from finished slave's child.
- Check for timeouts.
- Shutdown the slaves and terminate.
@see Trick::MonteCarlo
### Slaves
A slave consists of a parent and fork()ed children. A slave parent spawns a child using the fork() system call. A
slave child runs the simulation in its own address space. Only one child exists at a time in a slave. Per slave,
simulation execution is sequential.
A slave is responsible for requesting work from the master, running a Trick simulation with inputs given by the master,
dumping recorded data to disk and informing the master when it is finished running its task.
The slave's life cycle consists of the following:
- Initialize
- Connect to and inform the master of the port over which the slave is listening for dispatches.
- Until the connection to the master is lost or the master commands a shutdown:
- Wait for a new dispatch.
- Process the dispatch.
- Slave fork()s child.
- Child runs simulation with varied input.
- Write the run number processed to the master at child shutdown.
- Write the exit status to the master.
- Run the shutdown jobs and terminate.
@see Trick::MonteSlave
## Simulation Inputs
The goal of Monte Carlo is to run the simulation over a set of inputs. The inputs that the master passes to the slaves
are either generated by a statistical algorithm or they are hard-coded by the user in a data file. Inputs may also be
generated exclusively by user calculations.
## Monte Carlo Output
For each simulation run within a Monte Carlo suite of runs, a directory called "MONTE_<name>" is created. Slave output
is directed to this "MONTE_" directory. Trick recorded data is written in a set of "RUN_" directories within the parent
"MONTE_" directory. Along with recorded data, stdout, stderr, and send_hs files are written. A file that contains the
summary of all runs is written to the "MONTE_" directory.
### Data Processing
The trick_dp is desinged to understand "MONTE_" directories. When choosing to plot a "MONTE_" directory, trick_dp
will overlay all curves from each "RUN_" directory within the parent "MONTE_" directory. The plot widget has built
in features that allow the developer to distinguish what curve goes with what simulation run.
### Optimization
Optimization is made possible by creating a framework whereby the developer can change simulation inputs based on
simulation results. Trick offers a set of job classes that allow the developer to enter the Monte Carlo loop and
thereby enter the decision making on-the-fly. No canned optimization is available.
This special set of job classes work in concert together in master and slaves. Trick schedules jobs within the master
at critical points so that they may create inputs to send to the slave as well as receive results from the slave.
Slave jobs are scheduled to receive simulation inputs from the master as well as send simulation results back to the
master.
The jobs are specified in the S_define. The jobs are created by the developer.

50
docs/developer_docs/DesRealTimeClock.md Normal file
View File

@ -0,0 +1,50 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Real-Time Clock |
|------------------------------------------------------------------|
# Real-Time Clock Design
## Base Real-Time Clock
The base Clock object provides an interface for the following operations. The
base Clock does not implement the following operations.
1. Initializing the clock hardware
1. Resetting the clock (setting the reference time)
1. Getting the current real time
1. Spinning on the clock<br>
Spinning on the clock means to continually check the clock waiting for real-time to
catch up to the desired simulation elapsed time.
1. Stopping the clock
## Gettimeofday Clock
The GetTimeOfDayClock is a clock object that derives from the Base Real-Time Clock. The
GetTimeOfDayClock uses system clock as the real-time clock.
The GetTimeOfDayClock implements the interface laid out by the base clock class
### Initializing the Clock
@copydetails Trick::GetTimeOfDayClock::clock_init()
Trick::GetTimeOfDayClock::clock_init()
### Resetting the Clock (setting the reference time)
@copydetails Trick::GetTimeOfDayClock::clock_reset()
Trick::GetTimeOfDayClock::clock_reset()
### Getting the Current Real Time
@copydetails Trick::GetTimeOfDayClock::clock_time()
Trick::GetTimeOfDayClock::clock_time()
### Spinning on the Clock
@copydetails Trick::GetTimeOfDayClock::clock_spin()
Trick::GetTimeOfDayClock::clock_spin()
### Stopping the Clock
@copydetails Trick::GetTimeOfDayClock::clock_stop()
Trick::GetTimeOfDayClock::clock_stop()

98
docs/developer_docs/DesRealTimeSynchronization.md Normal file
View File

@ -0,0 +1,98 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Real-Time Synchronization |
|------------------------------------------------------------------|
# Real-Time Synchronization Design
A simulation is "real-time" when it can consistently and repetitively execute
its scheduled math models to completion within some predetermined interval
time frame for an indefinite period of time. This predetermined interval
time frame is referred to as the real-time frame. In this design the real-time
frame is set equal to the simulation software frame. Real-time is achieved
by periodically pausing simulation execution at the end of each real-time
frame. If simulation execution has not finished by the end of each real-time
frame then this is an overrun. The real-time synchronization capability has
several user settable options when a simulation overruns. If simulation
execution finishes before the real-time frame expires, this is an underrun. If
the simulation is underruning, the real-time synchronization capability allows
the simulation to "sleep" while waiting for the real-time frame to expire.
Real-time synchronization defaults off when initializing the simulation.
To enable real-time synchronization users or models may call enable() at any
time during the simulation. To exit real-time synchronization users or
models may call disable() at any time during the simulation. The actual
entering or exiting real-time happens at the end of the frame.
Listed below are the various modes of operation in a simulation and the
responsibilities of the real-time synchronization class in each.
## Pre-Initialization
During pre-initialization the user may specify the behavior of the real-time
synchronization class during overruns and underruns.
1. Set real-time synchronization enabled or disabled.
1. Set real-time sleep timer enabled or disabled.
1. Set the maximum number of consecutive overruns allowed. (default 100,000,000)
1. Set the maximum time allowed for single overruns (default 1e37 seconds)
1. Set action taken when maximum overruns achieved to freeze/terminate or
terminate immediately (default terminate immediately)
## Initialization and Restart
The real-time initialization and restart jobs are designed to be the last
jobs run before exiting initialization or restart mode.
Initialize the underlying hardware.
@copydetails Trick::RealtimeSync::initialize(long long , long long )
Trick::RealtimeSync::initialize(long long , long long )
## Start Real-Time
@copydetails Trick::RealtimeSync::initialize(long long , long long )
Trick::RealtimeSync::initialize(long long , long long )
## End of Frame
At the end of the real-time frame (also the software frame) the
real-time synchronization class will determine if the simulation is
underruning or overrunning and take actions to keep simulation time
and real-time in sync.
@copydetails Trick::RealtimeSync::rt_monitor( long long )
Trick::RealtimeSync::rt_monitor( long long )
## Freeze Mode
### Freeze Initialization
During freeze initialization the real-time synchronization class resets the real-time
frame to the freeze frame and restarts the real-time clock and timer.
@copydetails Trick::RealtimeSync::freeze_init( long long )
Trick::RealtimeSync::freeze_init( long long )
### Freeze Loop
During freeze loop processing the real-time synchronization pauses the freeze loop.
The pause is released when real-time equals the freeze loop period. No checks are
done for overrunning or underruning a freeze period.
@copydetails Trick::RealtimeSync::freeze_pause( long long )
Trick::RealtimeSync::freeze_pause( long long )
### UnFreeze
During unfreeze the real-time synchronization class resets the real-time
frame to the software frame and restarts the real-time clock and timer.
@copydetails Trick::RealtimeSync::unfreeze( long long , long long )
Trick::RealtimeSync::unfreeze( long long , long long )
## Shutdown Mode
During shutdown, the real-time synchronization class stops the clock and sleep
timer hardware.
@copydetails Trick::RealtimeSync::shutdown()
Trick::RealtimeSync::shutdown()

87
docs/developer_docs/DesScheduledJobQueue.md Normal file
View File

@ -0,0 +1,87 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Scheduled Job Queue |
|------------------------------------------------------------------|
# ScheduledJobQueue Design
The Scheduler typically calls upon the ScheduledJobQueue to add
a job, find the next job that matches the current sim time, get the next
job call time, and instrument the job queue.
Listed below are the design for each use case of the ScheduledJobQueue.
## Adding a job
Adding a job to the queue increases the queue size by 1. Jobs are ordered by
job class, then phase, then sim object id, and finally job_id. The job_id increases
with the ordering of jobs in a sim object.
@copydetails Trick::ScheduledJobQueue::push()
Trick::ScheduledJobQueue::push()
## Adding a job ignoring sim object order.
Somethimes To add a job to the queue, Setting the sim object id to a large nnumber effectively
removes the sim object id in the sorting.
@copydetails Trick::ScheduledJobQueue::push_ignore_sim_object()
Trick::ScheduledJobQueue::push_ignore_sim_object()
## Find Next Job with Current Sim Time
Advances curr_index to next job in list who's next call time matches the incoming time
argument. If the job is not a system_job as determined by the scheduler, the next job
call time is calculated before returning. (system_jobs set their own next job call time).
Also track the next_job_time which will be used to advance simulation time
at the end of frame.
@copydetails Trick::ScheduledJobQueue::find_next_job()
Trick::ScheduledJobQueue::find_next_job()
## Get the Next Job
A simple get the next job in the list.
@copydetails Trick::ScheduledJobQueue::get_next_job()
Trick::ScheduledJobQueue::get_next_job()
## Get the next job call time
As the find_next_job routine traverses the queue of jobs searching for jobs that match the
current simulation time step, it also saves the time of the next lowest job
call time. This is the next time step that the scheduler will use when
advancing simulation time.
@copydetails Trick::ScheduledJobQueue::get_next_job_call_time()
Trick::ScheduledJobQueue::get_next_job_call_time()
## Test Next Job Call Time
Some job types reschedule themselves and the next job call time are not tested by the find next job routine.
The test next job call time allows these job to check their next call time against the overall next
job call time. If the current job next job call time is lower than the overall job call time, the
overall next job call time is set to the current job's next call time.
@copydetails Trick::ScheduledJobQueue::test_next_job_call_time()
Trick::ScheduledJobQueue::test_next_job_call_time()
## Instrument a job before all jobs in the queue
To add an instrumentation job before a job or all jobs in the queue. Requirement [@ref r_exec_instrument_0]
@copydetails Trick::ScheduledJobQueue::instrument_before(JobData * , string , InstrumentationEvent * )
Trick::ScheduledJobQueue::instrument_before(JobData * , string , InstrumentationEvent * )
## Instrument a job after all jobs in the queue
To add an instrumentation job after a job or all jobs in the queue. Requirement [@ref r_exec_instrument_2]
@copydetails Trick::ScheduledJobQueue::instrument_after(JobData * , string , InstrumentationEvent * )
Trick::ScheduledJobQueue::instrument_after(JobData * , string , InstrumentationEvent * )
## Remove an instrumentation job
To remove an instrumentation job from the queue. Requirement [@ref r_exec_instrument_4]
@copydetails Trick::ScheduledJobQueue::instrument_remove(string , InstrumentationEvent * )
Trick::ScheduledJobQueue::instrument_remove(string , InstrumentationEvent * )

91
docs/developer_docs/DesSleepTimer.md Normal file
View File

@ -0,0 +1,91 @@
| [Home](/trick) → [Developer Docs](Developer-Docs-Home) → Sleep Timer |
|------------------------------------------------------------------|
# Sleep Timer Design
After all jobs in an execution frame have run, the default behavior (the timer is disabled)
is that the simulation will spin on the clock until real-time has caught up to simulation time.
Enabling the sleep timer gives control to the timer's pause() function, in which it may
release the processor and regain control when the timer elapses.
## Base Sleep Timer
The base Timer object provides a common interface for all Timers for the
following functions.
1. Enabling the Timer
1. Disabling the Timer
1. Getting the enabled/disabled status
The base Timer object provides an interface for the following operations. The
base Timer does not implement the following operations.
1. Initializing the Timer hardware
1. Setting the frame time for timer
1. Starting the timer
1. Resetting the timer
1. Pausing for the timer to expire
1. Stopping the timer
1. Shutting down the Timer hardware
## ITimer
The ITimer is a Timer object that derives from the Base Sleep Timer. The
ITimer uses system itimer functionality to allow the simulation to release the
cpu while waiting for the timer to expire. When the itimer expires it uses
the SIGALRM signal to set a semaphore. When the semaphore is set the simulation
wakes up.
The itimer does not expire exactly at the desired period. It actually exipres after
the desired period by up to 1ms. To compensate for the delay we subtract 2ms from
the desired period. The RealtimeSync class "spins" on the clock the remaining
1-2ms the timer does not sleep for.
The ITimer implements the interface laid out by the base Timer class
### Initialization
@copydetails Trick::ITimer::init()
Trick::ITimer::init()
### Setting the Frame Time
@copydetails Trick::ITimer::set_frame_times()
Trick::ITimer::set_frame_times()
### Starting the timer.
@copydetails Trick::ITimer::start()
Trick::ITimer::start()
### Restarting the timer.
@copydetails Trick::ITimer::reset()
Trick::ITimer::reset()
### Pausing for the Timer to Expire
Pausing for the timer has 2 parts. The first is the signal handler that posts
a semaphore in response to a SIGALRM signal. The signal handler is assigned
to SIGALRM during the initialization routine. The second is the pause routine
that waits for the semaphore to be posted.
#### Signal Handler
@copydetails Trick::ITimer::it_handler()
Trick::ITimer::it_handler()
### Pause for Semaphore
@copydetails Trick::ITimer::pause()
Trick::ITimer::pause()
### Stopping the timer.
@copydetails Trick::ITimer::stop()
Trick::ITimer::stop()
### Shutting Down the Timer
@copydetails Trick::ITimer::shutdown()
Trick::ITimer::shutdown()

11
docs/developer_docs/Developer-Docs-Home.md
View File

@ -12,4 +12,15 @@ Link documentation for Trick internals, processes, and plans here.
- [Python Environment](Python-Environment-Issues)
- [CI](CI)
Old design documentation - may or may not be up to date
- [Convert Swig](DesConvertSwig)
- [ICG](DesInterfaceCodeGenerator)
- [Realtime Clock](DesRealTimeClock)
- [Realtime Synchronization](DesRealTimeSynchronization)
- [Scheduled Job Queue](DesScheduledJobQueue)
- [Sleep Timer](DesSleepTimer)
- [Data Recording](DesDataRecording)
- [Monte Carlo](DesMonteCarlo)
- [Master Slave](DesMasterSlave)