trick/trick_source/sim_services/Executive/docs/design.dox_in

/**

@page LEVEL2 Scheduler Design 

    The scheduler design is divided based on the mode of the simulation.  The modes are 
    Pre-Initialization, Initialization, Run, Freeze, and Shutdown.  

    The scheduler routines are called directly by a main program in a simulation.
    A typical main program calls the following scheduler routines.

    -# Call add_sim_object() for each sim_object to be added to the scheduler.  
    -# Call init() to initialize the simulation. 
    -# Call loop() to run the cyclic jobs of the simulation.  loop() contains the
       code for the Run and Freeze modes of operation. 
    -# Call shutdown() to shutdown the simulation. 

    @dot
    digraph finite_state_machine {
         rankdir=LR ;
         size="12,5"
         node [shape = doublecircle]; Initialization Shutdown;
         node [shape = circle] ;
         Initialization -> Run [ label = "no errors" ] ;
         Initialization -> Shutdown [ label = "error condition" ];
         Freeze -> Run [ label = "exec_command = RunCmd" ] ;
         Run -> Freeze [ label = "exec_command = FreezeCmd" ] ;
         Freeze -> Shutdown [ label = "exec_command = ExitCmd\nor error condition" ] ;
         Run -> Shutdown [ label = "exec_command = ExitCmd\nor error condition" ] ;
    }
    @enddot

    <center>Figure 1: Simulation Mode Finite State Machine</center>

    Listed below are the various modes of operation and scheduler responsibilities within each.

@section LEVEL3 Pre-Initialization 

    During Pre-Initialization the simulation tells the Executive scheduler which SimObjects
    are to be included in this simulation run.  The auto-generated initialization routine 
    in S_source.cpp calls add_sim_object() for each sim_object instantiated in the S_define file. 

@section LEVEL4 Adding Simulation Objects 

    In Pre-Initialization the auto-generated initialization routine calls add_sim_object() 
    for each sim_object instantiated in the S_define file.  Each add_sim_object() will add
    its jobs to the scheduing queues in order of SimObject instantiation.

    @copydetails Trick::Executive::add_sim_object(Trick::SimObject * in_object , const char * in_name )
    Trick::Executive::add_sim_object(Trick::SimObject * in_object , const char * in_name )

@section LEVEL4 Adding List of Jobs to Scheduling Queue 

    Each add_jobs_to_queue() will call add_job_to_queue() to add an individual job to the scheduler queue.

    @copydetails Trick::Executive::add_jobs_to_queue(Trick::SimObject *, bool)
    Trick::Executive::add_jobs_to_queue(Trick::SimObject *, bool)

@section LEVEL4 Adding Individual Job to Scheduling Queue 

    Adds an individual job to the scheduler queues.

    @copydetails Trick::Executive::add_job_to_queue(Trick::JobData *)
    Trick::Executive::add_job_to_queue(Trick::JobData *)

@section LEVEL3 Initialization Mode

    During initialization mode the scheduler readies the simulation for running.  This includes
    calling SimObject initialization jobs as well as initializing scheduler timers.  The Executive
    provides the routine to call all default data and initialization class jobs as well as provides
    jobs to initialize specific parts of the Executive itself.  The initialization jobs include

    -# Executive Initialization
    -# Initialize default signal handlers 
    -# Process Simulation arguments
    -# Post Initialization signal handler assignment
    -# Write the S_job_execution File 
    -# Write the S_run_summary file which describes the simulation build environment 
    -# Check if all jobs in all SimObjects are handled by a scheduler.
    -# Create scheduler threads 

@section LEVEL4 Executive Initialization 

    This routine is typically called by the simulation main program.  This routine initializes the
    scheduler itself as well as calls the initialization routines for all the math models.

    @copydetails Trick::Executive::init()
    Trick::Executive::init()

@section LEVEL4 Initialize Default Signal Handlers

    @copydetails Trick::Executive::init_signal_handlers()
    Trick::Executive::init_signal_handlers()

@section LEVEL4 Process Simulation Arguments

    @copydetails Trick::Executive::process_sim_args()
    Trick::Executive::process_sim_args()

@section LEVEL4 Post Initialization Signal Handler Assignment

    @copydetails Trick::Executive::post_init_signal_handlers()
    Trick::Executive::post_init_signal_handlers()

@section LEVEL4 Write the S_job_execution File

    The S_job_execution file contains all of the jobs currently scheduled in the Executive.  The file
    lists the jobs in order in the queues and shows the job name, id, class, phase, cycle time, start time, and
    end time.

    @copydetails Trick::Executive::write_s_job_execution()
    Trick::Executive::write_s_job_execution()

@section LEVEL4 Write the S_run_summary File

    @copydetails Trick::Executive::write_s_run_summary()
    Trick::Executive::write_s_run_summary()

@section LEVEL3 Run Mode

    After initialization the simulation enters the run/freeze loop.  A single loop to handle jobs for
    both modes are integrated to provide easy entry and exit points from one loop to the other.

    During Run mode the executive/scheduler advances simulation time.
    The scheduler calls the runtime cyclic jobs when simulation time equals the next call
    time of the job.  The scheduler has a software frame time.  Users may specify override 
    the default software frame of 1 second.  When the simulation time equals the next software 
    frame time, the scheduler calls end of software frame jobs.

    Multiple threads may have been spawned to execute the Run mode jobs.  Threads may be specified 
    as synchronous or asynchronous.  Synchronous threads are synchronized to the main scheduler
    thread at the start of each time step.  Asynchronous threads are further divided into two types
    Asynchronous and Asynchronous Must Finish (AMF) threads. Asynchronous threads are never
    synchronized to the main scheduler thread once started.  If an asynchronous thread completes
    execution of all of its jobs, the jobs are started again at the next time step.  AMF threads
    are only synchronized to the main scheduler thread at a user specified thread specific
    frequency.

    At the beginning of Run mode the number of threads of execution is checked.
    If there is only one thread of execution, a simplified run loop where
    multi-thread dependency checks are removed is executed.  Otherwise a
    multi-threaded aware loop is called.

    There are 5 routines suppoprting Run mode.

    -# Loop Entry Point
    -# Single Thread Loop
    -# Multi Thread Main Thread Loop
    -# Multi Thread Child Thread Loop
    -# Advance Simulation Time

@section LEVEL4 Loop Entry Point

    The loop entry point determines if the simulation is multi threaded or not.  It calls the corresponding
    loop to do the actual work.

    @copydetails Trick::Executive::loop()
    Trick::Executive::loop()

@section LEVEL4 Single Thread Loop

    Called by loop().  Calls jobs in the main thread.

    @copydetails Trick::Executive::loop_single_thread()
    Trick::Executive::loop_single_thread()

@section LEVEL4 Multi Thread Main Thread Loop

    Called by loop().  Coordinates scheduled job calls for the main thread and all child threads.

    @copydetails Trick::Executive::loop_multi_thread()
    Trick::Executive::loop_multi_thread()

@section LEVEL4 Multi Thread Child Thread Loop

    Child threads are signalled by the main thread to execute their jobs.

    @copydetails Trick::Threads::child()
    Trick::Threads::child()

@section LEVEL4 Advance Sim Time

    The Executive advances simulation time during each time step with a system_advance_sim_time class
    job.  This job advances time to the next lowest job call time.

    @copydetails Trick::Executive::advance_sim_time()
    Trick::Executive::advance_sim_time()

@section LEVEL3 Freeze Mode

    During Freeze mode, simulation elapsed time does not advance.  The scheduler enters a 
    loop that calls all freeze jobs at a monotonic frequency.  Jobs may be called to start 
    freeze mode as well as when exiting freeze mode.  All freeze mode jobs are executed within
    the main thread.

    @copydetails Trick::Executive::freeze_loop()
    Trick::Executive::freeze_loop()

@section LEVEL3 Shutdown Mode

    During Shutdown mode the scheduler waits for asynchronous jobs to finish if so directed.  
    The scheduler then calls shutdown jobs. The scheduler prints the reason for simulation
    termination.

    @copydetails Trick::Executive::shutdown()
    Trick::Executive::shutdown()

@section LEVEL3 Signal Handling

    Active through all modes, the scheduler assigns signal handlers to attempt a graceful 
    shutdown in case of a signal being thrown.

    @copydetails Trick::Executive::signal_handler()
    Trick::Executive::signal_handler()

*/