Repurposed -OO for all sim run outputs including S_sie.resource being saved in the specified directory. (#1714)

* Repurposed -OO so all sim run outputs including S_sie.resource are saved to the specified folder. The files are placed in the sub-dir either RUN_xxx or DP_Product of the specified folder.

* Updated for repurposed -OO.

* Added quoting code markdown for such as in order to show <name> as is.

* Added quoting code for missing items.

* Updated to raise an error when the --read-only-sim flag is used without the -O or -OO. Also made updates to the related document accordingly.

* Fixed the error message to be more clear.

docs/documentation/running_a_simulation/Running-a-Simulation.md

@@ -16,14 +16,27 @@ S_main_${TRICK_HOST_CPU}.exe [trick_version] [sie]
        [-u <user_defined_arguments>]
 ```
 
-- The first argument in the command line must be the simulation input file name. The input file name can be in the form of a full path name but MUST have a RUN_<name> directory immediately above the input file name. By default, all the simulation output is directed to this RUN_<name> directory. The standard <input_file_name> is input.py; however, a simulation could be started from a checkpoint file by substituting chkpnt_<time> in for <input_file_name> for non-Master/Slave and non-Import/Export simulations. For Master/Slave and Import/Export simulations you must have the simulation running, and the simulation must be in a freeze state before reloading a checkpoint.
+- The first argument in the command line must be the simulation input file name. The input file name can be in the form of a full path name but MUST have a `RUN_<name>` directory immediately above the input file name. By default, all the simulation output is directed to this `RUN_<name>` directory. The standard `<input_file_name>` is input.py; however, a simulation could be started from a checkpoint file by substituting `chkpnt_<time>` in for `<input_file_name>` for non-Master/Slave and non-Import/Export simulations. For Master/Slave and Import/Export simulations you must have the simulation running, and the simulation must be in a freeze state before reloading a checkpoint.
-- The trick_version option will tell what version of Trick built the S_main executable.
+- The `trick_version` option will tell what version of Trick built the S_main executable.
-- The sie option will generate the smart input editor (SIE) resource file (CP will by default invoke the S_main executable with the sie option to generate this file).
+- The `sie` option will generate the smart input editor (SIE) resource file.
-- The '-d' argument is optional and, if specified, starts the simulation in an input file verification mode. In this mode the entire input file is read, echoed to standard out, and then the simulation exits without calling any jobs listed in the S_define file. This mode helps debug input file syntax errors.
+- The `-d` argument is optional and, if specified, starts the simulation in an input file verification mode. In this mode the entire input file is read, echoed to standard out, and then the simulation exits without calling any jobs listed in the S_define file. This mode helps debug input file syntax errors.
-- The '-O <output_file_path>' option allows the user to specify the directory to which simulation data log files will be written. If this option is omitted, the RUN_<name> directory is used.
+- The `-O <output_file_path>` option allows the user to specify the directory to which simulation data log files will be written. If this option is omitted, the `RUN_<name>` directory is used. 
-- The '-OO <output_file_path>' option allows the user to specify the directory to which ALL simulation output files will be written. If this option is omitted, the RUN_<name> directory is used.
+  - Data Products specification DP_xxx.xml files if generated such as for frame logging are saved in `DP_Product` directory.
-- The '--read-only-sim' flag can be used to redirect all files written at simulation runtime into the output directory.
+- The `-OO <output_file_path>` option allows the user to specify the directory to which ALL simulation output files will be written. If this option is omitted, the `RUN_<name>` directory is used.
-- The '-u' option specifies that all remaining arguments are meant to be used by user supplied jobs. All arguments after the -u can be accessed internal to the simulation jobs by calling the get_cmnd_args() function of the executive as illustrated below. In a master/slave simulation, the master's -u args will be passed to the slave.
+  - Two subdirectories are automatically created in the designated `<output_file_path>`:
+    - `DP_Product`
+      - Data Products sepcification DP_xxx.xml files are saved in this folder.
+    - `RUN_<name>`
+      - All simulation output files, excluding DP_xxx.xml files, are saved in this folder. Additionally, the S_sie.resource file is copied to this directory.
+- The `--read-only-sim` flag can be used to redirect all files written at simulation runtime into the output directory.
+  - The `--read-only-sim` flag requires either the `-O` or `-OO` option to be used.
+  - When the `-O <output_file_path>` option is used and `trick.trick.sie_append_runtime_objs()` is called from the input file, the S_sie.resource file with appended runtime objects is saved in `<output_file_path>`.
+  - When the `-OO <output_file_path>` option is used and `trick.trick.sie_append_runtime_objs()` is called from the input file, the S_sie.resource file with runtime objects appended is saved in `<output_file_path>/RUN_<name>`.
+- If `trick.trick.sie_append_runtime_objs()` is called from the input file, the S_sie.resource file is to be appended with runtime objects.
+  - When neither the `-O` nor `-OO` option is used, the S_sie.resource file with runtime objects appended is saved in the current directory from which the simulation is executed. 
+  - When the `-O <output_file_path>` option is used without the `--read-only-sim` flag, the S_sie.resource file with runtime objects appended is saved in the current directory from which the simulation is executed.
+  - When the `-OO <output_file_path>` option is used without the `--read-only-sim` flag, the S_sie.resource file with runtime objects appended is saved in `<output_file_path>/RUN_<name>`.
+- The `-u` option specifies that all remaining arguments are meant to be used by user supplied jobs. All arguments after the -u can be accessed internal to the simulation jobs by calling the get_cmnd_args() function of the executive as illustrated below. In a master/slave simulation, the master's `-u` args will be passed to the slave.
 
 The following code example shows how a function can access the command line arguments during execution.
 

test_sims.yml

@@ -304,7 +304,7 @@ SIM_test_output_dir:
     RUN_test/input.py -OO sim_output --read-only-sim:
       returns: 0
       analyze: 'python3 test/SIM_test_output_dir/ref_files/check_file_endings.py test/SIM_test_output_dir/ref_files/ref_compiletime_S_sie.resource test/SIM_test_output_dir/S_sie.resource'
-      analyze: 'python3 test/SIM_test_output_dir/ref_files/check_file_endings.py test/SIM_test_output_dir/ref_files/ref_runtime_S_sie.resource test/SIM_test_output_dir/sim_output/S_sie.resource'
+      analyze: 'python3 test/SIM_test_output_dir/ref_files/check_file_endings.py test/SIM_test_output_dir/ref_files/ref_runtime_S_sie.resource test/SIM_test_output_dir/sim_output/RUN_test/S_sie.resource'
 
 
 # The variable server client and SIM_amoeba sometimes fail to connect and need to be retried

trick_source/sim_services/CommandLineArguments/CommandLineArguments.cpp

@@ -253,8 +253,9 @@ int Trick::CommandLineArguments::process_sim_args(int nargs , char **args) {
                 }
                 /* Output data directory */
                 output_dir = user_output_dir = argv[++ii];
-
+                if (!strncmp("-OO", argv[ii-1], (size_t) 3)) {
-
+                    output_dir = output_dir + "/" + run_dir;
+                } 
             }
         }
 

trick_source/sim_services/FrameLog/FrameLog.cpp

@@ -713,6 +713,11 @@ int Trick::FrameLog::create_DP_files() {
 int Trick::FrameLog::create_DP_Product_dir() {
     int ret=0;
     DP_dir = "DP_Product";
+    if (std::string(command_line_args_get_user_output_dir()) != std::string(command_line_args_get_output_dir())) {
+        if (!std::string(command_line_args_get_user_output_dir()).empty()) {
+            DP_dir = std::string(command_line_args_get_user_output_dir()) + "/DP_Product";
+        }
+    } 
     ret = mkdir(DP_dir.c_str(), 0777);
     if (ret == -1) {
         if (errno == EEXIST) {

trick_source/sim_services/Sie/Sie.cpp

@@ -27,6 +27,26 @@ Trick::Sie::Sie() {
     the_sie = this ;
 }
 
+void copy_file (const std::string& original_filename, const std::string& copy_filename) {
+    std::ifstream original;
+    std::ofstream copy;
+
+    original.open(original_filename.c_str(), std::ios::binary);
+    copy.open(copy_filename.c_str(), std::ios::binary);
+
+    copy << original.rdbuf();
+
+    original.close();
+    copy.close();
+}
+
+// Helper function for copying S_sie.resource from default dir to output dir
+void copy_sie_resource() {
+    std::string original_sie_filename = std::string(command_line_args_get_default_dir()) + "/" + "S_sie.resource";
+    std::string copy_sie_filename = std::string(command_line_args_get_output_dir()) + "/" + "S_sie.resource";
+    copy_file(original_sie_filename, copy_sie_filename);
+}
+
 int Trick::Sie::process_sim_args() {
 
     int argc ;
@@ -46,13 +66,38 @@ int Trick::Sie::process_sim_args() {
             exit(0) ;
         }
 
+        bool read_only = false;
+        bool oo_dir = false;
+        bool o_dir = false;
+
         // Otherwise, go through the rest of the sim args and look for --read-only-sim
         for (int i = 1; i < argc; i++) {
-            if (strcmp("--read-only-sim", argv[i]) == 0) {
+            if ((strcmp("--read-only-sim", argv[i]) == 0))  {
                 // Set this flag to move runtime generation of sie into the output directory 
                 move_runtime_generation = true;
+                read_only = true;
+                break;
             }
         }
+
+        // Set oo_dir and o_dir flags accordingly
+        for (int i = 1; i < argc; i++) {
+            if (strncmp("-OO", argv[i], (size_t) 3) == 0) {
+                oo_dir = true;
+            } else if (strncmp("-O", argv[i], (size_t) 2) == 0) {
+                o_dir = true;
+            }
+        }
+
+        // If --read-only-sim is provided without either -OO or -O, exit with error message
+        // For -OO, save S_sie.resource to the output directory with runtime jobs data
+        if (read_only && !oo_dir && !o_dir) {
+            std::cerr << "\nERROR: Missing -O or -OO argument with --read-only-sim flag" << std::endl;
+            exit(1);
+        } else if (oo_dir) {
+            copy_sie_resource();
+            move_runtime_generation = true;
+        }
     }
 
     return(0) ;
@@ -158,26 +203,11 @@ void Trick::Sie::sie_print_xml() {
     sie_out.close() ;
 }
 
-void copy_file (const std::string& original_filename, const std::string& copy_filename) {
-    std::ifstream original;
-    std::ofstream copy;
-
-    original.open(original_filename.c_str(), std::ios::binary);
-    copy.open(copy_filename.c_str(), std::ios::binary);
-
-    copy << original.rdbuf();
-
-    original.close();
-    copy.close();
-}
-
 void Trick::Sie::sie_append_runtime_objs() {
     std::fstream sie_out ;
 
     if (move_runtime_generation) {
-        std::string original_sie_filename = std::string(command_line_args_get_default_dir()) + "/" + "S_sie.resource" ;
+        copy_sie_resource();
-        std::string copy_sie_filename = std::string(command_line_args_get_output_dir()) + "/" + "S_sie.resource" ;
-        copy_file(original_sie_filename, copy_sie_filename);
     }
 
     std::string sie_filename = get_runtime_sie_dir() + "/" + "S_sie.resource" ;
@@ -223,9 +253,9 @@ void Trick::Sie::sie_append_runtime_objs() {
 
 std::string Trick::Sie::get_runtime_sie_dir() {
     if (move_runtime_generation) {
-        return std::string(command_line_args_get_output_dir()) ;
+        return std::string(command_line_args_get_output_dir());
     } else {
-        return std::string(command_line_args_get_default_dir()) ;
+        return std::string(command_line_args_get_default_dir());
     }
 }