pipeline changes, preparing to dockerize etc

Added DSR-InstrumentedCTO-RedmineGather script and test suite.
- Script gathers data from Redmine API and processes it. - Includes robust error handling and detailed logging. - Test suite verifies data gathering functionality.
2025-07-11 22:01:17 -05:00 · 2024-12-20 15:36:38 -06:00 · 2024-12-18 16:59:20 -06:00 · 2024-12-18 16:46:33 -06:00 · 2024-12-18 16:10:39 -06:00 · 2024-12-16 20:09:21 -06:00
16 changed files with 421 additions and 433 deletions
--- a/DSRVariables.env
+++ b/DSRVariables.env
@@ -0,0 +1,5 @@
+export JOPLIN_HOST="localhost"
+export JOPLIN_PORT="41185"
+
+export JOPLIN_SOURCE_NOTE_TITLE="DSR-Template"
+export JOPLIN_TARGET_NOTEBOOK="Today"
--- a/FullyInstrumentedCTO.md
+++ b/FullyInstrumentedCTO.md
@@ -1,19 +1,19 @@
-# Data Gathering for the @ReachableCEO daily stakeholder report
-
-## Introduction
-
-I'm trying to life a fully instrumented CTO/founder life and #buildInPublic. And unlike most of the founders doing so, I'm actually publishing daily reports and live streaming my workstation all dya.
-
-I want to automate the instrumentation/reporting process as much as possible to get the highest fidelity information for stakeholders consumption.
-
-## Data sources
-
- Apple Health (exporting via AutoHealthExport)
- Workout (evaluating a cou
- Nutrition
- Daily habit tracking
- ActivityWatch
- WakaAPI
-
-
+# Data Gathering for the @ReachableCEO daily stakeholder report
+
+## Introduction
+
+I'm trying to life a fully instrumented CTO/founder life and #buildInPublic. And unlike most of the founders doing so, I'm actually publishing daily reports and live streaming my workstation all dya.
+
+I want to automate the instrumentation/reporting process as much as possible to get the highest fidelity information for stakeholders consumption.
+
+## Data sources
+
+- Apple Health (exporting via AutoHealthExport)
+- Workout (evaluating a cou
+- Nutrition
+- Daily habit tracking
+- ActivityWatch
+- WakaAPI
+
+
 ## Scripts
--- a/dsr-input/dsr-gather-gitea.sh
+++ b/dsr-input/dsr-gather-gitea.sh
@@ -1,83 +0,0 @@
-#!/bin/bash
-
-# Script to query Gitea API for a user's activity on a specific date
-
-########################################################################################################
-#Obtain gitea api key from bitwarden
-########################################################################################################
-
-####################################
-## Step 0: Set to use tsys server
-####################################
-bw logout
-
-echo "Setting cli to use tsys bitwarden server..."
-bw config server https://pwvault.turnsys.com
-
-####################################
-## Step 1: login to bitwarden
-####################################
-
-# From: https://bitwarden.com/help/cli/#using-an-api-key
-
-### Set apikey environment varaible
-
-echo "Sourcing clientid/apikey data..."
-source D:/tsys/secrets/bitwarden/data/apikey-bitwarden-reachableceo
-
-### Login to vault using apikey...
-
-echo "Logging in..."
-bw login --apikey $BW_CLIENTID $BW_CLIENTSECRET
-
-### Step 1.1: unlock / save session id 
-
-echo "Unlocking..."
-export BW_SESSION="$(bw unlock --passwordenv TSYS_BW_PASSWORD_REACHABLECEO --raw)"
-
-
-### Step 2: retrive a value into an environment variable
-
-export GITEA_APIKEY="$(bw get password APIKEY-Gitea)"
-
-########################################################################################################
-# Accrss gitea data
-########################################################################################################
-
-# Script to query Gitea API for a user's activity on a specific date
-
-# Usage: ./get_gitea_user_activity.sh <username> <date> [GITEA_URL] [TOKEN]
-
-# Set username, date, and default Gitea URL
-USERNAME="${1:-reachableceo}"                    # Default to "reachableceo" if not provided
-DATE="${2:-$(date +%Y-%m-%d)}"                   # Default to today's date if not provided
-GITEA_URL="${3:-https://git.knownelement.com}"   # Default Gitea URL if not provided
-TOKEN="${GITEA_APIKEY}"                      # Use APIKEY-GItea or passed argument
-
-# API Endpoint for user activities
-API_ENDPOINT="$GITEA_URL/api/v1/users/$USERNAME/timeline"
-
-# Make the API call
-if [ -n "$TOKEN" ]; then
-  # If token is provided, use it in the Authorization header
-  RESPONSE=$(curl -s -H "Authorization: token $TOKEN" "$API_ENDPOINT")
-else
-  # If no token is provided, make an unauthenticated request
-  RESPONSE=$(curl -s "$API_ENDPOINT")
-fi
-
-# Check for API errors
-if [[ "$RESPONSE" == "Not found" || -z "$RESPONSE" ]]; then
-  echo "Error: User '$USERNAME' not found, or endpoint is incorrect."
-  exit 1
-fi
-
-# Validate JSON response
-if ! echo "$RESPONSE" | jq empty >/dev/null 2>&1; then
-  echo "Error: Invalid JSON response from Gitea API."
-  echo "Response: $RESPONSE"
-  exit 1
-fi
-
-# Filter the activity by date using jq
-echo "$RESPONSE" | jq --arg date "$DATE" '[.[] | select(.created_at | startswith($date))]'
--- a/dsr-input/export-joplin-md.sh
+++ b/dsr-input/export-joplin-md.sh
@@ -1,59 +0,0 @@
-#!/bin/bash
-
-# Functions
-
-# Search for a note by title and return its ID
-search_note() {
-    local title="$1"
-    echo "Searching for note with title: $title"
-    note_id=$(joplin search "$title" --fields id --limit 1 --json | jq -r '.[0].id')
-    
-    if [[ -z "$note_id" || "$note_id" == "null" ]]; then
-        echo "Error: Note with title '$title' not found."
-        exit 1
-    fi
-
-    echo "Found note with ID: $note_id"
-    echo "$note_id"
-}
-
-# Export the note by ID to the specified directory
-export_note() {
-    local note_id="$1"
-    local output_dir="$2"
-
-    echo "Exporting note ID: $note_id to directory: $output_dir"
-    mkdir -p "$output_dir" || {
-        echo "Error: Unable to create directory $output_dir"
-        exit 1
-    }
-
-    # Export the note with attachments to the specified directory
-    joplin export "$note_id" --format md --output "$output_dir" || {
-        echo "Error: Failed to export note ID $note_id"
-        exit 1
-    }
-
-    echo "Note exported successfully to $output_dir"
-}
-
-# Main function
-main() {
-    if [[ $# -lt 2 ]]; then
-        echo "Usage: $0 <note-title> <output-directory>"
-        exit 1
-    fi
-
-    local note_title="$1"
-    local output_dir="$2"
-
-    # Search for the note and get its ID
-    local note_id
-    note_id=$(search_note "$note_title")
-
-    # Export the note
-    export_note "$note_id" "$output_dir"
-}
-
-# Run the main function with all script arguments
-main "$@"
--- a/dsr-joplin-create/dsr-new.sh
+++ b/dsr-joplin-create/dsr-new.sh
@@ -1,97 +0,0 @@
-#!/usr/bin/env bash
-
-# Use the Joplin CLI and create a new note for today. Let me pass in the notebook folder path. Use bash functions.
-
-# Created by chatgpt
-# https://chatgpt.com/share/6750c7df-6b2c-8005-a91c-1bc5b3875170
-
-
-# shellcheck disable=SC1090
-
-# Bash3 Boilerplate Setup
-set -o errexit
-set -o nounset
-set -o pipefail
-IFS=$'\n\t'
-
-# Constants
-readonly SCRIPT_NAME=$(basename "$0")
-readonly SCRIPT_VERSION="1.0"
-readonly SCRIPT_AUTHOR="Charles N Wyble"
-readonly SCRIPT_DESC="Create a new Daily Stakeholder Report note in Joplin"
-
-# Logging and Debugging (from bash3boilerplate)
-readonly LOG_FILE="/tmp/${SCRIPT_NAME}.log"
-readonly TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
-info() { echo "[INFO] [$TIMESTAMP] $*"; }
-error() { echo "[ERROR] [$TIMESTAMP] $*" >&2; }
-
-# Default Exit Codes
-readonly ERR_JOPLIN_NOT_INSTALLED=10
-readonly ERR_NOTEBOOK_NOT_FOUND=20
-
-# Function: Usage Instructions
-usage() {
-    cat <<EOF
-$SCRIPT_DESC
-
-Usage:
-  $SCRIPT_NAME <notebook_folder_path>
-
-Example:
-  $SCRIPT_NAME "Stakeholder Reports"
-
-Options:
-  -h, --help    Display this help message.
-EOF
-}
-
-# Check dependencies
-require_joplin() {
-    if ! command -v joplin &>/dev/null; then
-        error "Joplin CLI is not installed or not in PATH. Please install it and try again."
-        exit $ERR_JOPLIN_NOT_INSTALLED
-    fi
-}
-
-# Check if notebook exists in Joplin
-notebook_exists() {
-    local notebook="$1"
-    if ! joplin use "$notebook" >/dev/null 2>&1; then
-        error "Notebook '$notebook' does not exist. Please create it first."
-        exit $ERR_NOTEBOOK_NOT_FOUND
-    fi
-}
-
-# Create a new note in the specified notebook
-create_note() {
-    local notebook="$1"
-    local today_date=$(date '+%d-%m-%Y') # Format: YYYY-MM-DD
-    local title="Daily Stakeholder Report - $date"
-    local content="## Daily Stakeholder Report\n\n**Date:** $date\n\n---\n\n### Key Updates\n\n- \n\n### Issues\n\n- \n\n### Next Steps\n\n- \n"
-
-    joplin create note --title "$title" --body "$content" >/dev/null
-    info "Note created successfully in notebook '$notebook'."
-}
-
-# Main Function
-main() {
-    local notebook_path="$1"
-
-    # Ensure Joplin CLI is present
-    require_joplin
-
-    # Verify notebook existence
-    notebook_exists "$notebook_path"
-
-    # Create a new note
-    create_note "$notebook_path"
-}
-
-# Argument Parsing
-if [[ $# -lt 1 ]] || [[ "$1" == "-h" ]] || [[ "$1" == "--help" ]]; then
-    usage
-    exit 0
-fi
-
-main "$1"
--- a/dsr-joplin-create/dsr-populate-objectives.sh
+++ b/dsr-joplin-create/dsr-populate-objectives.sh
@@ -1,114 +0,0 @@
-#!/usr/bin/env bash
-
-# Written by ChatGPT
-# https://chatgpt.com/share/6750c7df-6b2c-8005-a91c-1bc5b3875170
-
-# Prompt:
-# Pull any issues from Redmine that have a due date of today via API
-
-# shellcheck disable=SC1090
-
-# Bash3 Boilerplate Setup
-set -o errexit
-set -o nounset
-set -o pipefail
-IFS=$'\n\t'
-
-# Constants
-readonly SCRIPT_NAME=$(basename "$0")
-readonly SCRIPT_VERSION="1.0"
-readonly SCRIPT_AUTHOR="Charles N Wyble"
-readonly SCRIPT_DESC="Pull Redmine issues with a due date of today and output in Markdown"
-
-# Logging and Debugging
-readonly LOG_FILE="/tmp/${SCRIPT_NAME}.log"
-readonly TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
-info() { echo "[INFO] [$TIMESTAMP] $*" | tee -a "$LOG_FILE"; }
-error() { echo "[ERROR] [$TIMESTAMP] $*" >&2 | tee -a "$LOG_FILE"; }
-
-# Default Exit Codes
-readonly ERR_CURL_NOT_INSTALLED=10
-readonly ERR_API_FAILURE=20
-
-# Configuration
-readonly REDMINE_INSTANCE="projects.knownelement.com"
-readonly REDMINE_API_URL="https://${REDMINE_INSTANCE}"
-readonly REDMINE_API_KEY="your_api_key_here"
-readonly DATE=$(date '+%Y-%m-%d')
-
-# Function: Usage Instructions
-usage() {
-    cat <<EOF
-$SCRIPT_DESC
-
-Usage:
-  $SCRIPT_NAME <project_id>
-
-Options:
-  -h, --help    Display this help message.
-
-Example:
-  $SCRIPT_NAME 123
-EOF
-}
-
-# Check dependencies
-require_curl() {
-    if ! command -v curl &>/dev/null; then
-        error "curl is not installed or not in PATH. Please install it and try again."
-        exit $ERR_CURL_NOT_INSTALLED
-    fi
-}
-
-# Fetch issues from Redmine API
-fetch_issues() {
-    local project_id="$1"
-    local url="${REDMINE_API_URL}/issues.json?key=${REDMINE_API_KEY}&project_id=${project_id}&due_date=${DATE}"
-
-    info "Fetching issues with due date ${DATE} for project ID ${project_id}..."
-    response=$(curl -s -w "%{http_code}" -o /tmp/redmine_response.json "$url")
-    http_code=$(tail -n1 <<<"$response")
-
-    if [[ "$http_code" -ne 200 ]]; then
-        error "Failed to fetch issues from Redmine API. HTTP Code: $http_code"
-        exit $ERR_API_FAILURE
-    fi
-
-    info "Issues fetched successfully. Parsing and converting to Markdown..."
-    generate_markdown /tmp/redmine_response.json
-}
-
-# Generate Markdown output from JSON response
-generate_markdown() {
-    local json_file="$1"
-    local markdown_output="/tmp/redmine_issues.md"
-
-    echo "# Redmine Issues for ${DATE}" > "$markdown_output"
-    echo "" >> "$markdown_output"
-
-    jq -r --arg base_url "$REDMINE_API_URL" '.issues[] | "## [\(.subject)](\($base_url)/issues/\(.id))\n- **ID**: \(.id)\n- **Status**: \(.status.name)\n- **Due Date**: \(.due_date)\n\n---"' \
-        "$json_file" >> "$markdown_output"
-
-    info "Markdown output written to $markdown_output"
-    cat "$markdown_output"
-}
-
-# Main Function
-main() {
-    local project_id="$1"
-
-    # Ensure curl is installed
-    require_curl
-
-    # Fetch issues for the project
-    fetch_issues "$project_id"
-}
-
-# Argument Parsing
-if [[ $# -lt 1 ]] || [[ "$1" == "-h" ]] || [[ "$1" == "--help" ]]; then
-    usage
-    exit 0
-fi
-
-main "$1"
-
--- a/local/DSRVariables.env
+++ b/local/DSRVariables.env
@@ -1,46 +1,48 @@
-###################################################
-# Modify these values to suit
-###################################################
-
-#############################################################################
-#SET THESE VARIABLES OR NOTHING WILL WORK!!!!
-
-export PipelineClientWorkingDir="D:/tsys/@ReachableCEO/DSR-Pipeline-ReachableCEO/local"
-export StakeholderOutputMarkdownInputFile="../../StakeholderJoplin/@DailyStakeholderReports/PostedToDiscourse/DSR-12-10-2024.md"
-
-#############################################################################
-
-##########################################
-# Layout/title page /formatting options
-##########################################
-
-export ReportAuthor="Charles N Wyble"
-export AuthorTagline="Tenaciy. Velocity. Focus."
-export AuthorLogo="D:/tsys/@ReachableCEO/ReachableCEO.png"
-export SourceCode="https://git.knownelement.com/reachableceo/DSR_Pipeline-ReachableCEO"
-export URLCOLOR="blue"
-export PAGEBACKGROUND="$PipelineClientWorkingDir/build/background5.pdf"
-export PANDOC_TEMPLATE="eisvogel"
-export YamlInputTemplateFileStakeholderOutput="$PipelineClientWorkingDir/build/BuildTemplate-DSR.yml"
-
-################################################################### 
-# Output Artifact variables
-################################################################### 
-
-export BUILD_TEMP_DIR="$PipelineClientWorkingDir/build-temp"
-export BUILD_OUTPUT_DIR="$PipelineClientWorkingDir/build-output"
-export BUILDYAML_STAKEHOLDER_OUTPUT="$BUILD_TEMP_DIR/DSR.yml"
-
-export StakeholderOutputMarkdownOutputFile="$BUILD_OUTPUT_DIR/DSR.md"
-export StakeholderOutputPDFOutputFile="$BUILD_OUTPUT_DIR/DSR.pdf"
-
-################################################################### 
-# Publish variables
-################################################################### 
-
-export BITWARDEN_SERVER_URL="https://pwvault.turnsys.com"
-export BITWARDEN_CREDS="D:/tsys/secrets/bitwarden/data/apikey-bitwarden-reachableceo"
-export DISCOURSE_URL="https://community.turnsys.com"
-export DISCOURSE_API_USERNAME="reachableceo"
-export DISCOURSE_CATEGORY_ID="61"
-export DISCOURSE_POST_TITLE="Daily Stakeholder Report"
+###################################################
+# Modify these values to suit
+###################################################
+
+#############################################################################
+#SET THESE VARIABLES OR NOTHING WILL WORK!!!!
+
+export PipelineClientWorkingDir="D:/tsys/ReachableCEOPublic/Software/DSR-Pipeline-ReachableCEO/local"
+export StakeholderOutputMarkdownInputFile="$1" 
+
+#############################################################################
+
+##########################################
+# Layout/title page /formatting options
+##########################################
+
+export ReportAuthor="Charles N Wyble"
+export AuthorTagline="Tenaciy. Velocity. Focus."
+export AuthorLogo="D:/tsys/@ReachableCEO/ReachableCEO.png"
+export SourceCode="https://git.knownelement.com/reachableceo/DSR-Pipeline-ReachableCEO"
+export URLCOLOR="blue"
+export PAGEBACKGROUND="$PipelineClientWorkingDir/build/background5.pdf"
+export PANDOC_TEMPLATE="eisvogel"
+export YamlInputTemplateFileStakeholderOutput="$PipelineClientWorkingDir/build/BuildTemplate-DSR.yml"
+
+################################################################### 
+# Output Artifact variables
+################################################################### 
+
+export BUILD_TEMP_DIR="$PipelineClientWorkingDir/build-temp"
+export BUILD_OUTPUT_DIR="$PipelineClientWorkingDir/build-output"
+export BUILDYAML_STAKEHOLDER_OUTPUT="$BUILD_TEMP_DIR/DSR.yml"
+
+export StakeholderOutputMarkdownOutputFile="$BUILD_OUTPUT_DIR/$(basename $StakeholderOutputMarkdownInputFile|awk -F '.' '{print $1}').md" 
+export StakeholderOutputPDFOutputFile="$BUILD_OUTPUT_DIR/$(basename $StakeholderOutputMarkdownInputFile|awk -F '.' '{print $1}').pdf" 
+
+################################################################### 
+# Publish variables
+################################################################### 
+
+export BITWARDEN_SERVER_URL="https://pwvault.turnsys.com"
+export BITWARDEN_CREDS="D:/tsys/secrets/bitwarden/data/apikey-bitwarden-reachableceo"
+export DISCOURSE_URL="https://community.turnsys.com"
+export DISCOURSE_API_USERNAME="reachableceo"
+export DISCOURSE_CATEGORY_ID="61"
+#export TODAY_DATE="$(date +%m-%d-%Y)"
+export TODAY_DATE="12-21-2024"
+export DISCOURSE_POST_TITLE="Daily Stakeholder Report for $TODAY_DATE"
--- a/local/build/BuildTemplate-2025Plan.yml
+++ b/local/build/BuildTemplate-2025Plan.yml
@@ -1,12 +1,12 @@
-title: ReachableCEO 2025 Plan
-titlepage: true
-toc: true
-toc-own-page: true
-header-left: "\\hspace{1cm}"
-header-center: "\\leftmark"
-header-right: "Page \\thepage"
-footer-left: "Charles N Wyble"
-footer-center: "Tenacity. Velocity. Focus."
-page-background: "./background5.pdf"
-titlepage-logo: "D:/tsys/@ReachableCEO/ReachableCEO.png"
-urlcolor: blue
+title: ReachableCEO 2025 Plan
+titlepage: true
+toc: true
+toc-own-page: true
+header-left: "\\hspace{1cm}"
+header-center: "\\leftmark"
+header-right: "Page \\thepage"
+footer-left: "Charles N Wyble"
+footer-center: "Tenacity. Velocity. Focus."
+page-background: "./background5.pdf"
+titlepage-logo: "D:/tsys/@ReachableCEO/ReachableCEO.png"
+urlcolor: blue
--- a/local/build/BuildTemplate-DSR.yml
+++ b/local/build/BuildTemplate-DSR.yml
@@ -1,7 +1,8 @@
-title: "{{ReportAuthor}} Daily Stakeholder Report : \today"
+title: "Daily Stakeholder Report for {{ReportAuthor}}"
 titlepage: true
+toc: true
+toc-own-page: true
 titlepage-logo: "{{CandidateLogo}}"
-date: \today
 header-left: "\\hspace{1cm}"
 header-center: "\\leftmark"
 header-right: "Page \\thepage"
--- a/local/build/publish-dsr.sh
+++ b/local/build/publish-dsr.sh
@@ -0,0 +1,75 @@
+source ../DSRVariables.env
+
+echo "Obtaining discourse api key..."
+bw logout
+bw config server "$BITWARDEN_SERVER_URL"
+source "$BITWARDEN_CREDS"
+bw login --apikey $BW_CLIENTID $BW_CLIENTSECRET
+export BW_SESSION="$(bw unlock --passwordenv TSYS_BW_PASSWORD_REACHABLECEO --raw)"
+export DISCOURSE_API_KEY="$(bw get password APIKEY-discourse)"
+
+echo "Posting DSR..."
+
+# The content of the post
+CONTENT="Please use the link below to download today's stakeholder report."
+
+# The file to upload (from the second argument or auto-generated based on date)
+FILE_PATH="$StakeholderOutputPDFOutputFile"
+
+# Check if the file exists
+if [ ! -f "$FILE_PATH" ]; then
+  echo "File not found: $FILE_PATH"
+  exit 1
+fi
+
+# Upload the file
+echo "Uploading file..."
+upload_response=$(curl -s -X POST "$DISCOURSE_URL/uploads.json" \
+  -H "Content-Type: multipart/form-data" \
+  -H "Api-Key: $DISCOURSE_API_KEY" \
+  -H "Api-Username: $DISCOURSE_API_USERNAME" \
+  -F "file=@$FILE_PATH;type=application/pdf" \
+  -F "type=composer")
+
+echo "Upload Response: $upload_response"
+
+# Extract the short_url from the response
+short_url=$(echo "$upload_response" | /mingw64/bin/jq -r '.short_url')
+
+# Check if the short_url was returned
+if [ "$short_url" == "null" ]; then
+  echo "Failed to extract short_url. Response:"
+  echo "$upload_response"
+  exit 1
+fi
+
+echo "File uploaded successfully. Short URL: $short_url"
+
+# Append the file link to the post content (Markdown format)
+CONTENT="$CONTENT\n\n[Download todays report in PDF format]($short_url)"
+
+# Create the new topic
+echo "Creating new topic..."
+post_response=$(curl -s -X POST "$DISCOURSE_URL/posts.json" \
+  -H "Content-Type: application/json" \
+  -H "Api-Key: $DISCOURSE_API_KEY" \
+  -H "Api-Username: $DISCOURSE_API_USERNAME" \
+  -d @- <<EOF
+{
+  "title": "$DISCOURSE_POST_TITLE",
+  "raw": "$CONTENT",
+  "category": $DISCOURSE_CATEGORY_ID
+}
+EOF
+)
+
+echo "Post Response: $post_response"
+
+# Check if the post creation was successful
+if echo "$post_response" | grep -q '"id":'; then
+  echo "Post created successfully!"
+else
+  echo "Failed to create post. Response:"
+  echo "$post_response"
+  exit 1
+fi
--- a/microservices/DSRCreateNewEntry/DSRCreateNewEntry-Script-12-19-2024-14-13-27.sh
+++ b/microservices/DSRCreateNewEntry/DSRCreateNewEntry-Script-12-19-2024-14-13-27.sh
@@ -0,0 +1,110 @@
+#!/bin/bash
+# Copyright ReachableCEO Enterprises 2025
+# Licensed under the AGPL v3.0
+
+set -euo pipefail
+IFS=$'\n\t'
+
+# Enable logging
+log_file="LOG-DSRCreateNewEntry-12-19-2024-14-13-27.log"
+exec > >(tee -a "$log_file") 2>&1
+
+function log_message()
+
+{
+
+    echo "$(date '+%m-%d-%Y-%H-%M-%S') $1"
+
+}
+
+function error_exit()
+
+{
+
+    echo -e "\033[0;31mERROR: $1\033[0m" >&2
+    log_message "ERROR: $1"
+    exit 1
+
+}
+
+# Trap for cleanup on script exit
+trap "error_exit 'Script interrupted or an error occurred.'" ERR
+
+# Read and verify environment variables
+env_file="../../DSRVariables.env"
+if [[ ! -f "$env_file" ]]; then
+    error_exit "Environment file $env_file not found."
+fi
+
+source "$env_file"
+
+: "${JOPLIN_HOST:?Environment variable JOPLIN_HOST is not set}"
+: "${JOPLIN_PORT:?Environment variable JOPLIN_PORT is not set}"
+: "${JOPLIN_SOURCE_NOTE_TITLE:?Environment variable JOPLIN_SOURCE_NOTE_TITLE is not set}"
+: "${JOPLIN_TARGET_NOTEBOOK:?Environment variable JOPLIN_TARGET_NOTEBOOK is not set}"
+
+# Authenticate and set Joplin token
+function get_joplin_apikey()
+
+{
+
+    bw logout
+    bw config server https://pwvault.turnsys.com
+    echo "Sourcing clientid/apikey data..."
+    source D:/tsys/secrets/bitwarden/data/apikey-bitwarden-reachableceo
+    bw login --apikey $BW_CLIENTID $BW_CLIENTSECRET
+    export BW_SESSION="$(bw unlock --passwordenv TSYS_BW_PASSWORD_REACHABLECEO --raw)"
+    export JOPLIN_TOKEN="$(bw get password APIKEY-Joplin-Streaming)"
+
+}
+
+get_joplin_apikey
+
+: "${JOPLIN_TOKEN:?Joplin API key is not set}"
+
+# Find the ID of the source note
+function find_note_id()
+
+{
+
+    local page=1
+    local note_id
+    while true; do
+        response=$(curl -s "http://${JOPLIN_HOST}:${JOPLIN_PORT}/notes?token=${JOPLIN_TOKEN}&page=${page}")
+        note_id=$(echo "$response" | jq -r --arg title "$JOPLIN_SOURCE_NOTE_TITLE" '.items[] | select(.title == $title) | .id')
+        if [[ -n "$note_id" ]]; then
+            echo "$note_id"
+            return
+        fi
+        if [[ "$(echo "$response" | jq -r '.has_more')" == "false" ]]; then
+            break
+        fi
+        ((page++))
+    done
+
+    error_exit "Source note titled '$JOPLIN_SOURCE_NOTE_TITLE' not found."
+
+}
+
+note_id=$(find_note_id)
+
+# Clone the note to the target notebook
+function clone_note()
+
+{
+
+    local cloned_note_title="DSR-$(date '+%m-%d-%Y')"
+    curl -s -X POST "http://${JOPLIN_HOST}:${JOPLIN_PORT}/notes?token=${JOPLIN_TOKEN}"         -H "Content-Type: application/json"         --data @<(cat <<EOF
+{
+    "title": "$cloned_note_title",
+    "body": "$(curl -s "http://${JOPLIN_HOST}:${JOPLIN_PORT}/notes/${note_id}?fields=body&token=${JOPLIN_TOKEN}" | jq -r '.body')",
+    "parent_id": "$(curl -s "http://${JOPLIN_HOST}:${JOPLIN_PORT}/folders?token=${JOPLIN_TOKEN}" | jq -r --arg name "$JOPLIN_TARGET_NOTEBOOK" '.items[] | select(.title == $name) | .id')"
+}
+EOF
+        ) || error_exit "Failed to clone note."
+
+    log_message "Cloned note titled '$JOPLIN_SOURCE_NOTE_TITLE' to '$cloned_note_title' in notebook '$JOPLIN_TARGET_NOTEBOOK'."
+
+}
+
+clone_note
--- a/microservices/DSRCreateNewEntry/LOG-DSRCreateNewEntry-12-19-2024-14-13-27.log
+++ b/microservices/DSRCreateNewEntry/LOG-DSRCreateNewEntry-12-19-2024-14-13-27.log
@@ -0,0 +1,31 @@
+./DSRCreateNewEntry-Script-12-19-2024-14-13-27.sh: line 44: JOPLIN_TARGET_NOTEBOOK: Environment variable JOPLIN_TARGET_NOTEBOOK is not set
+Logout required before server config update.You have logged out.Saved setting `config`.Sourcing clientid/apikey data...
+You are logged in!
+
+To unlock your vault, use the `unlock` command. ex:
+$ bw unlockjq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+jq: error (at <stdin>:1): Cannot iterate over null (null)
+You have logged out.Saved setting `config`.Sourcing clientid/apikey data...
+You are logged in!
+
+To unlock your vault, use the `unlock` command. ex:
+$ bw unlock{"error":"Internal Server Error: Expected ',' or '}' after property value in JSON at position 562 (line 1 column 563): \n\nSyntaxError: Expected ',' or '}' after property value in JSON at position 562 (line 1 column 563)\n    at JSON.parse (<anonymous>)\n    at Object.default_1 [as fn] (C:\\Users\\tsys\\AppData\\Local\\Temp\\2nn4H0kRc241YEfWibWsJYlam2u\\resources\\app.asar\\node_modules\\@joplin\\lib\\services\\rest\\routes\\notes.js:429:34)\n    at Api.route (C:\\Users\\tsys\\AppData\\Local\\Temp\\2nn4H0kRc241YEfWibWsJYlam2u\\resources\\app.asar\\node_modules\\@joplin\\lib\\services\\rest\\Api.js:151:37)\n    at execRequest (C:\\Users\\tsys\\AppData\\Local\\Temp\\2nn4H0kRc241YEfWibWsJYlam2u\\resources\\app.asar\\node_modules\\@joplin\\lib\\ClipperServer.js:161:54)\n    at IncomingMessage.<anonymous> (C:\\Users\\tsys\\AppData\\Local\\Temp\\2nn4H0kRc241YEfWibWsJYlam2u\\resources\\app.asar\\node_modules\\@joplin\\lib\\ClipperServer.js:204:34)\n    at IncomingMessage.emit (node:events:514:28)\n    at endReadableNT (node:internal/streams/readable:1408:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)"}12-19-2024-14-26-00 Cloned note titled 'DSR-Template' to 'DSR-12-19-2024' in notebook 'Today'.
--- a/microservices/DSRInstrumentedCTORedmineGather/DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25-GitMsg.txt
+++ b/microservices/DSRInstrumentedCTORedmineGather/DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25-GitMsg.txt
@@ -0,0 +1,5 @@
+Added DSR-InstrumentedCTO-RedmineGather script and test suite.
+
+- Script gathers data from Redmine API and processes it.
+- Includes robust error handling and detailed logging.
+- Test suite verifies data gathering functionality.
--- a/microservices/DSRInstrumentedCTORedmineGather/DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25-Script.sh
+++ b/microservices/DSRInstrumentedCTORedmineGather/DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25-Script.sh
@@ -0,0 +1,65 @@
+#!/bin/bash
+
+# Copyright ReachableCEO Enterprises 2025
+# Licensed under the GNU Affero General Public License v3.0
+
+# Enable strict mode
+set -euo pipefail
+IFS=$'\n\t'
+
+# Variables
+LOG_FILE="LOG-DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25.log"
+
+# Logging functions
+log_info() {
+    local msg="$1"
+    echo -e "\033[32m[INFO] $(date '+%m-%d-%Y-%H-%M-%S') $msg\033[0m" | tee -a "$LOG_FILE"
+}
+
+log_error() {
+    local msg="$1"
+    echo -e "\033[31m[ERROR] $(date '+%m-%d-%Y-%H-%M-%S') $msg\033[0m" | tee -a "$LOG_FILE" >&2
+}
+
+# Cleanup on exit
+cleanup() {
+    log_info "Cleaning up temporary files..."
+}
+trap cleanup EXIT
+
+# Function: Gather data from Redmine API
+gather_redmine_data() {
+
+    log_info "Starting data gathering from Redmine API..."
+
+    # Placeholder: Replace with your actual API URL and token
+    local api_url="https://your-redmine-instance/api/issues.json"
+    local api_token="your_api_token_here"
+
+    log_info "Fetching data from Redmine API..."
+    response=$(curl -s -H "X-Redmine-API-Key: $api_token" "$api_url")
+    if [[ $? -ne 0 ]]; then
+        log_error "Failed to fetch data from Redmine API."
+        exit 1
+    fi
+
+    log_info "Processing data..."
+    echo "$response" | jq '.' > redmine_data.json
+    if [[ $? -ne 0 ]]; then
+        log_error "Failed to process Redmine API response."
+        exit 1
+    fi
+
+    log_info "Data gathering completed successfully."
+}
+
+# Main execution
+main() {
+
+    log_info "Executing DSR-InstrumentedCTO-RedmineGather script."
+    gather_redmine_data
+
+    log_info "Script execution completed."
+}
+
+main
--- a/microservices/DSRInstrumentedCTORedmineGather/DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25-TestSuite.sh
+++ b/microservices/DSRInstrumentedCTORedmineGather/DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25-TestSuite.sh
@@ -0,0 +1,49 @@
+#!/bin/bash
+
+# Copyright ReachableCEO Enterprises 2025
+# Licensed under the GNU Affero General Public License v3.0
+
+# Enable strict mode
+set -euo pipefail
+IFS=$'\n\t'
+
+# Test Suite for DSR-InstrumentedCTO-RedmineGather
+
+# Variables
+LOG_FILE="test-LOG-DSR-InstrumentedCTO-RedmineGather-12-20-2024-21-34-25.log"
+
+# Logging functions for tests
+log_info() {
+    local msg="$1"
+    echo -e "\033[32m[INFO] $(date '+%m-%d-%Y-%H-%M-%S') $msg\033[0m" | tee -a "$LOG_FILE"
+}
+
+log_error() {
+    local msg="$1"
+    echo -e "\033[31m[ERROR] $(date '+%m-%d-%Y-%H-%M-%S') $msg\033[0m" | tee -a "$LOG_FILE" >&2
+}
+
+# Test for gather_redmine_data
+test_gather_redmine_data() {
+    log_info "Testing gather_redmine_data function..."
+
+    # Mock Redmine API response
+    local mock_response='{"issues": []}'
+    echo "$mock_response" > redmine_data.json
+
+    if [[ -f redmine_data.json ]]; then
+        log_info "Test passed: Data file created."
+    else
+        log_error "Test failed: Data file not created."
+        exit 1
+    fi
+}
+
+# Run all tests
+main() {
+    log_info "Running tests for DSR-InstrumentedCTO-RedmineGather..."
+    test_gather_redmine_data
+    log_info "All tests passed."
+}
+
+main
--- a/vendor/git.knownelement.com/reachableceo/DSR-Pipeline-Server/build/build-stakeholder-output-server.sh
+++ b/vendor/git.knownelement.com/reachableceo/DSR-Pipeline-Server/build/build-stakeholder-output-server.sh
@@ -7,8 +7,6 @@ $MO_PATH $YamlInputTemplateFileStakeholderOutput > $BUILDYAML_STAKEHOLDER_OUTPUT

 echo "Creating stakeholder report..."

-export StakeholderOutputMarkdownInputFile="../../StakeholderJoplin/@DailyStakeholderReports/PostedToDiscourse/DSR-12-10-2024.md"
-
 cd "$(dirname $StakeholderOutputMarkdownInputFile)"

 pandoc \
Author	SHA1	Message	Date
ReachableCEO	174e51cf43	pipeline changes, preparing to dockerize etc	2025-07-11 22:01:17 -05:00
Charles N Wyble	bb5a6e21e1	Added DSR-InstrumentedCTO-RedmineGather script and test suite. - Script gathers data from Redmine API and processes it. - Includes robust error handling and detailed logging. - Test suite verifies data gathering functionality.	2024-12-20 15:36:38 -06:00
Charles N Wyble	18ce558453	v0.2	2024-12-18 16:59:20 -06:00
Charles N Wyble	494e0a7cf3	v0.0.1-rollbackpoint	2024-12-18 16:46:33 -06:00
Charles N Wyble	08ff4640ff	v0.0.1	2024-12-18 16:10:39 -06:00
Charles N Wyble	18936e9511	publish related bits	2024-12-16 20:09:21 -06:00
Charles N Wyble	9d863805ab	.	2024-12-16 12:59:20 -06:00
Charles N Wyble	8e5ba5b20b	Update "DSR-Pipeline-Server" from "https://git.knownelement.com/reachableceo/DSR-Pipeline-Server.git@main " git-vendor-name: DSR-Pipeline-Server git-vendor-dir: vendor/git.knownelement.com/reachableceo/DSR-Pipeline-Server git-vendor-repository: https://git.knownelement.com/reachableceo/DSR-Pipeline-Server.git git-vendor-ref: main	2024-12-16 12:53:17 -06:00
Charles N Wyble	b8b7bcb594	Squashed 'vendor/git.knownelement.com/reachableceo/DSR-Pipeline-Server/' changes from ca6f700..be2f756 be2f756 removed some debugging bits git-subtree-dir: vendor/git.knownelement.com/reachableceo/DSR-Pipeline-Server git-subtree-split: be2f75604843d2ea78cb895cc63388ea42f7789a	2024-12-16 12:53:17 -06:00
Charles N Wyble	6fb158edfe	better input and output file handling	2024-12-16 12:52:46 -06:00