111 lines
3.8 KiB
Markdown
111 lines
3.8 KiB
Markdown
# TSYS-AIOS-GIS-Tools-GIS-Processing Container
|
|
|
|
This container is part of the TSYS-AIOS-GIS project and provides advanced GIS data processing capabilities with Jupyter notebooks and workflow tools.
|
|
|
|
## Overview
|
|
|
|
The TSYS-AIOS-GIS-Tools-GIS-Processing container extends the base GIS container with advanced processing tools, Jupyter notebooks for interactive analysis, and workflow orchestration tools. This container is designed for in-depth geospatial data analysis and complex ETL workflows.
|
|
|
|
## Tools Included
|
|
|
|
### Extends from Base Container
|
|
- All tools from TSYS-AIOS-GIS-Tools-GIS-Base container
|
|
- GIS libraries, weather data processing libraries, visualization tools
|
|
|
|
### Advanced Processing Tools
|
|
- **Jupyter Notebook**: Interactive environment for data analysis
|
|
- **Node.js/npm**: JavaScript runtime and package manager
|
|
- **IPyKernel**: IPython kernel for Jupyter
|
|
|
|
### Workflow Tools
|
|
- **Apache Airflow**: Workflow orchestration platform
|
|
- **Prefect**: Modern workflow management
|
|
|
|
## Usage
|
|
|
|
### Building the Processing Container
|
|
```bash
|
|
# From this directory
|
|
cd /home/localuser/AIWorkspace/TSYS-AIOS-GIS/Docker/TSYS-AIOS-GIS-Tools-GIS-Processing
|
|
|
|
# Use the wrapper script to automatically detect and set user IDs
|
|
./docker-compose-wrapper.sh build
|
|
|
|
# Or run commands in the processing container with automatic user mapping
|
|
./docker-compose-wrapper.sh run tsys-gis-processing [command]
|
|
|
|
# Example: Start Jupyter notebook server
|
|
./docker-compose-wrapper.sh run tsys-gis-processing jupyter notebook --ip=0.0.0.0 --port=8888 --allow-root --notebook-dir=/workspace --no-browser
|
|
|
|
# Example: Start an interactive bash session
|
|
./docker-compose-wrapper.sh run tsys-gis-processing bash
|
|
```
|
|
|
|
### Using with docker-compose directly
|
|
```bash
|
|
# Set environment variables and run docker-compose directly
|
|
LOCAL_USER_ID=$(id -u) LOCAL_GROUP_ID=$(id -g) docker-compose up --build
|
|
|
|
# Or export variables first
|
|
export LOCAL_USER_ID=$(id -u)
|
|
export LOCAL_GROUP_ID=$(id -g)
|
|
docker-compose up
|
|
```
|
|
|
|
### Using the wrapper script
|
|
```bash
|
|
# Build and start the processing container with automatic user mapping
|
|
./docker-compose-wrapper.sh up --build
|
|
|
|
# Start without rebuilding (Jupyter will be available on port 8888)
|
|
./docker-compose-wrapper.sh up
|
|
|
|
# View container status
|
|
./docker-compose-wrapper.sh ps
|
|
|
|
# Stop containers
|
|
./docker-compose-wrapper.sh down
|
|
```
|
|
|
|
## Jupyter Notebook Access
|
|
|
|
When running the container with `docker-compose up`, Jupyter notebook will be available at:
|
|
- http://localhost:8888
|
|
|
|
The notebook server is preconfigured to:
|
|
- Use the workspace directory as the notebook directory
|
|
- Allow access without authentication (in container only)
|
|
- Accept connections from any IP address
|
|
|
|
## User ID Mapping (For File Permissions)
|
|
|
|
The container automatically detects and uses the host user's UID and GID to ensure proper file permissions. This means:
|
|
|
|
- Files created inside the container will have the correct ownership on the host
|
|
- No more root-owned files after container operations
|
|
- Works across different environments (development, production servers)
|
|
|
|
The container detects the user ID from the mounted workspace volume. If needed, you can override the default values by setting environment variables:
|
|
|
|
```bash
|
|
# Set specific user ID and group ID before running docker-compose
|
|
export LOCAL_USER_ID=1000
|
|
export LOCAL_GROUP_ID=1000
|
|
docker-compose up
|
|
```
|
|
|
|
Or run with inline environment variables:
|
|
```bash
|
|
LOCAL_USER_ID=1000 LOCAL_GROUP_ID=1000 docker-compose up
|
|
```
|
|
|
|
The container runs as a non-root user named `TSYS-Tools` with the detected host user's UID/GID.
|
|
|
|
## Data Processing Workflows
|
|
|
|
This container is optimized for:
|
|
- Interactive geospatial analysis using Jupyter notebooks
|
|
- Complex ETL workflows using Apache Airflow or Prefect
|
|
- Advanced visualization and reporting
|
|
- Model development and testing
|
|
- Integration with PostGIS and other databases |