Job ¶

The Job is the core execution unit in MultiFlexi that manages the complete lifecycle of application execution, from initialization to completion and artifact preservation.

Job Lifecycle ¶

Overview ¶

Every MultiFlexi job follows a well-defined lifecycle that ensures reliable execution, comprehensive logging, and automatic artifact preservation. The job lifecycle consists of several distinct phases:

Initialization - Job creation and environment setup
Pre-execution - Configuration loading and validation
Execution - Application launch and monitoring
Post-execution - Result processing and cleanup
Artifact Preservation - Automatic storage of outputs and results

Lifecycle Phases ¶

Initialization Phase ¶

When a job is created, it undergoes initial setup:

Job Object Creation: A new Job instance is instantiated with a unique ID
Environment Setup: Configuration fields and environment variables are initialized
Template Association: The job is linked to its RunTemplate for execution parameters
Logging Setup: Zabbix monitoring is configured if enabled
Status Tracking: Initial job state is set to “loaded”

Pre-execution Phase (runBegin)¶

Before execution starts, the job prepares the runtime environment:

Configuration Loading: Environment variables and credentials are loaded
Command Line Building: The full executable command is constructed
Resource Validation: Required files and dependencies are checked
Status Update: Job phase changes to “running”
Monitoring Setup: Execution tracking and logging begins

Execution Phase (launchJob)¶

The core execution phase where the actual application runs:

Process Launch: The configured application executable is started
Output Monitoring: stdout and stderr are captured in real-time
Status Tracking: Job status and progress are continuously monitored
Resource Management: System resources are tracked and managed
Error Handling: Exceptions and failures are captured and logged

Post-execution Phase (runEnd)¶

After execution completes, the job processes results:

Exit Code Capture: The application’s exit status is recorded
Output Processing: stdout and stderr are finalized and stored
Status Update: Job phase is updated based on success/failure
Performance Metrics: Execution time and resource usage are calculated
Action Execution: Configured actions (like Zabbix reporting) are triggered

Artifact Preservation Phase ¶

New in Latest Version: Every job automatically preserves its artifacts:

Result Files: Any output files generated by the application are stored
Standard Output: Complete stdout content is saved as stdout.txt
Standard Error: Complete stderr content is saved as stderr.txt
Metadata Storage: File metadata including MIME types and descriptions
Database Integration: All artifacts are linked to the job via foreign keys

Key Methods ¶

The Job class provides several key methods that drive the lifecycle:

performJob()¶

The main orchestration method that coordinates the entire job execution:

public function performJob(): void
{
    $this->runBegin();
    $this->executor->launchJob();
    $this->runEnd($exitCode, $stdout, $stderr);
}

runBegin()¶

Initializes the job execution environment and starts monitoring.

runEnd($statusCode, $stdout, $stderr)¶

Completes job execution, processes results, and triggers artifact preservation:

Records final job status and timing information
Automatically calls createJobArtifacts() for comprehensive output preservation
Executes configured actions (notifications, reporting, etc.)
Updates job state in the database

createJobArtifacts() (Private)¶

Automatically preserves job outputs as artifacts:

Result File Processing: Detects and stores application output files
Output Preservation: Saves stdout and stderr with proper content types
MIME Type Detection: Automatically determines file types for proper handling
Error Resilience: Continues processing even if individual artifacts fail

Artifact Storage System ¶

Architecture ¶

The artifact storage system provides automatic, comprehensive preservation of job outputs:

Database Schema ¶

Artifacts are stored in the artifacts table with the following structure:

id - Primary key
job_id - Foreign key reference to the job table
filename - Original filename or descriptive name
content_type - MIME type of the artifact content
content - The actual artifact data (BLOB)
note - Optional description or metadata
created - Timestamp of artifact creation

Automatic Preservation ¶

Every job execution automatically creates artifacts for:

Result Files: Any files produced by the application in the job’s output directory are detected and stored with proper MIME type identification.
Standard Output (stdout.txt): Complete standard output stream is preserved as a text artifact, enabling debugging and result analysis.
Standard Error (stderr.txt): Complete standard error stream is preserved, facilitating troubleshooting and error analysis.

Benefits ¶

The automatic artifact preservation system provides several key benefits:

Complete Audit Trail: Every job execution leaves a complete record of its outputs
Debugging Support: Access to stdout/stderr simplifies troubleshooting failed jobs
Result Preservation: Application outputs are never lost, regardless of action configuration
Consistent Behavior: All jobs follow the same artifact preservation pattern
Action Independence: Actions can focus on their specific functionality without managing artifacts

Command-Line Job Management ¶

MultiFlexi provides comprehensive command-line tools for job management through the multiflexi-cli utility.

Installation ¶

The CLI tool is available through the multiflexi-cli package:

# Installation via package manager
sudo apt install multiflexi-cli

# Or run directly from source
./bin/multiflexi-cli

Available Commands ¶

Job Operations ¶

The job command provides comprehensive job management capabilities:

# List all jobs
multiflexi-cli job:list

# Show job details
multiflexi-cli job:show <job-id>

# Create a new job
multiflexi-cli job:create --template <template-id> --company <company-id>

# Execute a job immediately
multiflexi-cli job:run <job-id>

# Delete a job
multiflexi-cli job:delete <job-id>

Queue Management ¶

Manage job execution queues:

# Show job queue status
multiflexi-cli queue:status

# Process pending jobs
multiflexi-cli queue:process

# Clear failed jobs from queue
multiflexi-cli queue:clear

Artifact Management ¶

Manage job artifacts through the CLI:

# List artifacts for a job
multiflexi-cli artifact:list --job <job-id>

# Download an artifact
multiflexi-cli artifact:get <artifact-id> --output <filename>

# Show artifact details
multiflexi-cli artifact:show <artifact-id>

Script Generation ¶

Generate standalone scripts from job configurations:

# Generate shell script for a job
multiflexi-job2script <job-id> > job_script.sh

# Generate environment file
multiflexi-job2env <job-id> > job.env

Configuration Management ¶

Manage job-related configurations:

# List run templates
multiflexi-cli runtemplate:list

# Show template configuration
multiflexi-cli runtemplate:show <template-id>

# List companies
multiflexi-cli company:list

Environment Configuration ¶

The CLI tool requires proper database configuration through environment files:

# Default configuration file
/etc/multiflexi/.env

# Custom environment file
multiflexi-cli --environment /path/to/.env job:list

Required environment variables:

DB_CONNECTION - Database type (mysql, pgsql, etc.)
DB_HOST - Database host
DB_PORT - Database port
DB_DATABASE - Database name
DB_USERNAME - Database username
DB_PASSWORD - Database password

Integration and Automation ¶

Monitoring and Alerting ¶

Combined with Zabbix integration, job execution provides comprehensive monitoring:

Real-time Status: Job phases and status updates sent to Zabbix
Performance Metrics: Execution time and resource usage tracking
Failure Alerts: Automatic notification when jobs fail
Artifact Availability: All job outputs preserved for analysis

Best Practices ¶

Job Design ¶

Idempotent Operations: Design jobs to be safely re-runnable
Clear Output: Ensure applications produce clear, parseable output
Exit Codes: Use proper exit codes to indicate success/failure status
Resource Cleanup: Clean up temporary resources within the application

Artifact Management ¶

File Organization: Structure output files logically for easy artifact retrieval
Size Considerations: Monitor artifact storage usage for large-output applications
Retention Policies: Implement cleanup procedures for old artifacts as needed
Content Types: Use standard file extensions to ensure proper MIME type detection

Monitoring and Maintenance ¶

Regular Monitoring: Monitor job success rates and execution times
Log Analysis: Review job logs and artifacts for optimization opportunities
Queue Management: Ensure job queues don’t become bottlenecks
Database Maintenance: Implement periodic cleanup of old job and artifact data