Job#

The Job is the core execution unit in MultiFlexi that manages the complete lifecycle of application execution, from initialization to completion and artifact preservation.

Job Lifecycle#

Overview#

Every MultiFlexi job follows a well-defined lifecycle that ensures reliable execution, comprehensive logging, and automatic artifact preservation. The job lifecycle consists of several distinct phases:

  1. Initialization - Job creation and environment setup

  2. Pre-execution - Configuration loading and validation

  3. Execution - Application launch and monitoring

  4. Post-execution - Result processing and cleanup

  5. Artifact Preservation - Automatic storage of outputs and results

Lifecycle Phases#

Initialization Phase#

When a job is created, it undergoes initial setup:

  • Job Object Creation: A new Job instance is instantiated with a unique ID

  • Environment Setup: Configuration fields and environment variables are initialized

  • Template Association: The job is linked to its RunTemplate for execution parameters

  • Logging Setup: Zabbix monitoring is configured if enabled

  • Status Tracking: Initial job state is set to “loaded”

Pre-execution Phase (runBegin)#

Before execution starts, the job prepares the runtime environment:

  • Configuration Loading: Environment variables and credentials are loaded

  • Command Line Building: The full executable command is constructed

  • Resource Validation: Required files and dependencies are checked

  • Status Update: Job phase changes to “running”

  • Monitoring Setup: Execution tracking and logging begins

Execution Phase (launchJob)#

The core execution phase where the actual application runs:

  • Process Launch: The configured application executable is started

  • Output Monitoring: stdout and stderr are captured in real-time

  • Status Tracking: Job status and progress are continuously monitored

  • Resource Management: System resources are tracked and managed

  • Error Handling: Exceptions and failures are captured and logged

Post-execution Phase (runEnd)#

After execution completes, the job processes results:

  • Exit Code Capture: The application’s exit status is recorded

  • Output Processing: stdout and stderr are finalized and stored

  • Status Update: Job phase is updated based on success/failure

  • Performance Metrics: Execution time and resource usage are calculated

  • Action Execution: Configured actions (like Zabbix reporting) are triggered

Artifact Preservation Phase#

New in Latest Version: Every job automatically preserves its artifacts:

  • Result Files: Any output files generated by the application are stored

  • Standard Output: Complete stdout content is saved as stdout.txt

  • Standard Error: Complete stderr content is saved as stderr.txt

  • Metadata Storage: File metadata including MIME types and descriptions

  • Database Integration: All artifacts are linked to the job via foreign keys

Key Methods#

The Job class provides several key methods that drive the lifecycle:

performJob()#

The main orchestration method that coordinates the entire job execution:

public function performJob(): void
{
    $this->runBegin();
    $this->executor->launchJob();
    $this->runEnd($exitCode, $stdout, $stderr);
}

runBegin()#

Initializes the job execution environment and starts monitoring.

runEnd($statusCode, $stdout, $stderr)#

Completes job execution, processes results, and triggers artifact preservation:

  • Records final job status and timing information

  • Automatically calls createJobArtifacts() for comprehensive output preservation

  • Executes configured actions (notifications, reporting, etc.)

  • Updates job state in the database

createJobArtifacts() (Private)#

Automatically preserves job outputs as artifacts:

  • Result File Processing: Detects and stores application output files

  • Output Preservation: Saves stdout and stderr with proper content types

  • MIME Type Detection: Automatically determines file types for proper handling

  • Error Resilience: Continues processing even if individual artifacts fail

Artifact Storage System#

Architecture#

The artifact storage system provides automatic, comprehensive preservation of job outputs:

Database Schema#

Artifacts are stored in the artifacts table with the following structure:

  • id - Primary key

  • job_id - Foreign key reference to the job table

  • filename - Original filename or descriptive name

  • content_type - MIME type of the artifact content

  • content - The actual artifact data (BLOB)

  • note - Optional description or metadata

  • created - Timestamp of artifact creation

Automatic Preservation#

Every job execution automatically creates artifacts for:

Result Files

Any files produced by the application in the job’s output directory are detected and stored with proper MIME type identification.

Standard Output (stdout.txt)

Complete standard output stream is preserved as a text artifact, enabling debugging and result analysis.

Standard Error (stderr.txt)

Complete standard error stream is preserved, facilitating troubleshooting and error analysis.

Benefits#

The automatic artifact preservation system provides several key benefits:

  • Complete Audit Trail: Every job execution leaves a complete record of its outputs

  • Debugging Support: Access to stdout/stderr simplifies troubleshooting failed jobs

  • Result Preservation: Application outputs are never lost, regardless of action configuration

  • Consistent Behavior: All jobs follow the same artifact preservation pattern

  • Action Independence: Actions can focus on their specific functionality without managing artifacts

Command-Line Job Management#

MultiFlexi provides comprehensive command-line tools for job management through the multiflexi-cli utility.

Installation#

The CLI tool is available through the multiflexi-cli package:

# Installation via package manager
sudo apt install multiflexi-cli

# Or run directly from source
./bin/multiflexi-cli

Available Commands#

Job Operations#

The job command provides comprehensive job management capabilities:

# List all jobs
multiflexi-cli job:list

# Show job details
multiflexi-cli job:show <job-id>

# Create a new job
multiflexi-cli job:create --template <template-id> --company <company-id>

# Execute a job immediately
multiflexi-cli job:run <job-id>

# Delete a job
multiflexi-cli job:delete <job-id>

Queue Management#

Manage job execution queues:

# Show job queue status
multiflexi-cli queue:status

# Process pending jobs
multiflexi-cli queue:process

# Clear failed jobs from queue
multiflexi-cli queue:clear

Artifact Management#

Manage job artifacts through the CLI:

# List artifacts for a job
multiflexi-cli artifact:list --job <job-id>

# Download an artifact
multiflexi-cli artifact:get <artifact-id> --output <filename>

# Show artifact details
multiflexi-cli artifact:show <artifact-id>

Script Generation#

Generate standalone scripts from job configurations:

# Generate shell script for a job
multiflexi-job2script <job-id> > job_script.sh

# Generate environment file
multiflexi-job2env <job-id> > job.env

Configuration Management#

Manage job-related configurations:

# List run templates
multiflexi-cli runtemplate:list

# Show template configuration
multiflexi-cli runtemplate:show <template-id>

# List companies
multiflexi-cli company:list

Environment Configuration#

The CLI tool requires proper database configuration through environment files:

# Default configuration file
/etc/multiflexi/.env

# Custom environment file
multiflexi-cli --environment /path/to/.env job:list

Required environment variables:

  • DB_CONNECTION - Database type (mysql, pgsql, etc.)

  • DB_HOST - Database host

  • DB_PORT - Database port

  • DB_DATABASE - Database name

  • DB_USERNAME - Database username

  • DB_PASSWORD - Database password

Integration and Automation#

Monitoring and Alerting#

Combined with Zabbix integration, job execution provides comprehensive monitoring:

  • Real-time Status: Job phases and status updates sent to Zabbix

  • Performance Metrics: Execution time and resource usage tracking

  • Failure Alerts: Automatic notification when jobs fail

  • Artifact Availability: All job outputs preserved for analysis

Best Practices#

Job Design#

  • Idempotent Operations: Design jobs to be safely re-runnable

  • Clear Output: Ensure applications produce clear, parseable output

  • Exit Codes: Use proper exit codes to indicate success/failure status

  • Resource Cleanup: Clean up temporary resources within the application

Artifact Management#

  • File Organization: Structure output files logically for easy artifact retrieval

  • Size Considerations: Monitor artifact storage usage for large-output applications

  • Retention Policies: Implement cleanup procedures for old artifacts as needed

  • Content Types: Use standard file extensions to ensure proper MIME type detection

Monitoring and Maintenance#

  • Regular Monitoring: Monitor job success rates and execution times

  • Log Analysis: Review job logs and artifacts for optimization opportunities

  • Queue Management: Ensure job queues don’t become bottlenecks

  • Database Maintenance: Implement periodic cleanup of old job and artifact data