Taskpack Versioning Architecture

Overview

While Flows are simple configurations that don’t need versioning, Taskpacks are repositories of executable code that require proper version management for:

• Algorithm improvements
• Bug fixes
• Security patches
• Compatibility updates
• Performance optimizations

Version Management System

1. Semantic Versioning

Taskpacks follow semantic versioning (MAJOR.MINOR.PATCH):

# manifest.yaml
metadata:
  name: fedavg-resnet50-pathology
  version: 2.1.3
  # MAJOR: Breaking changes in API or data format
  # MINOR: New features, backwards compatible
  # PATCH: Bug fixes and minor improvements

2. Version Registry

# Taskpack Registry Structure
packages:
  fedavg-resnet50-pathology:
    latest: 2.1.3
    versions:
      2.1.3:
        released: "2024-03-22"
        url: "s3://packages/fedavg-resnet50-pathology-2.1.3.tar.gz"
        changelog: "Fixed memory leak in aggregation step"
        compatibility: ">=1.0.0"
      2.1.2:
        released: "2024-03-15"
        url: "s3://packages/fedavg-resnet50-pathology-2.1.2.tar.gz"
        changelog: "Improved GPU utilization"
      2.0.0:
        released: "2024-02-01"
        url: "s3://packages/fedavg-resnet50-pathology-2.0.0.tar.gz"
        changelog: "Breaking: New data format, 30% faster training"

3. Version Selection in Flows

When creating a Flow, users can:

• Use latest stable version (default)
• Pin to specific version for reproducibility
• Use version ranges for flexibility

interface FlowConfig {
  // ... other config
  taskPackage: {
    name: string;
    version?: string; // Optional: defaults to "latest"
    // Examples:
    // version: "2.1.3"     // Exact version
    // version: "^2.1.0"    // Compatible with 2.1.x
    // version: ">=2.0.0"   // Minimum version
  };
}

UI Implementation

Taskpack Selection with Versions

// In Mission Package Selector
interface MissionPackage {
  id: string;
  name: string;
  currentVersion: string;
  availableVersions: Version[];
  // ...
}
 
interface Version {
  version: string;
  released: Date;
  changelog: string;
  stability: "stable" | "beta" | "experimental";
  downloads: number;
}

Version Display in UI

1. Package Selection Screen:

   • Show current/recommended version prominently
   • Provide dropdown for version selection
   • Display changelog for each version
   • Warn about breaking changes

2. Flow History:

   • Track which package version was used
   • Allow re-running with same version
   • Suggest upgrades when available

Benefits

1. Reproducibility: Research results can be reproduced with exact package versions
2. Stability: Production flows can pin versions to avoid unexpected changes
3. Innovation: New algorithms can be tested without affecting existing flows
4. Compliance: Version history for regulatory requirements
5. Rollback: Easy rollback if issues are discovered

Example Use Cases

Research Scenario

# Researcher testing new algorithm
flow:
  name: "Test New Biomarker Algorithm v3"
  taskPackage:
    name: "biomarker-discovery-spatial"
    version: "3.0.0-beta.1" # Testing beta version

Production Scenario

# Hospital running validated workflow
flow:
  name: "Daily Cancer Screening"
  taskPackage:
    name: "cancer-detection-ensemble"
    version: "1.5.8" # Pinned to validated version

Pharma Compliance

# Pharma company with regulatory requirements
flow:
  name: "FDA Submission Analysis"
  taskPackage:
    name: "toxicity-prediction"
    version: "2.3.1" # FDA-validated version
    locked: true # Prevent accidental updates

Migration Strategy

When breaking changes occur:

1. Deprecation Notice: Mark old versions as deprecated
2. Migration Guide: Provide clear upgrade instructions
3. Compatibility Mode: Support old data formats temporarily
4. Gradual Rollout: Allow testing before forcing upgrades

Security Considerations

1. Package Signing: All versions cryptographically signed
2. Vulnerability Scanning: Automated security checks
3. License Compliance: Track dependencies and licenses
4. Audit Trail: Complete history of version usage

Conclusion

Taskpack versioning is essential for a professional platform where:

• Code evolves and improves
• Results must be reproducible
• Compliance is required
• Multiple teams collaborate

This system provides the flexibility of continuous improvement while maintaining the stability required for production use.