API Response Patterns

This document defines the standardized patterns for API responses in the BitFlow platform to ensure consistency across all endpoints.

Core Principles

1. Consistency is Power: All API responses follow the same patterns
2. No Redundant Data: Avoid exposing both ID fields and nested objects
3. Nested Relationships: Use full object relationships instead of ID-only references
4. Standardized Wrappers: All responses use consistent wrapper structures

Response Wrapper Types

Single Resource Response

All single resource endpoints return data wrapped in APIResponse[T]:

type APIResponse[T any] struct {
    Data       T                   `json:"data,omitempty"`
    Message    string             `json:"message,omitempty"`
    Pagination *PaginationResponse `json:"pagination,omitempty"`
}

Example Response:

{
  "data": {
    "id": "wf_123",
    "name": "Training Workflow",
    "owner": {
      "id": "user_456",
      "username": "john_doe"
    }
  },
  "message": "Workflow retrieved successfully"
}

List Response

All list endpoints return data wrapped in ListResponse[T]:

type ListResponse[T any] struct {
    Items      []T                 `json:"items"`
    Pagination *PaginationResponse `json:"pagination"`
}

Example Response:

{
  "items": [
    {
      "id": "wf_123",
      "name": "Training Workflow",
      "owner": {
        "id": "user_456",
        "username": "john_doe"
      }
    }
  ],
  "pagination": {
    "page": 1,
    "limit": 20,
    "total": 1,
    "pages": 1
  }
}

Pagination Structure

type PaginationResponse struct {
    Page  int   `json:"page"`
    Limit int   `json:"limit"`
    Total int64 `json:"total"`
    Pages int64 `json:"pages"`
}

Model Relationship Patterns

✅ Correct Pattern: Nested Objects

Model Definition:

type Workflow struct {
    ID          string     `json:"id" gorm:"primaryKey"`
    Name        string     `json:"name"`
    // Hide ID fields from JSON
    OwnerID     string     `json:"-" gorm:"not null;index"`
    WorkspaceID *string    `json:"-" gorm:"index"`

    // Include full nested objects
    Owner       User       `json:"owner" gorm:"foreignKey:OwnerID"`
    Workspace   *Workspace `json:"workspace,omitempty" gorm:"foreignKey:WorkspaceID"`
}

Handler Implementation:

// Always preload relationships
db := h.db.Preload("Owner").Preload("Workspace")
var workflow Workflow
if err := db.First(&workflow, "id = ?", id).Error; err != nil {
    return err
}

JSON Response:

{
  "data": {
    "id": "wf_123",
    "name": "Training Workflow",
    "owner": {
      "id": "user_456",
      "username": "john_doe",
      "email": "john@example.com"
    },
    "workspace": {
      "id": "ws_789",
      "name": "AI Research",
      "slug": "ai-research"
    }
  }
}

❌ Incorrect Pattern: ID Fields + Objects

// DON'T DO THIS
type Workflow struct {
    ID          string     `json:"id"`
    Name        string     `json:"name"`
    OwnerID     string     `json:"owner_id"`     // ❌ Redundant
    WorkspaceID *string    `json:"workspace_id"` // ❌ Redundant
    Owner       User       `json:"owner"`        // ✅ Keep this
    Workspace   *Workspace `json:"workspace"`    // ✅ Keep this
}

This creates redundant data and inconsistent responses.

Handler Response Patterns

Success Response Helpers

// Single resource success
func SuccessResponse[T any](data T) APIResponse[T] {
    return APIResponse[T]{Data: data}
}

// Success with message
func SuccessWithMessage[T any](data T, message string) APIResponse[T] {
    return APIResponse[T]{Data: data, Message: message}
}

// List with pagination
func ListWithPagination[T any](items []T, pagination *PaginationResponse) ListResponse[T] {
    return ListResponse[T]{Items: items, Pagination: pagination}
}

Usage in Handlers

func (h *WorkflowHandler) GetWorkflow(c *gin.Context) {
    id := c.Param("id")

    var workflow models.Workflow
    if err := h.db.Preload("Owner").Preload("Workspace").
        First(&workflow, "id = ?", id).Error; err != nil {
        c.JSON(http.StatusNotFound, gin.H{"message": "Workflow not found"})
        return
    }

    c.JSON(http.StatusOK, SuccessResponse(workflow))
}

func (h *WorkflowHandler) ListWorkflows(c *gin.Context) {
    // ... query logic ...

    var workflows []models.Workflow
    db := h.db.Preload("Owner").Preload("Workspace")

    // Get paginated results
    result := db.Offset(offset).Limit(limit).Find(&workflows)

    // Get total count
    var total int64
    h.db.Model(&models.Workflow{}).Count(&total)

    pagination := &PaginationResponse{
        Page:  page,
        Limit: limit,
        Total: total,
        Pages: (total + int64(limit) - 1) / int64(limit),
    }

    c.JSON(http.StatusOK, ListWithPagination(workflows, pagination))
}

CLI Type Alignment

CLI Types Must Match API Response

CLI Model (in pkg/types/types.go):

type Workflow struct {
    ID          string     `json:"id"`
    Name        string     `json:"name"`
    // NO ID fields - only nested objects
    Owner       User       `json:"owner"`
    Workspace   *Workspace `json:"workspace,omitempty"`
}

Response Type Aliases

// Use type aliases for consistency
type WorkflowListResponse = ListResponse[Workflow]
type DatasetListResponse = ListResponse[Dataset]
type RunListResponse = ListResponse[Run]

Validation Checklist

When creating new endpoints or modifying existing ones:

• Model uses json:"-" for ID fields that have corresponding nested objects
• Handler preloads all necessary relationships
• Response uses appropriate wrapper (APIResponse[T] or ListResponse[T])
• CLI types match API response structure exactly
• No redundant ID fields exposed in JSON
• Pagination included for list endpoints
• Error responses follow standard format
• ALL models consistently hide relationship ID fields
• Nested objects follow same serialization depth
• Cross-reference with existing models for consistency

Examples by Resource Type

Workflows

• Model: models.Workflow with json:"-" for OwnerID, WorkspaceID
• Response: Always includes Owner and Workspace objects
• CLI Type: types.Workflow with nested Owner, Workspace structs

Datasets

• Model: models.Dataset with json:"-" for OwnerID, WorkspaceID
• Response: Always includes Owner, Workspace, Datacenter objects
• CLI Type: types.Dataset with nested relationship structs

Runs

• Model: models.Run with appropriate relationship preloading
• Response: Includes User, Workspace, Workflow objects as needed
• CLI Type: types.Run with nested relationship structs

Migration Notes

When updating existing endpoints:

1. API Models: Add json:"-" tags to ID fields that have corresponding nested objects
2. Handlers: Ensure proper Preload() calls for all relationships
3. CLI Types: Remove ID fields, keep only nested object fields
4. Testing: Verify response structure matches documentation

Benefits of This Pattern

1. Consistency: All resources follow the same relationship pattern
2. Efficiency: Single request provides complete nested data
3. Maintainability: Clear separation between internal IDs and API responses
4. Developer Experience: Predictable API structure across all endpoints
5. Future-Proof: Easy to extend relationships without breaking changes

Preventing API Inconsistencies

Root Causes of Inconsistencies

1. Model Definition Drift: Different models evolve independently without cross-referencing
2. Missing Documentation: No central reference for relationship patterns
3. Lack of Validation: No automated checks for response structure consistency
4. Copy-Paste Errors: Duplicating patterns without ensuring they match existing ones

Systematic Prevention Strategies

1. Model Definition Standards

// ✅ STANDARD PATTERN - Follow this for ALL models
type ResourceModel struct {
    ID          string    `json:"id" gorm:"primaryKey"`
    Name        string    `json:"name"`
    // Hide ALL relationship ID fields
    OwnerID     string    `json:"-" gorm:"not null;index"`
    WorkspaceID *string   `json:"-" gorm:"index"`

    // Include nested objects
    Owner       User      `json:"owner" gorm:"foreignKey:OwnerID"`
    Workspace   *Workspace `json:"workspace,omitempty" gorm:"foreignKey:WorkspaceID"`
}

2. Handler Preloading Standards

// ✅ STANDARD PATTERN - Always preload core relationships
func (h *Handler) GetResource(c *gin.Context) {
    var resource models.Resource
    // ALWAYS preload Owner and Workspace for consistency
    if err := h.db.Preload("Owner").Preload("Workspace").
        First(&resource, "id = ?", id).Error; err != nil {
        // ... error handling
    }
    c.JSON(http.StatusOK, SuccessResponse(resource))
}

3. Automated Consistency Checks

Create tests to verify response structure:

func TestAPIResponseConsistency(t *testing.T) {
    // Test that all resources follow same Owner/Workspace pattern
    workflows := getWorkflows()
    datasets := getDatasets()

    // Verify no ID fields exposed
    assert.NoField(t, workflows[0], "owner_id")
    assert.NoField(t, workflows[0], "workspace_id")
    assert.NoField(t, datasets[0], "owner_id")
    assert.NoField(t, datasets[0], "workspace_id")

    // Verify nested objects present
    assert.HasField(t, workflows[0], "owner")
    assert.HasField(t, datasets[0], "owner")
}

4. Model Review Process

Before adding/modifying any model:

1. Check existing models for similar patterns
2. Use identical field naming (OwnerID, WorkspaceID, etc.)
3. Apply consistent JSON tags (json:"-" for relationship IDs)
4. Follow same relationship depth (don’t over-nest or under-nest)
5. Update this documentation with new patterns

5. CLI Type Generation

Consider auto-generating CLI types from API models:

//go:generate go run tools/generate-cli-types.go

// This would ensure CLI types automatically match API structure

Consistency Enforcement Rules

1. All relationship ID fields MUST use json:"-"
2. All models with Owner MUST include Owner User relationship
3. All models with Workspace MUST include Workspace *Workspace relationship
4. Handler preloading MUST be consistent across similar resources
5. Response wrappers MUST be used consistently (APIResponse[T], ListResponse[T])
6. CLI types MUST mirror API response structure exactly

Red Flags - Immediate Review Required

• ❌ Any model exposing both owner_id and owner fields
• ❌ Inconsistent relationship naming (user vs owner)
• ❌ Different serialization depths between similar resources
• ❌ Missing preload calls in handlers
• ❌ CLI types with ID fields that APIs don’t expose

This document should be updated whenever API response patterns are modified.

📋 Action Items for New Features:

1. Reference this document before creating new models
2. Cross-check with existing similar resources
3. Run consistency tests after implementation
4. Update CLI types to match API responses exactly