API Response Patterns

This document defines the standardized patterns for API responses in the BitFlow platform to ensure consistency across all endpoints.

Core Principles

Consistency is Power: All API responses follow the same patterns
No Redundant Data: Avoid exposing both ID fields and nested objects
Nested Relationships: Use full object relationships instead of ID-only references
Standardized Wrappers: All responses use consistent wrapper structures

Response Wrapper Types

Single Resource Response

All single resource endpoints return data wrapped in APIResponse[T]:


type APIResponse[T any] struct {
    Data       T                   `json:"data,omitempty"`
    Message    string             `json:"message,omitempty"`
    Pagination *PaginationResponse `json:"pagination,omitempty"`
}

Example Response:


{
  "data": {
    "id": "wf_123",
    "name": "Training Workflow",
    "owner": {
      "id": "user_456",
      "username": "john_doe"
    }
  },
  "message": "Workflow retrieved successfully"
}

List Response

All list endpoints return data wrapped in ListResponse[T]:


type ListResponse[T any] struct {
    Items      []T                 `json:"items"`
    Pagination *PaginationResponse `json:"pagination"`
}

Example Response:


{
  "items": [
    {
      "id": "wf_123",
      "name": "Training Workflow",
      "owner": {
        "id": "user_456",
        "username": "john_doe"
      }
    }
  ],
  "pagination": {
    "page": 1,
    "limit": 20,
    "total": 1,
    "pages": 1
  }
}

Pagination Structure


type PaginationResponse struct {
    Page  int   `json:"page"`
    Limit int   `json:"limit"`
    Total int64 `json:"total"`
    Pages int64 `json:"pages"`
}

Model Relationship Patterns

✅ Correct Pattern: Nested Objects

Model Definition:


type Workflow struct {
    ID          string     `json:"id" gorm:"primaryKey"`
    Name        string     `json:"name"`
    // Hide ID fields from JSON
    OwnerID     string     `json:"-" gorm:"not null;index"`
    WorkspaceID *string    `json:"-" gorm:"index"`
 
    // Include full nested objects
    Owner       User       `json:"owner" gorm:"foreignKey:OwnerID"`
    Workspace   *Workspace `json:"workspace,omitempty" gorm:"foreignKey:WorkspaceID"`
}

Handler Implementation:


// Always preload relationships
db := h.db.Preload("Owner").Preload("Workspace")
var workflow Workflow
if err := db.First(&workflow, "id = ?", id).Error; err != nil {
    return err
}

JSON Response:


{
  "data": {
    "id": "wf_123",
    "name": "Training Workflow",
    "owner": {
      "id": "user_456",
      "username": "john_doe",
      "email": "john@example.com"
    },
    "workspace": {
      "id": "ws_789",
      "name": "AI Research",
      "slug": "ai-research"
    }
  }
}

❌ Incorrect Pattern: ID Fields + Objects


// DON'T DO THIS
type Workflow struct {
    ID          string     `json:"id"`
    Name        string     `json:"name"`
    OwnerID     string     `json:"owner_id"`     // ❌ Redundant
    WorkspaceID *string    `json:"workspace_id"` // ❌ Redundant
    Owner       User       `json:"owner"`        // ✅ Keep this
    Workspace   *Workspace `json:"workspace"`    // ✅ Keep this
}

This creates redundant data and inconsistent responses.

Handler Response Patterns

Success Response Helpers


// Single resource success
func SuccessResponse[T any](data T) APIResponse[T] {
    return APIResponse[T]{Data: data}
}
 
// Success with message
func SuccessWithMessage[T any](data T, message string) APIResponse[T] {
    return APIResponse[T]{Data: data, Message: message}
}
 
// List with pagination
func ListWithPagination[T any](items []T, pagination *PaginationResponse) ListResponse[T] {
    return ListResponse[T]{Items: items, Pagination: pagination}
}

Usage in Handlers


func (h *WorkflowHandler) GetWorkflow(c *gin.Context) {
    id := c.Param("id")
 
    var workflow models.Workflow
    if err := h.db.Preload("Owner").Preload("Workspace").
        First(&workflow, "id = ?", id).Error; err != nil {
        c.JSON(http.StatusNotFound, gin.H{"message": "Workflow not found"})
        return
    }
 
    c.JSON(http.StatusOK, SuccessResponse(workflow))
}
 
func (h *WorkflowHandler) ListWorkflows(c *gin.Context) {
    // ... query logic ...
 
    var workflows []models.Workflow
    db := h.db.Preload("Owner").Preload("Workspace")
 
    // Get paginated results
    result := db.Offset(offset).Limit(limit).Find(&workflows)
 
    // Get total count
    var total int64
    h.db.Model(&models.Workflow{}).Count(&total)
 
    pagination := &PaginationResponse{
        Page:  page,
        Limit: limit,
        Total: total,
        Pages: (total + int64(limit) - 1) / int64(limit),
    }
 
    c.JSON(http.StatusOK, ListWithPagination(workflows, pagination))
}

CLI Type Alignment

CLI Types Must Match API Response

CLI Model (in pkg/types/types.go):


type Workflow struct {
    ID          string     `json:"id"`
    Name        string     `json:"name"`
    // NO ID fields - only nested objects
    Owner       User       `json:"owner"`
    Workspace   *Workspace `json:"workspace,omitempty"`
}

Response Type Aliases


// Use type aliases for consistency
type WorkflowListResponse = ListResponse[Workflow]
type DatasetListResponse = ListResponse[Dataset]
type RunListResponse = ListResponse[Run]

Validation Checklist

When creating new endpoints or modifying existing ones:

Examples by Resource Type

Workflows

Model: models.Workflow with json:"-" for OwnerID, WorkspaceID
Response: Always includes Owner and Workspace objects
CLI Type: types.Workflow with nested Owner, Workspace structs

Datasets

Model: models.Dataset with json:"-" for OwnerID, WorkspaceID
Response: Always includes Owner, Workspace, Datacenter objects
CLI Type: types.Dataset with nested relationship structs

Runs

Model: models.Run with appropriate relationship preloading
Response: Includes User, Workspace, Workflow objects as needed
CLI Type: types.Run with nested relationship structs

Migration Notes

When updating existing endpoints:

API Models: Add json:"-" tags to ID fields that have corresponding nested objects
Handlers: Ensure proper Preload() calls for all relationships
CLI Types: Remove ID fields, keep only nested object fields
Testing: Verify response structure matches documentation

Benefits of This Pattern

Consistency: All resources follow the same relationship pattern
Efficiency: Single request provides complete nested data
Maintainability: Clear separation between internal IDs and API responses
Developer Experience: Predictable API structure across all endpoints
Future-Proof: Easy to extend relationships without breaking changes

Preventing API Inconsistencies

Root Causes of Inconsistencies

Model Definition Drift: Different models evolve independently without cross-referencing
Missing Documentation: No central reference for relationship patterns
Lack of Validation: No automated checks for response structure consistency
Copy-Paste Errors: Duplicating patterns without ensuring they match existing ones

Systematic Prevention Strategies

1. Model Definition Standards


// ✅ STANDARD PATTERN - Follow this for ALL models
type ResourceModel struct {
    ID          string    `json:"id" gorm:"primaryKey"`
    Name        string    `json:"name"`
    // Hide ALL relationship ID fields
    OwnerID     string    `json:"-" gorm:"not null;index"`
    WorkspaceID *string   `json:"-" gorm:"index"`
 
    // Include nested objects
    Owner       User      `json:"owner" gorm:"foreignKey:OwnerID"`
    Workspace   *Workspace `json:"workspace,omitempty" gorm:"foreignKey:WorkspaceID"`
}

2. Handler Preloading Standards


// ✅ STANDARD PATTERN - Always preload core relationships
func (h *Handler) GetResource(c *gin.Context) {
    var resource models.Resource
    // ALWAYS preload Owner and Workspace for consistency
    if err := h.db.Preload("Owner").Preload("Workspace").
        First(&resource, "id = ?", id).Error; err != nil {
        // ... error handling
    }
    c.JSON(http.StatusOK, SuccessResponse(resource))
}

3. Automated Consistency Checks

Create tests to verify response structure:


func TestAPIResponseConsistency(t *testing.T) {
    // Test that all resources follow same Owner/Workspace pattern
    workflows := getWorkflows()
    datasets := getDatasets()
 
    // Verify no ID fields exposed
    assert.NoField(t, workflows[0], "owner_id")
    assert.NoField(t, workflows[0], "workspace_id")
    assert.NoField(t, datasets[0], "owner_id")
    assert.NoField(t, datasets[0], "workspace_id")
 
    // Verify nested objects present
    assert.HasField(t, workflows[0], "owner")
    assert.HasField(t, datasets[0], "owner")
}

4. Model Review Process

Before adding/modifying any model:

Check existing models for similar patterns
Use identical field naming (OwnerID, WorkspaceID, etc.)
Apply consistent JSON tags (json:"-" for relationship IDs)
Follow same relationship depth (don’t over-nest or under-nest)
Update this documentation with new patterns

5. CLI Type Generation

Consider auto-generating CLI types from API models:


//go:generate go run tools/generate-cli-types.go
 
// This would ensure CLI types automatically match API structure

Consistency Enforcement Rules

All relationship ID fields MUST use json:"-"
All models with Owner MUST include Owner User relationship
All models with Workspace MUST include Workspace *Workspace relationship
Handler preloading MUST be consistent across similar resources
Response wrappers MUST be used consistently (APIResponse[T], ListResponse[T])
CLI types MUST mirror API response structure exactly

Red Flags - Immediate Review Required

❌ Any model exposing both owner_id and owner fields
❌ Inconsistent relationship naming (user vs owner)
❌ Different serialization depths between similar resources
❌ Missing preload calls in handlers
❌ CLI types with ID fields that APIs don’t expose

This document should be updated whenever API response patterns are modified.

📋 Action Items for New Features:

Reference this document before creating new models
Cross-check with existing similar resources
Run consistency tests after implementation
Update CLI types to match API responses exactly