Testing & Development

Table of Contents

• Mock Mode
• Dry Run Mode
• Development Workflow

Mock Mode

Test workflows without real addons using mock mode:

# Run with built-in mocks
nexroo run config.json --mock

# Run with custom mock configuration
nexroo run config.json --mock --mock-config mocks.json

Mock Configuration

Create a mocks.json file for custom mock responses:

{
  "my-addon::llm_completion": {
    "completion": "Mock response to: {{prompt}}",
    "model": "mock-gpt-4",
    "tokens_used": 150,
    "finish_reason": "stop"
  },
  "my-addon::semantic_search": {
    "results": [
      {
        "id": "doc_1",
        "title": "Mock Document 1",
        "content": "Mock content for query: {{query}}",
        "score": 0.95
      }
    ],
    "total": 1
  },
  "my-db::query": {
    "rows": [
      {"id": 1, "name": "Mock User", "email": "mock@example.com"}
    ],
    "count": 1
  }
}

Built-in Mock Actions

The system includes pre-configured mocks for common operations:

Available Default Mocks:

• my-addon::llm_completion - LLM completion responses with realistic token usage
• my-addon::semantic_search - Semantic search results with relevance scores
• my-addon::database_query - Database query results with rows and metadata
• my-addon::api_call - HTTP API responses with status codes and data
• example_action_name - Generic action responses for testing

Example Mock Files: The repository includes example mock configuration files:

• mocks.json - Sample mock configurations with template variables
• mock_workflow.json - Complete workflow example using mocks with step-level overrides

Mock Response Examples

Actual Mock Response Format: All mock responses are wrapped with metadata:

{
  "mock": true,
  "action": "my-addon::llm_completion",
  "output": {
    "completion": "Mock LLM response to prompt: {{prompt}}",
    "model": "mock-gpt-4", 
    "tokens_used": 150,
    "finish_reason": "stop",
    "confidence": 0.92
  },
  "timestamp": "2024-01-01T00:00:00Z"
}

LLM Completion Mock (from mocks.json):

{
  "completion": "Mock LLM response to prompt: {{prompt}}",
  "model": "mock-gpt-4",
  "tokens_used": 150,
  "finish_reason": "stop",
  "confidence": 0.92
}

Semantic Search Mock (from mocks.json):

{
  "results": [
    {
      "id": "doc_1",
      "title": "Mock Document 1",
      "content": "This is a mock document for semantic search testing with query: {{query}}",
      "score": 0.95,
      "metadata": {
        "source": "mock_database",
        "timestamp": "2024-01-01T00:00:00Z"
      }
    },
    {
      "id": "doc_2", 
      "title": "Mock Document 2",
      "content": "Another mock document with relevant content",
      "score": 0.87,
      "metadata": {
        "source": "mock_database",
        "timestamp": "2024-01-01T00:00:00Z"
      }
    }
  ],
  "total": 2,
  "query": "{{query}}",
  "execution_time": 0.05
}

Database Query Mock (from mocks.json):

{
  "rows": [
    {"id": 1, "name": "Mock Record 1", "value": "test_value_1"},
    {"id": 2, "name": "Mock Record 2", "value": "test_value_2"}
  ],
  "count": 2,
  "query": "{{query}}",
  "execution_time": 0.02
}

Mock Features

• Template Variables: Mocks can use {{variables}} from execution context
• Dynamic Behavior: Use functions for dynamic mock responses
• Realistic Data: Built-in mocks for common AI operations
• Step Overrides: Override mocks per step for specific testing scenarios
• Performance Simulation: Realistic response times and resource usage

Step-Level Mock Overrides

Override mocks for specific steps using the mockOutput parameter:

{
  "id": "step-mock-override",
  "action": "my-addon::semantic_search",
  "parameters": {
    "query": "custom test query",
    "mockOutput": {
      "results": [
        {
          "id": "override_doc_1",
          "title": "Custom Mock Document",
          "content": "This is a custom mock override for testing step-level mocks",
          "score": 1.0,
          "metadata": {
            "source": "step_override",
            "custom": true
          }
        }
      ],
      "total": 1,
      "query": "custom test query",
      "execution_time": 0.01
    }
  }
}

Note: Step-level mock overrides take precedence over global mock configurations. See mock_workflow.json for a complete example.

Dry Run Mode

Analyze workflow structure without executing actions:

# Analyze workflow structure
nexroo run config.json --dry-run

# Dry run with verbose output
nexroo run config.json --dry-run --verbose

Dry Run Analysis

Dry run provides comprehensive workflow analysis:

• Step Dependencies: Shows which steps depend on others
• Template Variables: Identifies all template variables used
• Execution Plan: Simulates the execution order
• Potential Issues: Identifies problems like missing dependencies or infinite loops

Example dry run output:

[INFO] DRY RUN MODE - No actions will be executed
[INFO] Analyzing workflow structure...
[INFO] [DRY RUN] Would execute step: Validate Input (id: validate-input)
[INFO] [DRY RUN] Action: builtin::log
[INFO] [DRY RUN] Step validate-input simulation completed
[WARNING] [DRY RUN] Potential issue: Step 'process-data' depends on 'fetch-data' but it might not execute
[INFO] [DRY RUN] Simulated execution of 8 step instances

Development Workflow

1. Design Phase: Use dry run to validate workflow structure
2. Development Phase: Use mock mode for development and testing
3. Integration Testing: Test with custom mock scenarios
4. Production Deployment: Run with real addons

# 1. Design and validate
nexroo run workflow.json --dry-run

# 2. Develop with mocks
nexroo run workflow.json --mock

# 3. Integration test with custom scenarios
nexroo run workflow.json --mock --mock-config test-scenarios.json

# 4. Deploy to production
nexroo run workflow.json