AI-Focused Features

The Nexroo Engine provides specialized features designed specifically for LLM-based workflows and agent systems. These features enable advanced context management, memory systems, and intelligent prompt construction.

Token Counting & Prediction System
Context Management System
Named Context Blocks
Dynamic Context Assembly
Storage Tools (useStorage)
Context Tools (useContext)
Semantic Memory System
Template Engine Integration

Token Counting & Prediction System

The Nexroo Engine provides comprehensive token tracking and prediction capabilities for agent-type addons, enabling cost monitoring and workflow optimization.

Token Tracking

All agent-type addons automatically track token usage with detailed breakdown:

{
  "token_usage": {
    "step_tokens": 1574,
    "total_workflow_tokens": 42786,
    "model": "claude-3-5-sonnet-20241022",
    "input_tokens": 41212,
    "output_tokens": 1574,
    "is_prediction": false,
    "source": "addon"
  }
}

Token Fields:

step_tokens: Total tokens used in this step
total_workflow_tokens: Running total across entire workflow
input_tokens: Tokens in the input/prompt
output_tokens: Tokens in the response
is_prediction: Whether this was a prediction or actual execution
source: Whether tokens came from tiktoken (tiktoken) or addon API response (addon)

Token Prediction Mode (for now only OpenAI)

Use predictionOnly to estimate token usage without making actual API calls:

{
  "name": "Cost Estimation Step",
  "id": "step-estimate",
  "action": "my-agent::chat_completion",
  "predictionOnly": true,
  "parameters": {
    "message": "Analyze this complex dataset and provide insights...",
    "system": "You are a data analyst."
  }
}

Workflow Analytics

Token usage is automatically tracked in workflow execution with detailed analytics available in your observability file :

{
  "token_summary": {
    "total_tokens": 45360,
    "prediction_steps": 1,
    "execution_steps": 2,
    "step_breakdown": [
      {
        "step_id": "step-estimate",
        "tokens": 2574,
        "is_prediction": true,
        "source": "tiktoken_prediction"
      }
    ],
    "model_breakdown": {
      "claude-3-5-sonnet-20241022": {
        "total_tokens": 45360,
        "input_tokens": 43786,
        "output_tokens": 1574,
        "step_count": 3
      }
    }
  }
}

Context Management System

The Context Management System is a first-class module that provides LLM-first context features, allowing agents to maintain, build, retrieve, and reason over meaningful context across steps, workflows, sessions, and executions.

Core Capabilities

Tools Native: Use all actions from our engine as tools for the agent in workflow to build powerful automation.
Named Context Blocks: Create reusable context values that persist across workflow steps
Dynamic Context Assembly: Automatically build token-aware, structured context for LLM prompts
Semantic Memory: Store and retrieve contextual information using vector search and tagging
Template Integration: Access context directly in templates using {{context.name}} syntax

Named Context Blocks

Create and manage named context values that can be referenced throughout your workflow.

Setting Context Values

{
  "action": "builtin::context_set",
  "parameters": {
    "name": "task_objective",
    "value": "Create a personalized email campaign for premium customers",
    "session_id": "session_123",
    "metadata": {
      "category": "user_intent",
      "priority": "high"
    }
  }
}

Parameters:

name (required): Unique name for the context
value (required): Any data structure (string, object, array)
session_id (optional): Session identifier for isolation
metadata (optional): Additional metadata for the context

Returns:

{
  "context_name": "task_objective",
  "success": true
}

Retrieving Context Values

{
  "action": "builtin::context_get",
  "parameters": {
    "name": "task_objective"
  },
  "output": "retrieved_objective"
}

Parameters:

name (required): Name of the context to retrieve

Returns:

{
  "value": "Create a personalized email campaign for premium customers",
  "found": true
}

Using Context in Templates

Reference context values directly in your prompts and parameters:

{
  "action": "my-llm::completion",
  "parameters": {
    "prompt": "Using this objective: {{context.task_objective}}, create email copy that resonates with our premium customer segment."
  }
}

Dynamic Context Assembly

Automatically assemble structured, token-aware context blocks from multiple sources for optimal LLM consumption.

Basic Context Assembly

{
  "action": "builtin::context_assembler",
  "parameters": {
    "include": [
      "step-customer_analysis.output",
      "context.user_preferences",
      "payload.campaign_requirements"
    ],
    "max_tokens": 3000,
    "format": "json"
  },
  "output": "assembled_context"
}

Parameters:

include (required): Array of data sources to include in assembly
max_tokens (optional): Maximum tokens for assembled context (default: 3000)
format (optional): Output format - "json" or "text" (default: "text")

Returns:

{
  "assembled_context": "{\"step-customer_analysis.output\": {...}, \"context.user_preferences\": {...}}",
  "format": "json"
}

Using Assembled Context

{
  "action": "my-llm::completion",
  "parameters": {
    "prompt": "Given this comprehensive context: {{assembled_context}}, generate a personalized marketing strategy."
  }
}

Storage Tools (`useStorage`)

Storage tools allow AI agents to access any method from any Nexroo addon as callable functions during chat completion. This includes MongoDB, PostgreSQL, file systems, APIs, and any other addon functionality.

You can configure tools from a single addon or multiple addons in the same step.

Single Database Analysis Example

{
  "id": "analyze-customer-database",
  "action": "claude-1::chat_completion",
  "useStorage": {
    "addonId": "storage-mongo-1",
    "action": [
      {"name": "describe", "description": "Get details about the database."},
      {"name": "describe_collection", "description": "Get detail about a specific collection."}
    ]
  },
  "parameters": {
    "message": "Please examine our customer database and provide a comprehensive analysis of the data structure, collection health, and any insights you can gather.",
    "system": "You are a senior database analyst with provided tools to be used to interact with the database."
  }
}

Multi-Addon Tool Integration

{
  "id": "comprehensive-analysis",
  "action": "claude-1::chat_completion",
  "useStorage": [
    {
      "addonId": "storage-mongo-1",
      "action": [
        {"name": "describe", "description": "Get database schema and collection information."},
        {"name": "describe_collection", "description": "Get detailed collection structure and data samples."}
      ]
    },
    {
      "addonId": "api-connector-1", 
      "action": [
        {"name": "get_routes_definition", "description": "Get available API endpoints and their specifications."},
        {"name": "post", "description": "Send POST requests to API endpoints."},
        {"name": "get", "description": "Send GET requests to API endpoints."}
      ]
    }
  ],
  "parameters": {
    "message": "Analyze customer data using both database and API tools, then report findings.",
    "system": "You have access to both database inspection tools and API client tools. Use them to gather comprehensive data before analysis."
  }
}

useStorage Actions

Actions depend on your installed addons. See each addon's documentation.

Context Tools (`useContext`)

useContext allows AI agents to access builtin context management functions as callable tools during execution. When you configure useContext, the following builtin actions become available as tools for your AI agent:

Context Operations

context_set: Store named context values with optional session isolation
context_get: Retrieve previously stored context values
context_assembler: Build structured context from multiple data sources

Basic Context Management Example

{
  "id": "manage-user-session",
  "action": "claude-1::chat_completion",
  "useContext": {
    "action": [
      {"name": "context_set", "description": "Store data in workflow context."},
      {"name": "context_get", "description": "Retrieve data from workflow context."},
    ]
  },
  "parameters": {
    "message": "....",
    "system": "...."
  }
}

What the AI agent can do:

Call context_set(name="user_preferences", value={"theme": "dark", "notifications": true}, metadata={"user_id": "john123"})
Call memory_push(content="User interested in premium features", tags=["user_intent", "upgrade_interest"], vectorize=true)

useContext Actions

See Context Management System documentation above for available builtin context and memory functions.

Semantic Memory System

Store contextual information with semantic tags and retrieve it later using vector search or keyword matching.

Storing Information

{
  "action": "builtin::memory_push",
  "parameters": {
    "content": "Customer expressed strong interest in eco-friendly packaging options during the consultation call. Mentioned sustainability is a key factor in their purchasing decisions.",
    "tags": ["customer_preferences", "sustainability", "packaging"],
    "vectorize": true,
    "metadata": {
      "customer_id": "CUST_123",
      "call_date": "2024-01-15",
      "confidence": "high"
    }
  }
}

Parameters:

content (required): Text or stringified data to store
tags (required): Array of tags for categorization and filtering
vectorize (optional): Enable vector search capabilities (default: false)
metadata (optional): Additional metadata for the memory entry

Returns:

{
  "success": true,
  "tags": ["customer_preferences", "sustainability", "packaging"],
  "vectorized": true
}

Querying Memory

{
  "action": "builtin::memory_query",
  "parameters": {
    "query": "What packaging preferences has this customer mentioned?",
    "tags": ["customer_preferences"],
    "top_k": 3
  },
  "output": "relevant_memories"
}

Parameters:

query (required): Search query for finding relevant memories
tags (optional): Filter by specific tags
top_k (optional): Maximum number of results to return (default: 3)

Returns:

{
  "results": [
    {
      "content": "Customer expressed strong interest in eco-friendly packaging...",
      "tags": ["customer_preferences", "sustainability", "packaging"],
      "timestamp": "2024-01-15T10:00:00Z",
      "match_score": 6,
      "metadata": {
        "customer_id": "CUST_123",
        "call_date": "2024-01-15",
        "confidence": "high"
      }
    }
  ],
  "query": "What packaging preferences has this customer mentioned?",
  "count": 1
}

Template Engine Integration

The template engine has been enhanced to support direct context references:

Context Reference Patterns

// Direct access
{{context.task_objective}}

// Nested field access
{{context.user_profile.name}}
{{context.user_profile.subscription_tier}}
{{context.session_info.session_id}}

// All named contexts
{{context.named_contexts}}

Verified Working Examples

From the working Context Manager test workflow:

{
  "action": "builtin::log",
  "parameters": {
    "message": "Session: {{context.session_info.session_id}}, User: {{context.user_profile.name}}, Status: {{context.session_completion.status}}"
  }
}

Result: "Session: demo_session_001, User: John Doe, Status: completed"

Table of Contents​

Token Counting & Prediction System​

Token Tracking​

Token Prediction Mode (for now only OpenAI)​

Workflow Analytics​

Context Management System​

Core Capabilities​

Named Context Blocks​

Setting Context Values​

Retrieving Context Values​

Using Context in Templates​

Dynamic Context Assembly​

Basic Context Assembly​

Using Assembled Context​

Storage Tools (useStorage)​

Single Database Analysis Example​

Multi-Addon Tool Integration​

useStorage Actions​

Context Tools (useContext)​

Context Operations​

Basic Context Management Example​

useContext Actions​

Semantic Memory System​

Storing Information​

Querying Memory​

Template Engine Integration​

Context Reference Patterns​

Verified Working Examples​

Table of Contents

Token Counting & Prediction System

Token Tracking

Token Prediction Mode (for now only OpenAI)

Workflow Analytics

Context Management System

Core Capabilities

Named Context Blocks

Setting Context Values

Retrieving Context Values

Using Context in Templates

Dynamic Context Assembly

Basic Context Assembly

Using Assembled Context

Storage Tools (`useStorage`)

Single Database Analysis Example

Multi-Addon Tool Integration

useStorage Actions

Context Tools (`useContext`)

Context Operations

Basic Context Management Example

useContext Actions

Semantic Memory System

Storing Information

Querying Memory

Template Engine Integration

Context Reference Patterns

Verified Working Examples