The Task Engine is the core component of the EigenLayer-AVS system responsible for managing, compiling, and executing tasks.
The Task Engine is the core component of the EigenLayer-AVS system responsible for managing, compiling, and executing tasks. It provides a flexible execution environment that processes tasks as directed acyclic graphs (DAGs) of operations, allowing complex workflows to be defined and executed in a deterministic manner.
This document covers the architecture, components, and functionality of the Task Engine, including:
For information about task data structures and modeling, see Task Model. For details on individual processors, see VM Processors.
The Task Engine consists of two main components: the Engine and the Virtual Machine (VM). The Engine manages task coordination, scheduling, and lifecycle, while the VM handles the actual compilation and execution of tasks.
Sources:
The Engine component is responsible for:
The Engine exposes methods for creating, triggering, and monitoring tasks, as well as managing secrets that can be used within tasks.
Sources:
The VM is responsible for:
The VM has a lifecycle with distinct states (initialize, compiled, ready, executing, completed) and maintains execution context through the entire task run.
Sources:
Task execution follows a well-defined flow from creation to completion. The Engine coordinates with operators for task triggers, while the VM handles the actual execution.
Sources:
Tasks and VM executions have well-defined states that track progress through the lifecycle:
| Entity | State | Description |
|---|---|---|
| Task | Active | Task is active and can be triggered |
| Task | Executing | Task is currently executing |
| Task | Completed | Task has completed all executions |
| Task | Failed | Task has failed |
| Task | Canceled | Task has been canceled |
| VM | VMStateInitialize | VM is initializing |
| VM | VMStateCompiled | Task has been compiled into execution plan |
| VM | VMStateReady | VM is ready for execution |
| VM | VMStateExecuting | VM is currently executing |
| VM | VMStateCompleted | VM has completed execution |
Sources:
The VM compiles tasks into execution plans by analyzing the edges between nodes and identifying the entry point. The compilation process:
Sources:
The VM executes tasks node by node, following the execution plan created during compilation:
The VM supports various node types through specialized processors:
| Node Type | Processor | Description |
|---|---|---|
RestApi | RestProcessor | HTTP API calls |
ContractRead | ContractReadProcessor | Smart contract read operations |
ContractWrite | ContractWriteProcessor | Smart contract write operations |
GraphQLQuery | GraphQLQueryProcessor | GraphQL API queries |
CustomCode | JSProcessor | Custom JavaScript code execution |
Branch | BranchProcessor | Conditional branching logic |
Sources:
The VM includes a template processing system that allows for dynamic content in task definitions. Templates use {{ expression }} syntax and can reference:
The preprocessText method evaluates these expressions using a JavaScript engine:
Sources:
Tasks are defined as directed acyclic graphs (DAGs) with nodes representing operations and edges representing execution flow:
Tasks can be triggered manually or automatically via events, blocks, or time-based triggers.
Sources:
Outputs from each node are stored in the VM's variable map and can be referenced by subsequent nodes:
This allows for data passing between nodes in the execution flow.
Sources:
The Task Engine uses BadgerDB for persistent storage of:
The storage layer is accessed through the db field in both Engine and VM.
Sources:
The Task Engine interacts with operators through:
StreamCheckToOperator: Streams task definitions to operatorsAggregateChecksResult: Processes trigger notifications from operatorsOperators monitor for trigger conditions and notify the Engine when conditions are met.
Sources:
The Task Engine uses a queue system (apqueue) for asynchronous task execution:
Sources:
CreateTask(user *model.User, taskPayload *avsproto.CreateTaskReq) (*model.Task, error)
Creates a new task with the specified nodes, edges, and trigger.
Sources:
TriggerTask(user *model.User, payload *avsproto.UserTriggerTaskReq) (*avsproto.UserTriggerTaskResp, error)
Manually triggers a task execution. Can be run in blocking or non-blocking mode.
Sources:
ListTasksByUser(user *model.User, payload *avsproto.ListTasksReq) (*avsproto.ListTasksResp, error)
Lists tasks belonging to a user, with pagination support.
Sources:
ListExecutions(user *model.User, payload *avsproto.ListExecutionsReq) (*avsproto.ListExecutionsResp, error)
Lists executions for a given task or set of tasks, with pagination support.
Sources:
The Task Engine provides APIs for managing secrets that can be used in tasks:
CreateSecret(user *model.User, payload *avsproto.CreateOrUpdateSecretReq) (bool, error)
UpdateSecret(user *model.User, payload *avsproto.CreateOrUpdateSecretReq) (bool, error)
ListSecrets(user *model.User, payload *avsproto.ListSecretsReq) (*avsproto.ListSecretsResp, error)
DeleteSecret(user *model.User, payload *avsproto.DeleteSecretReq) (bool, error)
Secrets are stored securely and can be referenced in tasks using {{ apContext.configVars.secretName }}.
Sources:
The Task Engine integrates with smart wallets for:
Smart wallet configurations are passed to the VM for use in contract interactions.
Sources:
The Task Engine provides a powerful, flexible system for defining and executing complex workflows. It combines the management capabilities of the Engine with the execution capabilities of the VM to create a comprehensive task execution environment. The various processors support a wide range of operations, from HTTP calls to smart contract interactions, allowing for sophisticated automations and integrations.
Feature this wiki to auto refresh weekly