gRPC API
gRPC service definitions and streaming protocols.
Valka's gRPC API runs on port 50051 by default. Proto files are at proto/valka/v1/.
Every node serves ApiService and WorkerService. In a cluster, a call about a task is
forwarded to the node that owns the task's shard, and a worker connected to any node can run
tasks owned by any node.
Status Codes
| Code | When |
|---|---|
NOT_FOUND | Unknown task |
INVALID_ARGUMENT | Missing or invalid field |
ALREADY_EXISTS | The idempotency key is already used |
FAILED_PRECONDITION | The task is in the wrong state for the call (e.g. cancelling a finished task) |
UNAVAILABLE | The shard is moving, its owner is unreachable, or the bucket is failing. Retry |
INTERNAL | Server error |
ApiService
The main client-facing API for task management.
service ApiService {
rpc CreateTask(CreateTaskRequest) returns (CreateTaskResponse);
rpc GetTask(GetTaskRequest) returns (GetTaskResponse);
rpc ListTasks(ListTasksRequest) returns (ListTasksResponse);
rpc CancelTask(CancelTaskRequest) returns (CancelTaskResponse);
rpc SendSignal(SendSignalRequest) returns (SendSignalResponse);
rpc SubscribeEvents(SubscribeEventsRequest) returns (stream TaskEvent);
rpc SubscribeLogs(SubscribeLogsRequest) returns (stream LogEntry);
}CreateTask
Create a new task in a queue.
| Field | Type | Description |
|---|---|---|
queue_name | string | Target queue |
task_name | string | Task identifier |
input | string (JSON) | Task payload |
priority | int32 | Priority (higher = first) |
max_retries | int32 | Max retry attempts |
timeout_seconds | int32 | Lease timeout |
idempotency_key | string | Dedup key |
metadata | string (JSON) | Arbitrary metadata |
scheduled_at | Timestamp | Delayed execution |
SubscribeEvents
Server-streaming RPC. Returns a stream of TaskEvent messages for real-time monitoring.
message TaskEvent {
string event_id = 1;
string task_id = 2;
string queue_name = 3;
TaskStatus new_status = 4;
int64 timestamp_ms = 5;
}SubscribeLogs
Server-streaming RPC. Returns a stream of LogEntry messages for a specific task run.
WorkerService
The bidirectional streaming service used by workers.
service WorkerService {
rpc Session(stream WorkerRequest) returns (stream WorkerResponse);
// Persist the result of a completed step of a running task. Returns once durable.
rpc Checkpoint(CheckpointRequest) returns (CheckpointResponse);
}A single Session stream carries all communication between a worker and the server.
Checkpoint is a separate unary call so a handler can wait for its step to be durable.
Worker → Server Messages
graph LR
subgraph WorkerRequest
WH[WorkerHello<br/>Initial handshake]
TR[TaskResult<br/>Success or failure]
HB[Heartbeat<br/>Lease extension]
LB[LogBatch<br/>Log entries]
GS[GracefulShutdown<br/>Drain signal]
SA[SignalAck<br/>Signal confirmed]
end
| Message | When Sent | Description |
|---|---|---|
WorkerHello | On connect | Worker name, queues, concurrency, and the runs it is still executing (running), so they are reattached after a reconnect |
TaskResult | Task done | Success/failure with output/error; resent until a ResultAck says APPLIED or STALE |
Heartbeat | Every 10s | IDs of tasks running or awaiting a result ack, for lease extension |
LogBatch | During task | Structured log entries |
GracefulShutdown | Shutting down | Stop sending tasks; the stream stays open until in-flight results are acknowledged |
SignalAck | Signal received | Confirms signal delivery |
Server → Worker Messages
| Message | When Sent | Description |
|---|---|---|
TaskAssignment | Task matched | New task to execute, with checkpoints from earlier attempts |
TaskCancellation | Cancel request | Cancel a running task. Also sent for a run in the hello that has already ended |
HeartbeatAck | After heartbeat | Confirms heartbeat received |
ServerShutdown | Server stopping | The stream is about to close; the worker reconnects with backoff |
TaskSignal | Signal sent | Real-time signal for a task |
ResultAck | After a TaskResult | APPLIED, STALE or RETRY (see below) |
Result acknowledgement
Every TaskResult is answered with a ResultAck:
| Status | Meaning | Worker action |
|---|---|---|
RESULT_STATUS_APPLIED | The result is durable, now or from an earlier delivery of the same result | Forget it |
RESULT_STATUS_STALE | The run already ended another way (lease expired, cancelled, deleted); the result was not recorded | Forget it |
RESULT_STATUS_RETRY | Not recorded yet (storage trouble) | Send it again later |
Results are idempotent: sending the same result twice is safe. Every SDK keeps a result until it is APPLIED or STALE, resends it after a reconnect, and keeps heartbeating the task meanwhile, so a handler that finishes while the server is restarting still has its result recorded, without the task running again.
Step checkpoints
message CheckpointRequest {
string task_id = 1;
string task_run_id = 2;
string step = 3;
string output = 4; // JSON
}Only the task's current running run may checkpoint. The call returns once the checkpoint
is durable. Every later TaskAssignment for the task carries its checkpoints
(repeated StepCheckpoint checkpoints), which the SDKs' step helpers use to skip steps
an earlier attempt completed. Limits: step names up to 256 bytes, outputs up to 256 KiB of
JSON, at most 256 distinct steps per task.
Reconnect and reattach
WorkerHello.running lists the runs a reconnecting worker is still executing:
message RunningTask {
string task_id = 1;
string task_run_id = 2;
int32 attempt_number = 3;
}For each one, the task's owner reattaches the run to the new connection, adopts it if its
dispatch record was lost in a crash, or answers with a TaskCancellation if the run is stale.
The worker also resends every result not yet acknowledged.
InternalService
Node-to-node calls in a cluster, on the internal listener (cluster.internal_addr, default
:50052). Every call carries the cluster token (authorization: Bearer <token>), optionally
over mTLS. This service is not for clients; never expose its port publicly.
service InternalService {
// Run an id-routed command on the node that owns its shard.
rpc Execute(ExecuteRequest) returns (ExecuteResponse);
// Answer a cluster-wide query from this node's shards only (list, count, stats).
rpc Scatter(ScatterRequest) returns (ScatterResponse);
// This node's task events, for nodes whose subscribers want the whole cluster's stream.
rpc Events(EventsRequest) returns (stream TaskEvent);
// Health and identity.
rpc Ping(PingRequest) returns (PingResponse);
}A node that does not own the shard of an Execute command answers NOT_OWNER with a hint of
the owner it knows; the caller retries there.
Common Types
TaskStatus
enum TaskStatus {
TASK_STATUS_UNSPECIFIED = 0;
TASK_STATUS_PENDING = 1;
TASK_STATUS_DISPATCHING = 2;
TASK_STATUS_RUNNING = 3;
TASK_STATUS_COMPLETED = 4;
TASK_STATUS_FAILED = 5;
TASK_STATUS_RETRY = 6;
TASK_STATUS_DEAD_LETTER = 7;
TASK_STATUS_CANCELLED = 8;
}TASK_STATUS_DISPATCHING is kept for compatibility but never used: a dispatched task is
RUNNING.
LogLevel
enum LogLevel {
LOG_LEVEL_UNSPECIFIED = 0;
LOG_LEVEL_DEBUG = 1;
LOG_LEVEL_INFO = 2;
LOG_LEVEL_WARN = 3;
LOG_LEVEL_ERROR = 4;
}Using with grpcurl
# Create a task
grpcurl -plaintext -d '{
"queue_name": "emails",
"task_name": "send-email",
"input": "{\"to\": \"user@example.com\"}"
}' localhost:50051 valka.v1.ApiService/CreateTask
# Subscribe to events
grpcurl -plaintext localhost:50051 valka.v1.ApiService/SubscribeEvents