valka
SDKs

Rust SDK

Build workers and manage tasks with the Rust SDK.

Installation

cargo add valka-sdk --git https://github.com/IWhitebird/Valka --tag v0.2.0

The crate is not on crates.io yet; Cargo builds it from the repository.

The Rust SDK provides two main components:

  • ValkaClient: create, get, list, and cancel tasks
  • ValkaWorker: process tasks from queues

Worker

Build a worker with the builder pattern:

use valka_sdk::{TaskContext, ValkaWorker};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let worker = ValkaWorker::builder()
        .name("email-worker")
        .server_addr("http://127.0.0.1:50051")
        .queues(&["emails"])
        .concurrency(4)
        .handler(handle_task)
        .build()
        .await?;

    worker.run().await?;
    Ok(())
}

async fn handle_task(ctx: TaskContext) -> Result<serde_json::Value, String> {
    let input: serde_json::Value = ctx
        .input()
        .map_err(|e| format!("Failed to parse input: {e}"))?;

    let to = input
        .get("to")
        .and_then(|v| v.as_str())
        .unwrap_or("unknown");

    ctx.log(&format!("Sending email to {to}...")).await;
    tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;
    ctx.log("Email sent successfully").await;

    Ok(serde_json::json!({
        "delivered_to": to,
        "status": "sent"
    }))
}

Builder Options

MethodDescription
.name(name)Worker name (used for identification)
.server_addr(addr)Valka server gRPC address
.server_addrs(&[...])Several nodes; the worker moves to the next when its connection drops
.api_key(key)API key (default: VALKA_API_KEY)
.queues(&[...])List of queues to listen on
.concurrency(n)Max concurrent tasks
.handler(fn)Async function to process tasks

Task Context

The TaskContext provides access to task metadata and utilities:

async fn handle_task(ctx: TaskContext) -> Result<serde_json::Value, String> {
    // Access task metadata
    let name = &ctx.task_name;
    let attempt = ctx.attempt_number;
    let task_id = &ctx.task_id;

    // Parse input
    let input: MyInput = ctx.input().map_err(|e| e.to_string())?;

    // Send logs
    ctx.log("Processing...").await;
    ctx.debug("Debug info").await;

    // Return output
    Ok(serde_json::json!({"result": "done"}))
}

Step Checkpoints

ctx.step(name, f) runs f once per task: its result is checkpointed, and a retry of the task returns the recorded result instead of running f again.

async fn handle_task(ctx: TaskContext) -> Result<serde_json::Value, String> {
    let rows: Vec<Row> = ctx.step("fetch", || fetch_rows()).await?;
    let report: Report = ctx.step("transform", || build_report(&rows)).await?;
    ctx.step("upload", || upload(&report)).await?;
    Ok(serde_json::json!({ "rows": rows.len() }))
}

The step's value must implement Serialize and DeserializeOwned; its error type must implement Display. ctx.checkpoint(step, &value) and ctx.checkpoint_value::<T>(step) record and read checkpoints directly.

Client

The client uses the gRPC API (port 50051).

use valka_sdk::ValkaClient;

let mut client = ValkaClient::builder()
    .addrs(&["http://valka-1:50051", "http://valka-2:50051", "http://valka-3:50051"])
    .api_key("...") // or VALKA_API_KEY
    .connect()
    .await?;

let task = client
    .create_task("emails", "send-email", Some(serde_json::json!({"to": "user@example.com"})))
    .await?;
println!("created {}", task.id);

let task = client.get_task(&task.id).await?;
println!("status {}", task.status);

create_task_request(CreateTaskRequest { .. }) takes every option of the API (priority, retries, timeout, idempotency key, metadata, schedule). ValkaClient::connect(addr) is the one-node shorthand.

Several nodes, API keys and retries

  • Give the client every node's address. A call that fails because a node is unreachable or busy (UNAVAILABLE, DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED) is retried with backoff on the next node, up to .attempts(n) times (default 6); each attempt has a .timeout(..) (default 10 s).
  • Every create carries an idempotency key: yours, or a generated auto-… one. A retried create therefore returns the task the first attempt made instead of making a second. .auto_idempotency(false) turns this off.
  • Cancels and signals are sent once, since repeating them is not safe.
  • .api_key(..) (default: the VALKA_API_KEY environment variable) is sent as authorization: Bearer <key>.

Signal Handling

Workers can receive and respond to signals:

let worker = ValkaWorker::builder()
    .name("worker")
    .server_addr("http://127.0.0.1:50051")
    .queues(&["emails"])
    .concurrency(4)
    .handler(handle_task)
    .on_signal(|signal| async move {
        println!("Received signal: {:?}", signal.signal_type);
        // Handle the signal
    })
    .build()
    .await?;

Reconnects

The worker survives server restarts and node failures without losing work:

  • When its stream breaks (or the server sends ServerShutdown), the worker reconnects with exponential backoff, from 100 ms up to 30 s, to its node (the next one in its list when it has several). Running handlers keep going meanwhile.
  • On the new connection it resends every result not yet acknowledged (APPLIED or STALE), and keeps heartbeating those tasks until then.
  • Its hello lists the runs it is still executing. The task's owner reattaches each one to the new connection instead of running the task again, or records a dispatch that was lost in a crash. A run that ended meanwhile (cancelled, expired) gets a cancellation.

In a cluster, give the worker every node with .server_addrs(&[...]) (it moves to the next on each reconnect), or point it at a load balancer or DNS name that covers several nodes, so it lands on a live node when its own dies. See Clustering.

On this page