Introduction

What is Valka?

Valka is a distributed task queue built entirely in Rust with a single external dependency: PostgreSQL.

Most task queues bolt together a message broker, a database, and a cache. Every moving part is another thing to deploy, monitor, and debug at 3 AM. Valka takes a different approach: PostgreSQL is the single source of truth, and an in-memory matching engine with gRPC bidirectional streaming replaces the message broker entirely.

The result: simple to operate, easy to reason about, and fast where it matters.

Key Features

• One dependency: if you have Postgres, you can run Valka
• Zero-latency hot path: tasks are matched to waiting workers in-memory, no polling
• Polyglot SDKs: Rust, TypeScript, Go, and Python with the same builder pattern
• gRPC bidirectional streaming: single connection per worker, real-time task delivery
• Automatic retries: exponential backoff with dead letter queue
• Task signals: send real-time signals to running workers
• Observable: real-time log streaming, event feeds, Prometheus metrics, web dashboard
• Multi-node clustering: chitchat gossip + consistent hash ring
• Idempotency keys: prevent duplicate task execution
• Scheduled tasks: delayed and deferred task execution

How It Works

graph TD
  C[REST / gRPC Client] --> MS[MatchingService<br/>Partition Tree]

  MS --> HP[Hot Path<br/>In-Memory Oneshot]
  MS --> CP[Cold Path<br/>PG SKIP LOCKED]

  HP --> D[Dispatcher<br/>gRPC Bidi Stream]
  CP --> D

  D --> W1[Worker A]
  D --> W2[Worker B]
  D --> W3[Worker C]

  style MS fill:#4f46e5,stroke:#6366f1,color:#fff
  style HP fill:#059669,stroke:#10b981,color:#fff
  style CP fill:#0284c7,stroke:#38bdf8,color:#fff
  style D fill:#7c3aed,stroke:#8b5cf6,color:#fff

Hot path: When a task is created and a worker is already waiting, the task is matched in-memory via a oneshot channel and pushed over the gRPC stream instantly.

Cold path: When no worker is waiting, the task is persisted to PostgreSQL. Workers poll using SKIP LOCKED to claim tasks without contention.

SDKs

Any language can also interact via the REST API on port 8989.

Next Steps

• Quick Start: get Valka running in minutes
• Architecture: deep dive into how Valka works
• Task Lifecycle: understand task states and transitions