valka
SDKs

Go SDK

Build workers and manage tasks with the Go SDK.

Installation

go get github.com/IWhitebird/Valka/sdks/go@v0.2.0

The Go SDK provides:

  • Worker: process tasks from queues with functional options
  • Client: create, get, list, and cancel tasks

Worker

package main

import (
    "context"
    "fmt"
    "log"
    "time"

    valka "github.com/IWhitebird/Valka/sdks/go"
)

func main() {
    worker, err := valka.NewWorker(
        valka.WithName("email-worker"),
        valka.WithServerAddr("localhost:50051"),
        valka.WithQueues("emails", "notifications"),
        valka.WithConcurrency(4),
        valka.WithHandler(handleTask),
    )
    if err != nil {
        log.Fatalf("Failed to create worker: %v", err)
    }

    log.Println("Starting worker...")
    if err := worker.Run(context.Background()); err != nil {
        log.Fatalf("Worker error: %v", err)
    }
}

func handleTask(ctx *valka.TaskContext) (interface{}, error) {
    var input map[string]interface{}
    if err := ctx.Input(&input); err != nil {
        return nil, err
    }

    to := "unknown"
    if v, ok := input["to"].(string); ok {
        to = v
    }

    ctx.Log(fmt.Sprintf("Processing email task: %s", ctx.TaskName))
    ctx.Debug(fmt.Sprintf("Attempt #%d", ctx.AttemptNumber))

    time.Sleep(1 * time.Second)

    ctx.Log(fmt.Sprintf("Email sent to %s", to))
    return map[string]interface{}{
        "delivered_to": to,
        "status":       "sent",
    }, nil
}

Functional Options

OptionDescription
WithName(name)Worker name
WithServerAddr(addr)Valka server gRPC address
WithServerAddrs(addrs...)Several nodes; the worker moves to the next when its connection drops
WithAPIKey(key)API key (default: VALKA_API_KEY)
WithQueues(queues...)Queues to listen on
WithConcurrency(n)Max concurrent tasks
WithHandler(fn)Function to process tasks

Task Context

func handleTask(ctx *valka.TaskContext) (interface{}, error) {
    // Access metadata
    name := ctx.TaskName
    attempt := ctx.AttemptNumber
    taskID := ctx.TaskID

    // Parse input into struct
    var input MyInput
    if err := ctx.Input(&input); err != nil {
        return nil, err
    }

    // Logging
    ctx.Log("Processing...")
    ctx.Debug("Debug info")

    // Return output (any JSON-serializable value)
    return map[string]interface{}{"result": "done"}, nil
}

Step Checkpoints

valka.Step(ctx, name, fn) runs fn once per task: its result is checkpointed, and a retry of the task returns the recorded result instead of running fn again.

func handleTask(ctx *valka.TaskContext) (interface{}, error) {
    rows, err := valka.Step(ctx, "fetch", func() ([]Row, error) { return fetchRows(ctx) })
    if err != nil {
        return nil, err
    }
    return valka.Step(ctx, "upload", func() (string, error) { return upload(ctx, rows) })
}

Step results must round-trip through encoding/json. ctx.Checkpoint(step, value) and ctx.CheckpointValue(step, &dest) record and read checkpoints directly.

Client

The client uses the REST API (port 8989).

package main

import (
    "fmt"
    "log"

    valka "github.com/IWhitebird/Valka/sdks/go"
)

func main() {
    client := valka.NewClient("http://valka-1:8989",
        valka.WithBaseURLs("http://valka-2:8989", "http://valka-3:8989"), // optional: more nodes
        valka.WithClientAPIKey("..."),                                    // or VALKA_API_KEY
    )

    retries := 3
    task, err := client.CreateTask(valka.CreateTaskRequest{
        QueueName:  "emails",
        TaskName:   "send-email",
        Input:      map[string]interface{}{"to": "user@example.com"},
        MaxRetries: &retries,
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("created", task.ID)

    task, err = client.GetTask(task.ID)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("status", task.Status)
}

Several nodes, API keys and retries

  • Give the client every node's URL. A call that fails because a node is unreachable or busy (connection errors, 429, 502, 503, 504) is retried with backoff on the next node, up to WithAttempts(n) times (default 6).
  • Every create carries an idempotency key: yours, or a generated auto-… one. A retried create therefore returns the task the first attempt made instead of making a second. WithAutoIdempotency(false) turns this off.
  • Cancels and signals are sent once, since repeating them is not safe.
  • WithClientAPIKey (default: the VALKA_API_KEY environment variable) is sent as Authorization: Bearer <key>.

Signal Handling

worker, _ := valka.NewWorker(
    valka.WithName("worker"),
    valka.WithServerAddr("localhost:50051"),
    valka.WithQueues("emails"),
    valka.WithConcurrency(4),
    valka.WithHandler(handleTask),
    valka.WithSignalHandler(func(signal *valka.Signal) {
        fmt.Printf("Signal received: %s\n", signal.Type)
    }),
)

Reconnects

The worker survives server restarts and node failures without losing work:

  • When its stream breaks (or the server sends ServerShutdown), the worker reconnects with exponential backoff, from 100 ms up to 30 s, to its node (the next one in its list when it has several). Running handlers keep going meanwhile.
  • On the new connection it resends every result not yet acknowledged (APPLIED or STALE), and keeps heartbeating those tasks until then. The Go SDK also resends them on every heartbeat tick (10 s).
  • Its hello lists the runs it is still executing. The task's owner reattaches each one to the new connection instead of running the task again, or records a dispatch that was lost in a crash. A run that ended meanwhile (cancelled, expired) gets a cancellation.

In a cluster, give the worker every node with WithServerAddrs(...) (it moves to the next on each reconnect), or point it at a load balancer or DNS name that covers several nodes, so it lands on a live node when its own dies. See Clustering.

On this page