// Configure a queue with retry + DLQ
q := msgloop.NewQueue("orders",
msgloop.WithDelivery(ExactlyOnce),
msgloop.WithRetry(ExponentialBackoff{
Max: 5,
Initial: 500*time.Millisecond,
}),
msgloop.WithDLQ("orders-dlq"),
)Queues: The Foundation of Reliable Async Processing
A message queue decouples the producer of work from the consumer of that work, allowing each to operate at its own pace, scale independently, and fail without cascading. This simple abstraction underpins an enormous variety of software patterns: background job processing, async order fulfillment, notification delivery, data pipeline ingestion, and inter-service communication in microservices architectures. MsgLoop's queue primitives are designed to make these patterns reliable without requiring deep expertise in distributed systems.
Delivery Guarantees
Every queue in MsgLoop is configured with a delivery guarantee level. At-least-once delivery ensures that every message is delivered to at least one consumer, with the possibility of duplicate delivery if a consumer fails after processing but before acknowledging. This is the right choice for idempotent operations where duplicate processing is safe and performance matters. Exactly-once delivery uses a distributed transaction protocol to guarantee that each message is processed exactly once, at the cost of higher latency. Use exactly-once for financial transactions, inventory updates, and any operation where double-processing causes data corruption.
Consumer Groups and Competing Consumers
A consumer group is a named pool of consumers that cooperate to process messages from a queue. MsgLoop distributes messages across consumers in a group using a lease-based mechanism: each message is assigned to one consumer, which has a configurable window to acknowledge it. If the consumer fails or the lease expires, the message is reassigned to another consumer in the group. This pattern allows horizontal scaling of message processing by simply adding consumers to the group.
Dead Letter Queues
Some messages cannot be processed successfully: malformed payloads, dependent services that are unavailable, or business rule violations that the consumer cannot handle. After a configurable number of delivery attempts, MsgLoop moves unprocessable messages to a dead letter queue. The DLQ preserves the original message, delivery attempt history, and the most recent error from the consumer. Alerting on DLQ depth gives you early warning of systemic processing failures before they cause visible application failures.
Retry Policies
MsgLoop's retry policy framework lets you configure exponential backoff with jitter, maximum retry counts, and per-error-type retry eligibility. A transient network error should be retried immediately with backoff. A message that fails validation should go to the DLQ immediately without retrying, since retrying will not change the outcome. Configuring retry policies correctly prevents retry storms under load while ensuring that genuinely transient failures are handled automatically.
Visibility and Debugging
The MsgLoop message flow debugger shows the complete lifecycle of any message: when it was published, which consumer picked it up, how long processing took, whether it was acknowledged or requeued, and where it ended up. Filtering by message ID, producer, consumer group, or error type makes it possible to trace specific messages through complex multi-step workflows. The local development emulator replicates production queue behavior exactly, so debugging an async workflow on your laptop produces results that match production.