Building a High-Throughput UPI Payment Queuing System Using BullMQ and Docker

During a nationwide festival sale, flash launch, or ticket drop, an e-commerce platform can experience surges from 50 orders a minute to over 5,000 payment events per second.

If your backend attempts to verify HMAC signatures, query inventory, acquire database row locks, and dispatch confirmation emails synchronously inside the webhook request thread, your database connection pool will exhaust in seconds, returning HTTP 504 Gateway Timeouts to the acquiring bank.

Here is the enterprise architecture to scale payment event handling using BullMQ, Redis, and Docker.

---

The Flash Sale Bottleneck Problem

Direct Answer: Never execute business fulfillment logic inside the synchronous webhook handler. High-throughput architectures follow the Producer-Consumer pattern: the HTTP webhook handler simply validates the HMAC signature, enqueues the job into Redis via BullMQ in under 2ms, and immediately returns HTTP 200 OK, delegating heavy database writes and external APIs to isolated background workers.

Synchronous Failure Pattern (Crashes at 200 TPS):
Webhook Hits ──► Verify HMAC ──► DB Lock ──► Email API ──► Shipping API (Takes 2500ms)
                                                                 ▲
Connections Exhaust ─────────────────────────────────────────────┘

Asynchronous Queue Pattern (Scales to 10,000+ TPS):
Webhook Hits ──► Verify HMAC ──► Push to BullMQ (2ms) ──► Return 200 OK!
                                       │
                                       ▼ (Redis Buffer)
                      [ Autoscaled Worker Pool ]
                      ├── Worker 1: Updates DB
                      ├── Worker 2: Sends Email
                      └── Worker 3: Alerts Warehouse

---

System Architecture: Redis + BullMQ + Docker

1. Ingestion API (Producer): A lightweight containerized Express or FastAPI server dedicated solely to signature verification and job queuing.
2. Redis In-Memory Broker: Acts as an ultra-fast shock absorber buffering incoming events.
3. Worker Pool (Consumer): Scalable headless Node.js workers that consume jobs with controlled concurrency, exponential backoff, and dead-letter queue (DLQ) safeguards.

---

The Producer: Lightning-Fast Webhook Ingestion

Create producer.js:

import express from 'express';
import { Queue } from 'bullmq';
import Redis from 'ioredis';

const connection = new Redis(process.env.REDIS_URL || 'redis://redis:6379');
const paymentQueue = new Queue('payment_fulfillment', { connection });

const app = express();
app.use(express.json());

app.post('/api/v1/webhook', async (req, res) => {
  // 1. Enqueue job immediately with unique Job ID to prevent duplicate queues
  const { client_txn_id, status, amount, utr } = req.body;

  await paymentQueue.add(
    'process_payment',
    { client_txn_id, status, amount, utr, raw: req.body },
    {
      jobId: `pay_${client_txn_id}_${utr}`, // Deduplication key
      attempts: 5,
      backoff: {
        type: 'exponential',
        delay: 2000, // 2s, 4s, 8s, 16s...
      },
      removeOnComplete: true,
      removeOnFail: false, // Retain failed jobs in DLQ for audit
    }
  );

  // 2. Return 200 OK instantly (< 3ms response time)
  return res.status(200).json({ status: 'queued', txnId: client_txn_id });
});

app.listen(3000, () => console.log('Producer listening on port 3000'));

---

The Consumer: Resilient Worker Service with Backoff

Create worker.js:

import { Worker } from 'bullmq';
import Redis from 'ioredis';

const connection = new Redis(process.env.REDIS_URL || 'redis://redis:6379');

const worker = new Worker(
  'payment_fulfillment',
  async (job) => {
    const { client_txn_id, status, amount, utr } = job.data;
    console.log(`[Job ${job.id}] Processing Order ${client_txn_id}...`);

    if (status === 'SUCCESS') {
      // 1. Atomic Database Transaction (e.g. Postgres / MySQL)
      await updateOrderStatus(client_txn_id, utr, amount);

      // 2. External Notifications
      await sendCustomerAlerts(client_txn_id);
    }

    return { processed: true, client_txn_id };
  },
  {
    connection,
    concurrency: 50, // Processes 50 jobs concurrently per worker container!
    limiter: {
      max: 1000,
      duration: 1000, // Rate limits to 1,000 tasks/second to protect database
    },
  }
);

worker.on('failed', (job, err) => {
  console.error(`[Job ${job.id}] FAILED after ${job.attemptsMade} attempts:`, err.message);
});

---

Complete Docker Compose Production Setup

Save as docker-compose.payments.yml:

version: '3.8'

services:
  redis:
    image: redis:7-alpine
    container_name: payment_redis
    restart: always
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data
    command: redis-server --appendonly yes --maxmemory 1gb --maxmemory-policy noeviction

  api-producer:
    build:
      context: .
      dockerfile: Dockerfile.producer
    container_name: payment_producer
    restart: always
    environment:
      - REDIS_URL=redis://redis:6379
      - PORT=3000
    ports:
      - "3000:3000"
    depends_on:
      - redis

  worker:
    build:
      context: .
      dockerfile: Dockerfile.worker
    restart: always
    environment:
      - REDIS_URL=redis://redis:6379
    deploy:
      replicas: 4 # Scales to 4 worker containers (200 concurrent tasks)
    depends_on:
      - redis

volumes:
  redis_data:

With this architecture, your payment infrastructure effortlessly absorbs tens of thousands of simultaneous UPI payments with zero downtime.

Integrate with VyaparGateway’s Direct-to-Bank API to run high-throughput e-commerce with 0% platform fees.