Get Warned Before Your LLM Balance Runs Out: Alert Levels, Email and Signed Webhooks

A prepaid balance has one sharp edge: when it reaches zero, every request fails with 403 until someone tops up. For a side project that is a mild annoyance. For a support bot or a nightly batch job it is an outage that nobody notices until a customer does.

Relaykit can warn you before that happens. This tutorial covers how the alert decides when to fire, how to pick a level from your real spend, and how to receive the alert by email or by a signed webhook, with a small receiver in Node and in Python that you can test on your own machine with curl.

How the alert fires

The gateway checks your balance after every billed request. If the balance left after that request is below your alert level, it sends an alert. While the balance stays below the level, alerts repeat, at most two in any ten minutes.

Two details matter when you size the level:

• The alert follows the request. A request that has already started always finishes, so a long reply can take the balance slightly below $0. The alert is an early warning, not a hard stop.
• Spend limits on keys are separate. A spend limit on a key stops that one key; the low-balance alert watches the whole account balance that every key draws from. Use both.

Pick a level from your daily spend

A useful level is “what I spend in a day” multiplied by “how many days it takes me to notice and top up”. Leave room for weekends and holidays.

Take a support assistant on Claude Haiku 4.5 ($1.10 input / $5.50 output per 1M tokens) that answers 2,000 questions a day, with about 1,500 input and 400 output tokens each:

With 3 days of lead time, $7.70 × 3 = $23.10, so an alert level of $25.00 (rounded up to the next $5) gives you about 3 days of normal traffic after the first alert. Online checkout is not open yet: today a credit pack is bought by email from the billing page, so leave more lead time rather than less.

Your own numbers are in the console: the Usage page charts your spend per day, and the cost calculator turns a workload into a monthly figure if you are planning ahead.

Email or webhook

On the Settings page, under Low-balance alerts, you choose how alerts are sent:

• Email goes to the address on your account, never to another one. If your account has an email address, email alerts are on by default, even before you save anything, but then they only start when the balance is nearly used up. Save a level to be warned earlier.
• Webhook posts a JSON body to a URL you own. Use it when the warning should reach a chat channel, a pager or a ticket queue rather than one inbox, or when your account has no email address. The URL must be public HTTPS: the gateway does not send alerts to localhost or to private-network addresses.

Enter the level in Alert when the balance is below (USD), add the Webhook URL and a Signing secret if you chose a webhook, and choose Save alerts.

What the webhook sends

Each alert is a POST with a JSON body like this:

{
  "type": "quota_exceed",
  "title": "…",
  "content": "…",
  "values": ["…", "$0.437900", "…"],
  "timestamp": 1791555600
}

• type is quota_exceed for a low-balance alert. Decide what to do from this field.
• values[1] is your balance after the request, as text with a currency sign and six decimals. The sign is not always the plain ASCII $, so read the number and ignore the symbol.
• title, content and the other values are human-readable text. Their wording can change, so do not parse them.
• timestamp is when the alert was sent, in Unix seconds.

If you saved a signing secret, the request carries an X-Webhook-Signature header: the hex-encoded HMAC-SHA256 of the raw request body, keyed with your secret. Compute it over the bytes you received, before any JSON parsing, and compare in constant time. Any 2xx response counts as delivered.

A receiver in Node

This receiver has no dependencies. It checks the signature, answers 204 at once, ignores alerts older than five minutes, and tells a person at most once an hour, because the gateway keeps repeating the alert while the balance stays low. Set CHAT_WEBHOOK_URL to a Slack-style incoming webhook (it receives {"text": "..."}) or leave it unset to print to the log.

// alerts.mjs  (Node 18+, no dependencies)  run: ALERT_SECRET=... node alerts.mjs
import http from "node:http";
import { createHmac, timingSafeEqual } from "node:crypto";

const SECRET = process.env.ALERT_SECRET; // the Signing secret you saved in the console
const CHAT_URL = process.env.CHAT_WEBHOOK_URL; // optional: a Slack-style incoming webhook
const QUIET_MS = 60 * 60 * 1000; // tell a person at most once an hour
let lastNotified = 0;

function isValid(raw, signature) {
  const expected = Buffer.from(createHmac("sha256", SECRET).update(raw).digest("hex"));
  const given = Buffer.from(String(signature || ""));
  return expected.length === given.length && timingSafeEqual(expected, given);
}

// values[1] is the balance as text, such as "$0.437900". Read the number, not the sign.
function balanceOf(alert) {
  const match = /-?\d+(?:\.\d+)?/.exec(String((alert.values || [])[1] || ""));
  return match ? Number(match[0]) : null;
}

async function notify(text) {
  if (!CHAT_URL) return console.log(text);
  await fetch(CHAT_URL, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ text }),
  });
}

function handle(raw) {
  let alert;
  try {
    alert = JSON.parse(raw.toString("utf8"));
  } catch {
    return;
  }
  if (alert.type !== "quota_exceed") return;
  if (Math.abs(Date.now() / 1000 - Number(alert.timestamp)) > 300) return; // old or replayed
  if (Date.now() - lastNotified < QUIET_MS) return;
  lastNotified = Date.now();
  const left = balanceOf(alert);
  const amount = left === null ? "an unknown amount" : "$" + left.toFixed(2);
  notify("LLM balance is low: " + amount + " left. Top up before requests fail.").catch(console.error);
}

http
  .createServer((req, res) => {
    if (req.method !== "POST" || req.url !== "/hooks/llm-balance") return res.writeHead(404).end();
    const chunks = [];
    let size = 0;
    req.on("data", (chunk) => {
      size += chunk.length;
      if (size > 65536) req.destroy();
      else chunks.push(chunk);
    });
    req.on("end", () => {
      const raw = Buffer.concat(chunks);
      if (!isValid(raw, req.headers["x-webhook-signature"])) return res.writeHead(401).end();
      res.writeHead(204).end(); // any 2xx counts as delivered: answer first, then act
      handle(raw);
    });
  })
  .listen(Number(process.env.PORT || 8080));

The same receiver in Python

Standard library only, same behaviour:

# alerts.py  (Python 3.10+, standard library only)  run: ALERT_SECRET=... python alerts.py
import hashlib
import hmac
import json
import os
import re
import time
import urllib.request
from http.server import BaseHTTPRequestHandler, HTTPServer

SECRET = os.environ["ALERT_SECRET"].encode()  # the Signing secret you saved in the console
CHAT_URL = os.environ.get("CHAT_WEBHOOK_URL")  # optional: a Slack-style incoming webhook
QUIET_SECONDS = 60 * 60  # tell a person at most once an hour
last_notified = 0.0


def is_valid(raw: bytes, signature: str | None) -> bool:
    expected = hmac.new(SECRET, raw, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature or "")


def balance_of(alert: dict) -> float | None:
    # values[1] is the balance as text, such as "$0.437900". Read the number, not the sign.
    values = alert.get("values") or []
    match = re.search(r"-?\d+(?:\.\d+)?", str(values[1]) if len(values) > 1 else "")
    return float(match.group()) if match else None


def notify(text: str) -> None:
    if not CHAT_URL:
        print(text, flush=True)
        return
    body = json.dumps({"text": text}).encode()
    request = urllib.request.Request(CHAT_URL, data=body, headers={"Content-Type": "application/json"})
    urllib.request.urlopen(request, timeout=10)


def handle(raw: bytes) -> None:
    global last_notified
    try:
        alert = json.loads(raw)
    except ValueError:
        return
    if alert.get("type") != "quota_exceed":
        return
    if abs(time.time() - float(alert.get("timestamp") or 0)) > 300:  # old or replayed
        return
    if time.time() - last_notified < QUIET_SECONDS:
        return
    last_notified = time.time()
    left = balance_of(alert)
    amount = "an unknown amount" if left is None else "$%.2f" % left
    notify("LLM balance is low: %s left. Top up before requests fail." % amount)


class AlertHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        if self.path != "/hooks/llm-balance":
            return self.send_error(404)
        length = int(self.headers.get("Content-Length") or 0)
        if length > 65536:
            return self.send_error(413)
        raw = self.rfile.read(length)
        if not is_valid(raw, self.headers.get("X-Webhook-Signature")):
            return self.send_error(401)
        self.send_response(204)  # any 2xx counts as delivered: answer first, then act
        self.end_headers()
        handle(raw)


HTTPServer(("0.0.0.0", int(os.environ.get("PORT", "8080"))), AlertHandler).serve_forever()

Both keep their once-an-hour memory in the process, so a restart can send one extra message. If you run several copies behind a load balancer, keep that timestamp in your database or cache instead.

Test it before you rely on it

The console has no “send a test alert” button, so sign a sample alert yourself. This script builds the body with the current time, signs it the way the gateway does, and posts it:

# send-test.sh: sign a sample alert the way the gateway does and post it to your receiver.
# Usage: ALERT_SECRET=... sh send-test.sh [url]
URL="http://localhost:8080/hooks/llm-balance"
[ -n "$1" ] && URL="$1"
printf '{"type":"quota_exceed","title":"","content":"","values":["","$0.437900"],"timestamp":%s}' "$(date +%s)" > alert.json
SIG=$(openssl dgst -sha256 -hmac "$ALERT_SECRET" alert.json | sed 's/^.*= //')
curl -sS -o /dev/null -w "%{http_code}\n" "$URL" \
  -H "Content-Type: application/json" \
  -H "X-Webhook-Signature: $SIG" \
  --data-binary @alert.json

With the receiver running and the same ALERT_SECRET in both shells, you should see 204 and one message. Run the script again and you get 204 but no second message, because of the one-hour quiet period. Run it with a different secret and you get 401.

The body goes through a file and --data-binary on purpose: the receiver must see exactly the bytes that were signed, and some shells re-encode non-ASCII characters passed on the command line.

To receive real alerts, put the receiver behind HTTPS on a public host (your existing app server, a small VM, or a tunnel to your machine while you try it out), then save that URL in the console.

Checklist

• Alert level = daily spend × days you need to notice and top up, rounded up.
• Email if one inbox is enough; a webhook if the warning belongs in a shared channel.
• Set a signing secret, verify X-Webhook-Signature over the raw body, and reject anything else with 401.
• Act on type, read the balance as a number from values[1], and ignore the wording of title and content.
• Rate-limit what reaches people: the gateway repeats the alert while the balance stays low.
• Test with a signed sample before the real thing, and keep a spend limit on each key as the second line of defence.

The account security and alerts page in the docs lists every alert field and the other settings that protect your balance.