Daily

Get alerted when Datadog usage jumps

If log or host usage climbs well above normal, you hear about it the same day.

Runs: Daily
Apps: Datadog
AI: Not needed
Access: usage_read

Send this to your assistant

Create a ClickRuntime automation: every day at 10am, check Datadog usage and post to #platform in Slack if log ingestion is more than 30% above the average of the last 7 days.

Paste it into your assistant. It tells you what it will send or change before it runs live.

Add to ChatGPT Add to Claude

First time? Adding ClickRuntime to your assistant takes about two minutes.

Why use it

Why teams use this automation

Datadog bills grow when usage grows, and usage can grow overnight. A deploy doubles the log volume, a new service adds hundreds of hosts, or a debug flag is left on. The bill arrives weeks later, and by then the cause is buried under other changes.

Each day, this fetches the usage summary for hosts and logs, compares it with the average it has stored, and alerts you when usage spikes above it. You hear about an unusual jump the next day, while the change that caused it is still easy to identify.

Platform engineers and finance-minded engineering managers use it to keep observability costs under control. Usage is read from Datadog's own usage summary and compared with an average the automation keeps, so a spike is judged against your normal pattern and not an arbitrary number.

What you get out of it

Catch a log explosion early
A service logging ten times as much costs real money every day. Next-day detection limits the overspend to one day, instead of a whole month. A service logging ten times as much costs real money each day, and next-day detection limits the overspend to a single day.
Link cost changes to deploys
With yesterday's spike in hand, you can check what shipped. The deploy history usually shows the likely cause within minutes. With yesterday's spike in hand, you can check what shipped, and the deploy history usually shows the likely cause in minutes.
Avoid bill shock
Seeing unusual usage early lets you fix it before the invoice. Finance gets fewer unpleasant surprises, and engineering keeps its credibility. Seeing unusual usage early lets you fix it before the invoice, and finance gets fewer unpleasant surprises.
Keep host counts in check
A forgotten autoscaling group inflates host usage. The comparison with the stored average highlights it as soon as it appears. A forgotten autoscaling group inflates host usage, and the comparison with the average highlights it as soon as it appears.
Learn what normal looks like
The stored average builds a baseline over time. After a few weeks, you have a good sense of normal usage and the confidence to spot oddities. The stored average builds a baseline over time, and after a few weeks you know what normal looks like and spot oddities.

How it works

What happens, step by step

Every day ClickRuntime reads yesterday's Datadog usage, such as log volume and host count.
It compares each figure with the average it has kept for recent days.
If usage is well above normal, your Slack channel gets yesterday's figure, the average and the percentage jump.

Before it runs live

Connect Datadog

What you get

RunsDaily

Get alerted when Datadog usage jumpstoday, 9:00Succeeded

Slack · #platformDatadog log ingestion yesterday: 184.2 GB, 46% above the 7-day average (126.1 GB)

Build your own

Need it to work differently?

This template is a starting point. Copy the prompt and change it, or describe a different automation from scratch, with your own conditions, more apps or an AI step. Your assistant writes it, tests it and runs it on ClickRuntime.

Your own rules
Conditions, thresholds, filters and exceptions, written the way you'd explain them to a colleague.
Any number of apps
One automation can read from one app and update several others. Over 248 apps are ready, and anything with an API works too.
AI where it helps
Summaries, classification, extraction and drafts use ClickRuntime's built-in AI, with no extra key.
On a schedule or on each event
Runs as often as every 5 minutes, or the moment an app sends an event, and keeps track of what it already handled.

Changes you can ask for

Run it at a different time, more or less often, or only on weekdays
Add an AI step, for example to summarize, sort or draft a reply
Add another app, for example also send it to Slack
Combine it with other Datadog steps into one automation

Under the hood

The code that runs this automation

This is the Python script that runs each time the automation starts. Your AI assistant writes it for you when you send the prompt, and you allow each step in your assistant. To change how the automation works, describe the change to your assistant and it updates the script for you. For every function it can use, see the script reference.

get_a_daily_usage_and_cost_check.pyexample

from datetime import date, timedelta
from clickruntime import on_trigger, credentials, http, notify

DD = "https://api.datadoghq.com"

def dd(path, **params):
    resp = http.get(f"{DD}{path}", params=params, headers={"DD-API-KEY": credentials.secret("DD_API_KEY"),
                    "DD-APPLICATION-KEY": credentials.secret("DD_APP_KEY")})
    resp.raise_for_status()
    return resp.json()

@on_trigger
def run(event):
    end = date.today()
    usage = dd("/api/v1/usage/logs", start_hr=f"{end - timedelta(days=8)}T00", end_hr=f"{end}T00")["usage"]
    daily = {}
    for hour in usage:
        day = hour["hour"][:10]
        daily[day] = daily.get(day, 0) + (hour.get("ingested_events_bytes") or 0)
    days = [daily[k] for k in sorted(daily)]
    yesterday, average = days[-1], sum(days[:-1]) / max(len(days) - 1, 1)
    if average and yesterday > average * 1.3:
        notify.slack(f"Datadog log ingestion yesterday: {yesterday / 1e9:,.1f} GB, "
                     f"{(yesterday / average - 1) * 100:.0f}% above the 7-day average ({average / 1e9:,.1f} GB)", channel="#platform")
    return {"yesterday_gb": yesterday / 1e9, "average_gb": average / 1e9}

FAQ

Common questions

How does it decide that usage spiked?

It compares today's figure to the average it has stored from earlier runs and alerts when the difference passes a threshold. You can choose that threshold, such as 25% above average. Today's figure is compared with the stored average, and an alert goes out when the difference passes a threshold such as 25%.

Which products does it check?

Hosts and logs, as set up. Datadog reports other usage types too, such as custom metrics and APM, and your AI assistant can add them if you need broader coverage. Hosts and logs are covered, and custom metrics or APM usage can be added when you need broader coverage.

What happens on the first day?

There is no average yet, so it stores the first value and sends no alert. After a few days the baseline is useful, and later spikes will be detected. The first day has no average, so it stores the first value and sends nothing, and useful alerts begin after a few days of data.

What permissions does it need?

usage_read. Nothing else in your accounts is used.

Does it need AI?

No. It runs without AI, so it doesn't use your plan's AI allowance.

How many runs does it use?

One run each time it runs (daily). Checks that find nothing to do and stop right away aren't counted.

Can I change how it works?

Yes. Describe the change to your AI assistant, for example a different schedule, channel or filter. It updates the automation, tests it and tells you what changed.

Daily