Daily

Review yesterday's closed conversations for quality

Each morning you get the five weakest conversations from yesterday, scored for tone and resolution.

Runs: Daily
Apps: Intercom
AI: Yes, uses the built-in AI allowance
Access: read conversations

Send this to your assistant

Create a ClickRuntime automation: every morning, score yesterday's closed Intercom conversations for tone and whether the problem was solved, and email me the 5 lowest with links.

Paste it into your assistant. It tells you what it will send or change before it runs live.

Add to ChatGPT Add to Claude

First time? Adding ClickRuntime to your assistant takes about two minutes.

Why use it

Why teams use this automation

A support manager with eight agents would like to read a sample of closed conversations every day to check tone, accuracy and whether the customer's problem was actually solved. In reality she reads a handful a week, usually after a complaint, which means coaching happens late and only about the worst cases. Customers who got a curt or incomplete answer and did not complain are never noticed.

Each day, ClickRuntime searches Intercom for conversations closed the day before, fetches each conversation's messages, and has built-in AI score them for tone and for whether the issue was resolved. It then emails you the lowest-scoring conversations with links, so your review time goes to those most worth reading.

Support leads and customer experience managers use it to coach agents with real examples.

What you get out of it

Coaching uses specific examples
Telling an agent that a particular reply was curt, with the conversation open, is more useful than general advice. Daily examples let feedback arrive while the conversation is still fresh.
Problems are found before complaints
Many unhappy customers never write a bad review. They just leave. Surfacing the weak conversations yourself lets you apologize, follow up or fix a recurring mistake before it costs the account.
Review effort is focused
Reading everything is impossible, and random sampling wastes time on fine conversations. Seeing the lowest scores first lets a manager spend twenty minutes on the cases that carry the most risk.
Quality is measured on every conversation
Scoring all of yesterday's closed conversations gives a view of quality across the team, not just of the ones a manager happened to open. Trends in the scores show whether training is working.
Fairer conversations with agents
When feedback is based on a consistent daily process, it feels less personal than a manager noticing one bad reply. Agents see that the same standard is applied to everyone.

How it works

What happens, step by step

Every day, ClickRuntime collects the Intercom conversations closed yesterday.
AI reads each one and scores it out of 10, with a short reason.
You get an email with the lowest-scoring conversations, their reasons and links, so you can coach the team.

Before it runs live

Connect Intercom

What you get

RunsDaily

Review yesterday's closed conversations for qualitytoday, 9:00Succeeded

Email · QA review3/10 Closed without confirming the refund went through. app.intercom.com/a/inbox/_/inbox/conversation/1811

Build your own

Need it to work differently?

This template is a starting point. Copy the prompt and change it, or describe a different automation from scratch, with your own conditions, more apps or an AI step. Your assistant writes it, tests it and runs it on ClickRuntime.

Your own rules
Conditions, thresholds, filters and exceptions, written the way you'd explain them to a colleague.
Any number of apps
One automation can read from one app and update several others. Over 248 apps are ready, and anything with an API works too.
AI where it helps
Summaries, classification, extraction and drafts use ClickRuntime's built-in AI, with no extra key.
On a schedule or on each event
Runs as often as every 5 minutes, or the moment an app sends an event, and keeps track of what it already handled.

Changes you can ask for

Run it at a different time, more or less often, or only on weekdays
Change what the AI looks for, or how it writes the result
Add another app, for example also send it to Slack
Combine it with other Intercom steps into one automation

Under the hood

The code that runs this automation

This is the Python script that runs each time the automation starts. Your AI assistant writes it for you when you send the prompt, and you allow each step in your assistant. To change how the automation works, describe the change to your assistant and it updates the script for you. For every function it can use, see the script reference.

ai_qa_review_of_closed_conversations.pyexample

import re, time
from clickruntime import on_trigger, http, credentials, ai, notify

def intercom(method, path, **kw):
    r = http.request(method, f"https://api.intercom.io{path}", headers={
        "Authorization": f"Bearer {credentials.secret('INTERCOM_TOKEN')}", "Intercom-Version": "2.11"}, **kw)
    r.raise_for_status()
    return r.json()
INBOX = "https://app.intercom.com/a/inbox/_/inbox/conversation/"

@on_trigger
def run(event):
    now = int(time.time())
    q = {"operator": "AND", "value": [{"field": "state", "operator": "=", "value": "closed"},
                                      {"field": "updated_at", "operator": ">", "value": now - 86400}]}
    scored = []
    for c in intercom("POST", "/conversations/search", json={"query": q})["conversations"][:40]:
        parts = intercom("GET", f"/conversations/{c['id']}")["conversation_parts"]["conversation_parts"]
        thread = "\n".join(re.sub(r"<[^>]+>", "", p.get("body") or "")[:300] for p in parts)[:4000]
        r = ai.generate(thread, system="Score tone and whether the problem was solved.", max_output_tokens=80,
                        json_schema={"type": "object", "properties": {"score": {"type": "integer"}, "note": {"type": "string"}}}).json
        scored.append((r["score"], r["note"], c["id"]))
    worst = sorted(scored)[:5]
    notify.email_me("Lowest-scoring conversations yesterday", "\n\n".join(f"{s}/10 {n}\n{INBOX}{cid}" for s, n, cid in worst))
    return {"reviewed": len(scored)}

FAQ

Common questions

How does the AI score tone and resolution?

It reads each conversation's messages and rates how polite and clear the agent's replies were and whether the customer's problem appears to have been solved. The score is a guide for where to look, so a person should read the flagged conversation before giving feedback.

How many conversations does it email me about?

It emails the lowest scoring ones from the previous day, and you can decide the number, such as the weakest five or ten. A smaller number keeps the email short enough to act on, while a larger one is useful for bigger teams.

Is it appropriate to use for performance reviews?

Treat it as a coaching aid, not an official rating. AI scoring can miss context such as an unusually difficult customer. Using it to pick conversations for a human to read, then judging them yourself, is a fairer approach.

What permissions does it need?

read conversations. Nothing else in your accounts is used.

Does it need AI?

Yes. It uses ClickRuntime's built-in AI, so you don't need an OpenAI or Anthropic key. Every plan includes a monthly AI allowance.

How many runs does it use?

One run each time it runs (daily). Checks that find nothing to do and stop right away aren't counted.

Can I change how it works?

Yes. Describe the change to your AI assistant, for example a different schedule, channel or filter. It updates the automation, tests it and tells you what changed.

DailyAI