The moment it happens
Compare answers from two Anthropic models
One question goes to two models you pick, and you get both answers plus a third that reconciles them, by email.
- Runs
- The moment it happens
- Apps
- Anthropic API
- AI
- Not needed
- Access
- API key
Send this to your assistant
Create a ClickRuntime automation: give me a link I can post questions to. Send each question to two Anthropic models using my Anthropic API key, have the larger model reconcile the two answers, and email me all three with the token cost of each.
First time? Adding ClickRuntime to your assistant takes about two minutes.
Why use it
Why teams use this automation
Different Claude models suit different jobs, and choosing between them is often a guess. A founder deciding whether the cheaper model is good enough for customer replies, or a developer testing a prompt, ends up pasting the same question into several tools and comparing answers by eye. For important questions, a single answer is also a single point of failure: it may be confident and wrong, and there is nothing to check it against. Nobody records how much each answer cost, so the cheaper model never gets a fair trial.
When a question arrives, the automation calls the Messages API once for each model you pick, for example Sonnet and Opus. It can then make a judge call that merges the answers or checks them for disagreement, and it emails the results with the token costs of each call.
Developers testing prompts, teams choosing a model for production, and anyone who wants a second opinion on a high-stakes answer can use it.
What you get out of it
Model choice is based on evidence
Seeing two answers side by side, with their costs, shows whether the cheaper model is good enough. You choose with data, not on reputation, and may save real money at scale.
Disagreements reveal weak answers
When two models give different answers, that is a signal to look more closely. A merged or judged result catches errors that a single answer would have hidden. Disagreement is cheap to detect and expensive to miss.
Costs are visible per call
Token counts for each model come with the results, so you can calculate the price of a thousand questions before you move a prompt into production. A cheaper model may do the job for a tenth of the price.
Prompts can be tested quickly
Sending the same prompt to several models shows which handles your wording best, and shortens the time spent rewriting instructions by trial and error. A few runs often show which prompt style a model prefers, so you refine the instructions with real results.
Important answers get a second look
For decisions with real consequences, such as a contract summary or a customer reply on a refund, a second model's check is cheap compared with the cost of a wrong answer.
How it works
What happens, step by step
When you send a question to your personal ClickRuntime link, the automation starts straight away.
The question goes to two Claude models, a fast one and a stronger one, and each writes its own answer.
A third, most capable Claude model reads both answers, combines them into one best answer and notes where they disagree.
You get one email with all three answers and how many tokens each one used.
Before it runs live
- Connect Anthropic API
- Paste your automation's link into the other app's webhook settings
What you get
Build your own
Need it to work differently?
Your own rules
Conditions, thresholds, filters and exceptions, written the way you'd explain them to a colleague.
Any number of apps
One automation can read from one app and update several others. Over 248 apps are ready, and anything with an API works too.
AI where it helps
Summaries, classification, extraction and drafts use ClickRuntime's built-in AI, with no extra key.
On a schedule or on each event
Runs as often as every 5 minutes, or the moment an app sends an event, and keeps track of what it already handled.
Changes you can ask for
- Act only on the items that match your own rules, and skip the rest
- Add an AI step, for example to summarize, sort or draft a reply
- Add another app, for example also send it to Slack
- Combine it with other Anthropic API steps into one automation
Under the hood
The code that runs this automation
from clickruntime import on_event, credentials, notify, http
def claude(model, prompt, **extra):
resp = http.post("https://api.anthropic.com/v1/messages",
headers={"x-api-key": credentials.secret("ANTHROPIC_API_KEY"), "anthropic-version": "2023-06-01"},
json={"model": model, "max_tokens": 1024, "messages": [{"role": "user", "content": prompt}], **extra},
timeout=120)
resp.raise_for_status()
return resp.json()
def text(msg):
return "".join(b["text"] for b in msg["content"] if b["type"] == "text")
@on_event
def handle(event):
question = (event.json or {}).get("question") or event.body
a = claude("claude-haiku-5-5", question)
b = claude("claude-sonnet-5-5", question)
judge = claude("claude-opus-5-5", f"Question: {question}\n\nAnswer A:\n{text(a)}\n\nAnswer B:\n{text(b)}\n\n"
"Reconcile these into one best answer and note where they disagree.")
parts = [f"== {m['model']} ({m['usage']['input_tokens']} in / {m['usage']['output_tokens']} out tokens) ==\n{text(m)}"
for m in (a, b, judge)]
notify.email_me(f"Claude answers: {question[:80]}", "\n\n".join(parts))
return {"models": [m["model"] for m in (a, b, judge)]}FAQ
Common questions
Which Claude models can I compare?
What does the judge call do?
Is it much more expensive than asking one model?
What permissions does it need?
Does it need AI?
How many runs does it use?
Can I change how it works?
Related
You might also like
Pull details out of PDFs and photos
Send a document link and get back the fields you asked for, with personal ID numbers masked.
See how it worksGet a cited research brief every morning
Each weekday you get a few bullet points on what changed in your topic, each with a source link, and nothing you've already seen.
See how it worksGet warned when API spend runs high
If yesterday's Anthropic API cost went over your limit, you get an email showing which workspace spent it.
See how it worksKeep exploring
Set up “Compare answers from two Anthropic models” from your assistant
Copy the prompt above, or connect your assistant first. Free plan, no credit card.