Beyond the @mention

One of the hardest problems in building multiplayer AI agents for shared spaces like Slack is deciding when they should speak. Feed an agent every message and it will chime in on irrelevant chatter until someone kicks it from the channel. Run an LLM pre-pass on every message to ask "should I respond to this?" and you burn inference budget just to stay quiet 95% of the time. Naive keyword filters ("wake up on any message containing deploy") are cheap, but they miss nuance and fire on noise.

Most teams settle for the @mention pattern: the agent stays deaf to the room until someone tags it by name. When a workspace is shared by dozens of people and multiple bots, explicit mentions have been the only reliable way to keep signal-to-noise under control.

The catch is that @mention moves the routing work onto people. Someone has to notice that a message is the agent's business, remember which bot handles it, and tag it in. In a busy support channel, nobody connects a merchant saying checkout "feels off" to an error alert twenty minutes later. A teammate who had been reading the channel would. Your customer already explained the problem, but the agent shows up late, or not at all, because nobody invited it.

Jev and a new class of small decision models change the math. Instead of generating text, a decision model takes context and a set of yes/no questions and returns calibrated probabilities in milliseconds. At roughly $0.04 per million input tokens, it is cheap enough to score every message in a busy channel, wake the agent only when a message crosses your threshold, and assemble the right scrollback before a full LLM ever runs.

Here is how we use Jev to gate an incident agent in Runtype:

Score every message, rewind for context, skip the @mention

With Jev in front of the agent, the bot can sit in the channel like a teammate who knows their lane. Jev reads the agent's charter (its role, capabilities, and response rules) and scores each incoming message and thread reply against it. Routine chatter about trial extensions, password-reset macros, and lunch-and-learns scores near zero, so the agent stays silent without touching a frontier model.

When a message finally crosses the activation threshold (like Haruto's checkout-api 5xx alert in the demo above), the agent doesn't wake up cold. Jev also scores the recent window on a second question: does the agent need this earlier message to understand the trigger? That rewind step pulls in the two earlier merchant complaints and the screenshot buried in Inês's thread while dropping the other six messages in between. The agent runs once, with a clean context window that already connects the alert to the customer symptoms.

In the channel, the agent feels like an attentive colleague rather than a command-line tool you have to summon. And because Jev and other decision model providers support zero data retention (ZDR), you can run per-message classification in sensitive channels without messages lingering upstream.

Try judge mode in Runtype

Runtype's judge mode for Slack surfaces is now in preview. It uses your product's description, agent instructions, and optional responseRules and eagerness controls to decide when to join the conversation. You can also turn on shadow mode first to log what the judge would have done on live channel traffic before letting it speak unprompted.

The judge only decides whether the agent enters the room; what it can do once activated is still governed by the tool approvals and policies on your agent. Read the docs on setting up a Slack surface in judge mode.

Put an agent in your team's channels with Runtype.