Building a Slack Bot That Actually Gets Used: What We Learned Shipping Three Versions

We've built three versions of an internal Slack bot for surfacing deployment status and flagging issues that need attention. The first got ignored within a week. The second got muted by roughly half the team within a month. The third is still checked daily, unmuted, a year later. None of the three differed much in the underlying automation logic — what changed each time was almost entirely about notification design, and this is the specific story of what broke each version and what finally fixed it.

Version 1: notified on everything, got ignored on everything

The first version posted to a shared channel on every deploy, every test run, every merged pull request — the instinct being "more visibility is better." Within a week, the channel had enough traffic that nobody was reading individual messages anymore; they were scrolling past a wall of green checkmarks looking for something specific, which defeats the entire purpose of a notification system. A notification that arrives alongside forty other routine notifications a day isn't really a notification anymore, it's ambient noise, and ambient noise gets tuned out by the human brain specifically because that's what ambient noise is for.

# Version 1's logic, roughly
on_deploy_start -> post_message
on_deploy_success -> post_message
on_test_run_complete -> post_message
on_pr_merged -> post_message

Every single event got the same treatment: a message in the same channel, with the same visual weight, regardless of whether it was routine (a successful deploy, which happens dozens of times a week) or something someone actually needed to see and act on.

Version 2: filtered to failures only, still got muted

The second version cut the noise dramatically — only failures, only things that plausibly needed a human to look at them. This sounds like the obvious fix, and it helped, but the channel got muted by roughly half the team within a month anyway, for a reason that took actual conversations with muted teammates to surface: failure notifications were still going to everyone regardless of whether that specific failure related to anything they were working on. A frontend developer got paged, functionally, by every backend deploy failure, and vice versa — the notifications were more relevant on average than version 1's, but still not relevant to any specific individual most of the time, which produces the same tuning-out effect through a slightly different mechanism.

# Version 2's logic
on_deploy_failure -> post_message(channel: '#deploys')
on_test_failure -> post_message(channel: '#deploys')

Fewer messages, same underlying problem: broadcast to everyone regardless of individual relevance. The lesson from this specific failure was that "reduce volume" and "increase relevance" are two different fixes, and we'd only actually applied the first one.

Version 3: routed by ownership, and only escalated what needed a person

The version that actually stuck routes based on who owns the affected service, using a simple ownership mapping maintained in the repo itself, and distinguishes between "informational" and "needs a response" with genuinely different treatment for each:

# .github/CODEOWNERS-style mapping, used by the bot
services:
  api-gateway: [alex, priya]
  payment-service: [marcus, sarah]
  frontend-app: [dev, jamie]

on_deploy_failure(service) ->
  owners = services[service]
  post_direct_message(owners, urgency: 'needs_response')
  post_summary(channel: '#deploys-log', urgency: 'informational')

on_deploy_success(service) ->
  post_summary(channel: '#deploys-log', urgency: 'informational')

Two changes here compound: failures now go as a direct, specific mention to the actual people who own the affected service, rather than a broadcast everyone has to individually filter for relevance themselves. And a separate, lower-traffic log channel still exists for anyone who wants the full picture, but nobody's expected to actively monitor it in real time — it's there for context, not for interruption, which is a genuinely different job than the direct-mention path.

The specific week this got tested for real

A genuine multi-service outage, three services failing near-simultaneously due to a shared dependency going down, was the real test of whether the routing logic held up under exactly the load it was designed for. It worked as intended: three separate direct-mention groups got paged, each specifically for their own affected service, rather than one giant broadcast everyone had to parse to figure out which parts, if any, were actually their problem. The informational log channel captured the full picture for anyone doing post-incident review afterward, while the people who needed to act in the moment got exactly the specific, addressed message relevant to them and nothing else competing for their attention in that same channel.

How the routing logic handles a service with more than one owner

A detail that came up quickly once the mapping was real: several services genuinely have more than one owner, and the routing logic needed a deliberate rule for who actually gets the direct mention versus who's just informed after the fact, rather than paging every listed owner for every single failure regardless of who's actually best positioned to respond in the moment.

on_deploy_failure(service) ->
  owners = services[service]
  primary = owners[0]  // first-listed owner gets the direct page
  secondary = owners.slice(1)  // remaining owners get a lighter mention

  post_direct_message(primary, urgency: 'needs_response')
  post_mention(secondary, urgency: 'fyi', channel: '#deploys-log')

Listing owners in a specific, deliberate order in the mapping — not alphabetically, but by who's actually the most likely first responder for that specific service, based on genuine familiarity with it — turned an ambiguous "who's actually supposed to handle this" moment into an explicit, pre-decided default, without removing the option to loop the secondary owner in immediately if the primary owner is unavailable or the issue turns out to need more than one person.

What we measured, not just how it felt

Beyond anecdotal impressions, we tracked two numbers across all three versions: what percentage of the team had the relevant channel muted, and average time-to-acknowledgment on a failure notification. Version 1: channel mute rate climbed to nearly 40% within the first month, and there was no clean way to even measure acknowledgment time since nobody was reliably reading messages in real time. Version 2: mute rate dropped to about 25%, better but still substantial, with an average acknowledgment time north of forty minutes on failures — people saw them eventually, just not promptly. Version 3: mute rate on the direct-message path is effectively zero (muting a direct mention defeats its own purpose in a way people intuitively avoid), and average acknowledgment time dropped to under five minutes, because a direct, addressed message competes far less with everything else in a busy channel than a broadcast message does.

How this connects to a broader pattern in automation design

The core mistake across all three versions was treating "the automation ran successfully" and "the automation actually served its purpose" as the same thing, when they're genuinely different questions — version 1 and 2 both ran without a single technical error the entire time, and both still failed at the actual job of getting the right information to the right person promptly. This is a close cousin of the failure pattern covered in why automations keep silently failing, just manifesting as a design failure rather than a technical bug — nothing crashed, nothing threw an exception, and the automation was nonetheless not doing its actual job, which is arguably a harder failure mode to notice precisely because there's no error log pointing at it.

What almost broke version 3, a few months in

The CODEOWNERS-style mapping is only as good as its own upkeep, and about three months after version 3 shipped, a new service was added to the codebase without a corresponding entry in the ownership mapping. Its first production failure had nowhere specific to route to, and silently fell back to a generic, unaddressed message in the log channel — the exact failure mode version 3 was built to eliminate, just triggered by an omission in the mapping rather than a design flaw in the routing logic itself. The fix was adding a specific check: any deploy for a service missing from the ownership map now posts a loud, unmissable alert to a small on-call channel instead of silently falling back to the low-attention log — turning "nobody owns this, so nobody gets told" into "nobody owns this, so everyone on-call gets told immediately," which is a meaningfully safer default failure behavior for a routing table that will inevitably drift out of sync with the actual codebase over time as new services get added.

What we'd tell someone building their first internal bot

• Design for who needs to act, not just what happened. "Something happened" and "you specifically need to do something about this" are different messages and deserve genuinely different delivery, not the same channel with the same visual weight.
• Measure mute rate as an actual metric, not just a vague impression — it's a direct, honest signal of whether your notification design is working, and it's usually available for free from whatever platform you're building on.
• Keep an informational log separate from anything meant to interrupt someone. Both are legitimate needs — full visibility for review, and prompt action for real issues — but conflating them into one channel serves neither need well.
• Sketch the routing logic out before writing any bot code. We mapped version 3's actual ownership-routing flow using the AI Automation Builder before touching the Slack API at all, which made the ownership-mapping design decision explicit and reviewable before it was buried inside application code nobody would think to revisit later.

The underlying automation — detecting a deploy failure and sending a message — barely changed in complexity across all three versions. What changed, and what actually determined whether the bot got used or muted, was entirely about who received which message and how urgently it was framed, which is a design question about people and attention, not a technical question about triggers and webhooks at all.