Case study · 04 / 06 · 20264 min read
  • Python
  • LLM pipeline
  • Slack

Customer Feedback Monitor

What if the product team never had to go looking for what customers were saying?

Where
Built on my own, in my own repo. Transferred to a product team before I left, running in production on a weekday schedule then. I cannot see it from outside. A day of it costs little, the model calls and a few budget scrapers, so nothing I know of would have stopped it.
Role
Built it. Python, LLM pipeline, GitHub Actions
SEN-MO, the bot's avatar: an astromech droid with a sentiment readout on its dome, on a starfield
Read every weekdayOne team, one product

01Scope

  • 7

    sources in one bot, where the plan was one person per source

  • 1

    lane I was handed: the app store reviews

  • 5

    digests a week, one every weekday, until I left

The VP wanted every PM and designer holding a lane. Mine was the app store reviews, someone else had Reddit, someone else the feedback responses, then Google News, then TikTok, and so on. When it was pitched I wondered whether a bot could do the whole thing. The VP was all for it and let me explore, and after a lot of learning it ran.

02What it is

An automated pipeline that watches customer feedback across the App Store, Google Play, Reddit, X, Facebook, Trustpilot and Google News, analyses and clusters it, and posts a digest to Slack every weekday morning. During a launch window it also runs hourly to catch app-breaking issues in real time.

03Why it belongs here

  1. 01Multi-source ingestion with fallback chains per source, so one dead API does not take the digest down
  2. 02LLM enrichment and clustering, so the output is themes
  3. 03A feedback loop: the team replies with commands in the Slack thread and the bot reconfigures itself
  4. 04Orchestrated on GitHub Actions with an external scheduler, so there is no server to babysit and nothing that needs a hand. The only thing that stops it is the model credit running out. It was posting every weekday when I left

This site is about noticing that nobody reads the reviews, then building the thing that does.

04How it runs

The pipeline: seven sources feed fetch, dedupe, LLM enrichment and clustering, which post a digest to Slack in three layers, pulse, detail and thread. A reply in the thread loops back into the config for the next run. The digest archive feeds a planning board downstream
Structure only. Every source has a fallback chain, dropped items stay in the day's source file so the exclusions can be audited, and a reply in the Slack thread changes the next run.

Nothing it read is shown here. The feedback belonged to the company and the bot now runs inside their team. The diagram above is the shape. The digest below is that shape with invented numbers for an invented app.

05What I dropped

  • TikTok and Instagram as sources, each account watchedmainly the price of the Apify actors that read them.

06The digest

A sample morning digest as it renders in Slack: the bot's post with a one-sentence headline, today at a glance with counts against a usual day, sentiment breakdown bars, volume by platform, a seven-day net sentiment trend, then a negative, a positive and an app-breaking section, and three example thread commands
Sample data. The app, the numbers and the reviews are invented. The layout is the one that posted every weekday morning.

It does two jobs in one post. The top is a summary so the team can track how we are doing without reading anything: one headline, the day's counts against a usual day, sentiment, volume by platform, a seven-day trend. Under it is the detail, every item the summary was built from, urgent ones first, then by source, each with its link, its store and its version. The summary says whether today is normal, and the detail is where you act. Every item carries its source link, so any theme can be checked back to the comment it came from, and I checked the sources were real when I built it. The sentiment labels were never scored against a hand read.

The team steers it from the thread. Reply with a command and the next run picks it up: which sources to mute, which area to focus on, whether to go hourly. Nobody has to open a config file to change what the bot does. Commands came back in the thread, posts got reactions, the bot got quoted in other channels, and people brought up what it had said that morning in person.

07From digest to board

The digest archive, one file per day, fed a kanban I built next. Every item is deduped across days, categorised as bug, enhancement, new feature, praise, venting or off-topic, given an area and a severity, then grouped by the squad that owns the area and sorted by a severity score so a PM reviews the most important first. Each theme card carries its raw evidence rows and their links, so the decision is made from the quotes. Before I left I handed a board to every product manager to cycle their own domain through, and my VP took it as the format for the rest. The feedback was that it surfaced evidence they had not seen. What happened to it after I left I cannot see.

Statuses follow the review rather than the roadmap: unreviewed, discuss, ticketed, dismissed, and parked for feature asks that are out of scope for a bug pass. Praise and venting never become cards. They roll up per area on an overview tab, because a PM needs to know the mood without triaging it.