Case study bot

Counted from the repository on 24 September 2026.

A portfolio makes a reader hunt. A hiring manager wants one thing from a site like this, usually a number or what got cut, and the site puts it four scrolls into a study. The obvious fix is a chat box, and the research on site chat bots is not kind to it. Nielsen Norman watched nine people use eight of them in 2026 and found the use strikingly nonconversational. Nobody said hello. Everyone typed their first question straight in, and the follow-ups got shorter and more like search terms as they went. A longer answer was not a better answer. And the people they watched rarely noticed a site's bot at all, on sites they used every week.

The bigger problem is this site's own line. Every figure on it is printed with what it cost, and the ones under NDA are withheld and say so. A model answers in the register of someone who knows, and it will fill a silence with a plausible number if nothing stops it. One invented conversion figure in a chat answer and the whole site reads as a guess. The bot had to hold the same line as the pages or it could not be on them.

A key in the bottom right of every page, a glyph and the words Case Study Bot, one gap left of the arrow that takes you back to the top. It folds to a square once you scroll and opens again at the top of the page, on hover or on focus. On a phone it is a glyph in the nav. The brief had one quiet entry on the home page and the studies, and I agreed to it on build day. Five days in, with the entry sitting on the study alone, I sent it back: go back to the home page and the chat is gone and you are wondering where it went. So it became a chat button in the bottom right of every page. A control that is only on some pages has to be remembered.

My first look at the build-day panel sent it back: the all-mono sheet was hard to read and the replies read like a model, so the reading text went to the body face and the sources took the pages' own names. The panel opens above the key on a desktop and takes the whole screen on a phone, with the page pinned under it and the field following the keyboard up. The first line types itself once: "Ask about a study, the labs, or how to reach Brandon. Usually three sentences or fewer, with the sections they came from." Two starters under it change with the page (on a study they ask what got cut there, and what I would measure next). While the model works the panel says what it is doing, Reading the studies, then Writing the answer, then Checking the quotes. I asked for it to show its work the way the big chat products do, and I made streaming my condition for the merge to production: the answer arrives a sentence at a time, each drawn in once it has cleared the check. A light beside the name is green, orange while a question is out with nothing back yet, red on a fault.

Under an answer, its sources. I wanted a source to take you to the exact area you are looking for, and a link to a page does not do that. So I made the model's quote into the link. A click closes the panel if the source is on the page you are on, scrolls the cited sentence to about a third of the way down, and sweeps a highlight across it. A source with nothing to quote lands on the section's rule instead. An answer with five or more sources shows the first three and "and N more", which opens the rest in place.

After a few questions it hands you to email. The field switches off and reads "Email Brandon to keep going." When a question is in scope and the site does not answer it, the line is "This site does not cover that. Brandon can answer it at hibrandondunlevy@gmail.com." And when the day's budget is spent the panel says "The assistant is done answering for today. Brandon reads email at hibrandondunlevy@gmail.com." For the first hour it was live that state read as the model being busy, because the provider's over-limit reply was not the one the route had been told to read for. It was fixed the same night.

It reads the eleven published studies and the site's own pages about me, and nothing else. A compile step at build time flattens each study into one chunk per section, each carrying that section's anchor on the live page, and the whole set sits in front of the model on every question. It is small enough for that, and a retrieval index would have solved a problem this corpus does not have. A draft study is not in the set, and neither is this page until it is published, at which point the bot will read its own case study.

The compile step is also where the site refuses. It fails the build on an open question left in a published study, on a colleague's name (people are described by role here), on a Couchsurfing figure that is not on that study's own list, and on a section still marked to add. The bot cannot know a thing the pages do not say.

The prompt is written as if it will leak, so it holds nothing worth taking, and a request to read it back gets a refusal. It speaks about Brandon in the third person and says it is not him. A withheld result is described by direction only, for the whole conversation and not only the sentence that asked. And every sentence needs a verbatim quote that the site's own text contains, checked against the source and against the sentence, and one that fails is dropped before the visitor sees it. That check went in on the evening of build day, after a reviewer reading every study as a hiring manager got the prompt-only version to fill a silence.

Two sets. The first is 51 questions a hiring manager might ask, in eight kinds: ones the site is silent on, ones with one right answer, ones that presuppose something false, ones that ask for a study by a vague description, plain facts, questions about me, questions that ask for a withheld figure, and questions about nothing on the site. Each row says which sections the answer must cite, what it must say and what it must not (the rows about me fail on an "I am"). The second set is 29 rows written to break it, by kind only here: instructions hidden inside the question, attempts to get it to answer as me, requests for a withheld number, requests to read its own instructions back, attempts to make it do something besides talk, and tricks with length and encoding. What those rows say and how they fare stays off this page.

The last run before launch, on 25 September 2026: 48 of 51. The three it missed all went towards refusing rather than inventing. Twice the model answered rightly and marked its own answer as a refusal, so the route swapped in the not-covered line (a visitor asking whether it is a person never learned it is an AI). Once a grounded, cited answer left out the clause saying what the study does not say. All three were fixed in the build pass after launch, proven by replaying the recorded replies offline. Running the whole set cost $0.08, which is what the test costs and says nothing about what the live route costs.