The setup
For a few months now I've been running an AI newsletter pipeline for IndianCoffeeBeans.com. Every week it scans the web — Reddit, blogs, news, the whole noisy mess — and pulls out the coffee stories worth sending to the people who signed up to care about Indian coffee. It finds the story, grabs the author, the outlet, the URL, and sends a clean, curated issue.
It's been working. Quietly, reliably, without me thinking about it.
Which, in hindsight, is exactly the problem.
The struggle
This week I was glancing at the current scan and noticed a writer. A journalist, actually — someone who covers food and coffee, and who'd be a genuinely good person to reach out to. Not for the newsletter. For a PR relationship.
So I asked the obvious follow-up: "are we saving these searches?"
The answer was no.
The breaking point
Here's what the pipeline actually does. It saves the cut — the curated items that make the newsletter, each with author, outlet, and URL. The stuff that ships.
What it throws away is the raw scan — every search result, every source it read before deciding what mattered. All of it discarded the moment the "is this newsletter material?" filter said no.
And that filter is doing two jobs at once, and I never noticed the second one.
"Good enough for the newsletter" is one question. "Worth remembering" is a completely different question. I'd wired them together as if they were the same thing. A writer who isn't newsletter material might still be a PR contact, a collaborator, a future customer. But my pipeline had exactly one exit ramp, and it was labeled delete.
The resolution
So I went back through all eight earlier scans — late June to early October — and had Claude recover the writers the pipeline had found and thrown away.
It found five. Five writers I'd never contacted, none of them in my PR targets list. They'd been sitting in scans I'd already paid to run, fully identified, with their outlet and their beat — and I'd deleted every one of them, because they didn't make the newsletter.
The data had been there the whole time. The pipeline found them. I just never saved them.
The fix wasn't a new pipeline. It was one field and one rule. Tag the relevant scan results as pr_leads, and keep the raw scan instead of throwing it away. Now my PR pipeline picks those writers up on its next run instead of rediscovering them from scratch.
One field. One rule. Months of deleted relationships.
The lesson
This is the third time I've watched a pipeline of mine silently throw away the next thing's inputs.
The coffee scraper was already downloading every roaster's full product list — prices, stock, variants — and deleting everything that wasn't coffee. The gear directory didn't need a new scraper; it needed the discard step deleted.
The newsletter pipeline was already finding writers and journalists. The PR pipeline didn't need a new scanner; it needed the raw scan saved instead of discarded.
A curation pipeline that persists only its output throws away its raw input. And the raw input is where the next pipeline's inputs live. The filter that decides "what makes the cut" is quietly also deciding "what gets remembered" — and those are two different questions, with two different answers.
What's next
So here's my question for you: what does your pipeline throw away — the output, or the input?
Go find the discard. It's the if not coffee: continue, the skip, the route to nothing. Then ask what's in the branch you skip. Your next relationship, deal, or product might be sitting in it, fully identified, waiting for you to notice it's there.