It started, as most digital disasters do, before my first coffee
At 8:47 a.m., my client messaged:
“Our AI is sending nonsense to our customers.”
What followed was a journey through authentication hell, AI confusion, and brand-voice chaos — ending in a system that cut email response times from 48 hours to 3 minutes.
If you’ve ever wondered why your AI automation just suddenly breaks or how to build a more reliable system, this case study will walk you through what went wrong — and exactly how we fixed it.
The client’s email nightmare
A fast-growing e-commerce brand approached me with a familiar problem: their customer support inbox was drowning.
They faced repetitive questions about orders, returns, shipping and refunds. Staff were overworked, replying manually with copy-and-paste templates. The team’s enthusiasm had left the building and customers were growing impatient.
- Response time had ballooned to 48 hours
- Customer satisfaction scores had dropped below 70%
- One top team member resigned citing “email-induced carpal tunnel syndrome”
They needed AI customer service automation — fast, accurate and on-brand.
Building the AI workflow (and why it went wrong)
To solve the issue, I built a modern AI email automation pipeline using self-hosted tools for full control and transparency.
The stack:
- n8n – orchestrating the email workflows
- Claude API – the natural language brain
- Microsoft Graph API – secure email integration
- Supabase – storing conversation context
- Make.com – backup automation layer for resilience
The goal was simple: a new email arrives, the AI drafts a contextual reply, it’s checked if needed, then sent automatically. In theory? Seamless. In practice? A digital soap opera.
The three big problems
1. Authentication chaos
Microsoft Graph API’s authentication system is famously complex. OAuth tokens expired unpredictably, refresh tokens failed silently and permissions seemed to change overnight. Even with “Mail.Send” enabled, messages refused to send.
Lesson: use delegated permissions, refresh tokens every 45 minutes, and store credentials securely. Never assume “send” means “send”.
2. AI context amnesia
Claude initially performed brilliantly — until it forgot everything halfway through conversations. One moment it was discussing delivery times, the next it was recommending banana bread recipes.
Root cause: context window overload. I was feeding the AI too much history, too many documents and too little relevance.
Fix:
- Use only the last three message exchanges
- Inject only relevant knowledge base snippets dynamically
- Summarise the customer profile in under 200 words
- Add a “memory jogger” prompt reminding the AI what was discussed
The result: no more recipe suggestions in customer service replies.
3. Brand voice breakdown
The client’s tone was friendly, helpful and full of Northern charm. The AI’s tone? It swung between Victorian butler and excitable teenager.
Solution: I created a “voice calibration” prompt with twenty sample responses. Then I added a second AI layer — a “voice consistency checker” — to review each draft and correct anything off-brand.
That delivered 94% on-brand accuracy, with the rest flagged for human review.
The results (and what you can learn)
After days of refinement, the system transformed performance.
| Metric | Before | After |
|---|---|---|
| Average response time | 48 hours | 3 minutes |
| Customer satisfaction | 68% | 91% (+34%) |
| Time saved per day | – | 5.5 hours |
| Accuracy | – | 94% |
Even better — the team now enjoys using it. Their inbox is under control, and any uncertain replies are flagged for quick human input.
And yes — those earlier, garbled AI messages? That came from a staff member accidentally uploading their dissertation notes into the knowledge base. Lesson learned.
Key takeaways for AI automation success
- Start small and validate quickly – don’t over-engineer context at the start.
- Keep workflows modular – easier debugging and cleaner data flow.
- Design for fallbacks – assume an API or AI will fail occasionally.
- Define your brand voice early – train the AI on examples, not adjectives.
- Monitor continuously – log every error, correction and manual override.
Ready to fix your own AI email automation?
If your automation is sending nonsense, losing context or breaking tone, don’t scrap it — troubleshoot it.
At Crouch End Media, we build reliable AI automation workflows for small businesses that save time, reduce errors and deliver measurable results.
Book a free consultation call. We’ll review your current automation and show you how to improve performance, safely and cost-effectively.