"I want to cancel" is a dozen problems wearing one sentence.

The topic was easy. The reason was the whole game.

The intent model could hear that a cancellation was happening without much trouble. Hearing why is where it stalled. By early 2023 I'd spent four months tuning intents and the curve had gone flat: about 60 percent of reasons resolved, two in five coming back unknown, and every unknown routed like every other cancellation, straight toward the cancel steps.

Every unknown reason was a lost save.

A price complaint that reaches a specialist with context is a conversation about the plan. A price complaint that reaches nobody is a lost subscription. And the queue cuts the other way too: a specialist spending time on a call that was never savable, a closing business, a retirement, is a specialist not saving a call that was. The bar I set for the routing: if I send ten people to that queue, the ten should be truly savable. Not two.

Keep tuning, or change the tool.

My read was that the bottleneck was the tool, not the tuning, and I championed the swap at a time when the third option officially didn't exist: in early 2023 there was no sanctioned path to ship generative AI to production at the company. I partnered with the platform team to onboard the API that let a flow call a language model at all, then wrote the classifier prompt. Six-plus months of development per training cycle became days of prompt work.

Savable wasn't my call to make. Encoding it was.

The word doing all the work in this case is savable, and the Growth and Retention team defined it with the saves specialists who win these calls every day. They knew which reasons a specialist can turn around and which deserve a graceful exit instead of a pitch. I worked with them to get those guidelines out of their heads and into a form a model could follow, then structured them into the classifier's instructions.

The acknowledgment went through the same discipline. An early version answered every reason with the same line, "Thanks for the feedback," followed by the cancel link, even when the reason was a death. I rewrote the instructions so the acknowledgment reflects the reason with plain human empathy, while still never pitching a final one.

The model never argues with someone leaving.

It reads the reason, returns strict JSON, and lets deterministic flow logic decide what happens next. Nothing the model writes reaches the customer outside two contract fields, and the offer to connect with a specialist exists only when the reason is savable. The intent model still does what it was always good at, recognizing that a cancellation is happening; the classifier does what intents couldn't, reading why.

{
  "cancellation_reason": "too expensive",
  "is_savable": true,
  "reason_acknowledgment": "I understand that cost is an
    important factor. Thank you for your feedback.",
  "connect_to_expert_message": "I'm connecting you to a
    customer success expert to help with your cancellation."
}

{
  "cancellation_reason": "business closure",
  "is_savable": false,
  "reason_acknowledgment": "I'm sorry to hear about the
    passing of the business owner.",
  "connect_to_expert_message": ""
}

You are a classification assistant. Evaluate a
customer's reason for canceling and return a
structured JSON object.  …

- "cancellation_reason": a short label
  summarizing the reason  …

- "is_savable": false if the reason is final
  and unlikely to be saved  …
  true for all other reasons.

- "reason_acknowledgment": a short, neutral
  message  …  Do not offer help or take
  action in this field.

- "connect_to_expert_message": if "is_savable"
  is true, a brief message offering to connect
  the customer with an expert  …

Return valid JSON output only.

Verification came before trust: I tested the classifier on about 300 real customer utterances, a sample sized for 90 percent confidence, and drove it through the Responsible AI review. It cleared, free of hallucination, profanity, bias, and toxicity findings.

Same question. Two reasons. Two routes.

Recorded July 2023, on the working flow. The classifier reads each reason, and the flow acts on what it returns.

Right calls to the right humans.

The 2022 ask-why flow moved real results before the classifier existed: it directionally saved 12 percent of customers against a 5 percent target, on expert-reported saves. The classifier's result is routing precision, and that number is the case-rate lever: the higher the share of truly savable cases landing on the queue, the more of the specialists' time turns into saves instead of screening.