It looked great in the demo: how to ship an AI feature that survives production

Whether a task belongs to a language model comes down to three questions. Who pays when the model gets it wrong, who verifies the output and how fast, and what is your budget per request in money and in waiting time? If you can answer all three, the rest is ordinary engineering: gather the right context, measure, restrict permissions, keep records. If you cannot, you end up with an impressive demo and a feature that gets quietly disabled after the first customer complaint. The difference has little to do with how clever the model is. It comes from the scaffolding you build around it.

The failure is usually not the model

MIT Media Lab's NANDA project published "The GenAI Divide: State of AI in Business 2025", which looked at more than 300 enterprise initiatives and found no measurable profit-and-loss impact at the large majority of them. The headline number, 95 percent, was argued over at length and the methodology drew criticism. The more useful finding sits underneath it: in stalled projects, the model's capability is almost never the bottleneck. The data was not ready, the output did not fit into an existing workflow, and nobody had written down what counted as success before the build started.

The same report has a second finding worth reading carefully. Solutions built with a vendor who knows the domain succeeded around two thirds of the time, while purely internal builds succeeded roughly a third as often. That is not an argument against building. It is an argument about where the difficulty lives: not in training models, but in embedding domain knowledge into a workflow. If that knowledge is in your building, you have the advantage. If it is not, buying will not save you either.

Which tasks fit: cost of error and speed of verification

Two axes decide your first feature. How expensive is a wrong answer, and how many seconds does it take to know the answer is wrong?

Good first candidates keep showing up in the same places. Drafting (a quote, a reply, a product description), because a person approves before anything leaves the building. Classification and routing (send this request to the right team), because a wrong label costs one click to fix. Field extraction (the amount on an invoice, the date in a contract), because the user sees the value next to the source document. Summarizing long documents, as long as every claim links back to where it came from.

The bad first candidates are just as consistent: open-ended chat that gives customers binding information, a support bot explaining refund and warranty policy, any flow that can move money or delete records on its own. None of these are impossible. They are simply the wrong place to start, and nobody should attempt them before they can measure accuracy.

A bot speaking for you binds you

On 14 February 2024 the British Columbia Civil Resolution Tribunal held Air Canada responsible for its website chatbot, which told a passenger that a bereavement fare could be claimed after the flight. The airline argued that the chatbot was a separate entity accountable for its own statements. The tribunal rejected that outright: the bot is part of the website, and the company is responsible for everything on it. The damages ran to a few hundred Canadian dollars. The precedent is worth far more than that.

A more recent case is closer to home for software companies. In April 2025 the support bot for the code editor Cursor invented a policy that had never existed, telling users their subscription was limited to a single device. Developers started cancelling. The company had to apologise publicly. The nastiest detail: the bot did not give everyone the same answer, so users comparing notes could not even establish whether the rule was real.

The rule that follows is blunt. If you let the model answer policy questions, the model writes your policy. Answers must come from a fixed source, the model's job is to find and relay it, and when no source matches, the correct output is "I don't know, transferring you to a person." You cannot achieve this by politely asking for it in a prompt. It has to be enforced in the flow.

Model choice is the small part, context is the work

Teams spend weeks comparing models, then discover that quality is mostly determined by what they feed the model. If retrieval does not find the right paragraph, the best model on the market will confidently make one up.

So the first investment belongs on the retrieval side. Which documents count as authoritative, which version is current, why the superseded one is still in the folder, and which of the three near-identical copies of the same procedure wins. If half your internal documentation is out of date, all the model adds is a more persuasive delivery of stale information.

Permissions are part of context too. Retrieval must never surface a document the user could not open themselves, and the filter belongs before the query rather than after it. Our post on authorization models covers how to structure that. An AI feature inherits the existing model; it does not get to invent a parallel one.

You cannot improve accuracy you never measured

"We tried it, the answers look good" is not a measurement. A measurement is an agreed set of examples: real inputs, each paired with the output you expect.

Common practice is to start with 20 to 50 examples and grow toward 100 to 1000 as the feature matures. Variety beats volume, because a thousand near-duplicates teach you less than a hundred cases that each break the system differently. Make sure the set includes empty or partial input, two sources that contradict each other, a question outside the scope, and a deliberately hostile prompt.

Once the set exists the discipline is simple. Prompt changed, run it. Model version changed, run it. Retrieval logic touched, run it. Set a threshold (a minimum share of correct answers and zero forbidden outputs, for instance) and block releases that fall below it. This is the same gate we described in our post on test automation, except the result is a ratio rather than pass or fail.

Grading with another model is a practical shortcut, but calibrate it first: measure how often the judge agrees with human labels on the same examples. If agreement is poor, what you have is not a measurement, it is a second guess.

Cost is per request and shows up at month end

Pricing runs on tokens, input and output are priced separately, and output is significantly more expensive. Every document you add to the retrieved context enlarges the input, and therefore the price of every single request. Resending the whole conversation each turn does the same thing.

Three levers do most of the work. Cache the stable prefix (system instructions, product catalogue, procedure text) so repeated context is read at a fraction of full price, and keep that prefix byte-identical, because a timestamp inside it will silently miss the cache on every call. Push anything that does not keep a user waiting (overnight bulk labelling, archive summarisation) through batch endpoints, which usually run at around half price with major providers. And set quotas per user and per tenant before you launch.

Missing quotas are not only a billing problem. OWASP lists unbounded consumption as its own entry in the LLM top ten, because the same gap inflates the invoice and degrades service for everyone else. Set a latency budget as well: after how many seconds does the call time out, and what does the user see at that moment? Every flow needs a path that works without the model, whether that is classic search, a stored template or a handoff to a human. The queueing pattern for anything long-running is in our post on background jobs.

There is a contractual side too. Get it in writing whether your data is used for training, how long it is retained and in which region it is processed. What happens when employees cross that line on their own is the subject of our post on shadow AI.

Treat model output as untrusted input

The OWASP GenAI Security Project published the 2025 edition of its Top 10 for LLM Applications on 18 November 2024. Prompt injection has held the first position for two editions running, and the reason is architectural: the model reads instructions and data through the same channel, with no hard boundary between them.

Direct injection is the familiar version, where a user types "ignore your previous instructions." The dangerous variant is indirect: instructions hidden inside a document, a web page or an email that the model will read later. Picture a system that automatically summarises incoming support tickets, and an email crafted with hidden instructions in it. The user typed nothing, and the system was still steered.

Two more entries bite regularly in production. Improper output handling means passing generated text straight into a SQL query, into HTML, into a shell command or into an API call. Excessive agency is about what the tools you handed the model can actually do. Give the model a "cancel order" tool and prompt injection stops being a text problem and becomes an authorization problem. One sentence covers the defence: validate model output the way you validate anything arriving from the internet, grant tools the least privilege that works, and put a human in front of anything irreversible. The classic web equivalents are in our OWASP Top 10 post.

Log everything, release gradually

When a customer says "it told me something wrong," what you have on hand decides whether the feature survives. Log the input, the retrieved context chunks, the model and prompt version, the output and any correction the user made. That same log is the raw material for your next evaluation set. Our post on audit trail design covers the structure; add one field to it, the version of the model that produced the answer.

Release with the same discipline. Internal team first, then a small slice of users, then everyone. The plumbing for that is in our post on feature flags. The number to watch is the correction rate: what share of outputs is accepted as is, what share is edited, what share is thrown away.

Telling users they are talking to a machine is now an obligation

The transparency duties in Article 50 of the EU AI Act started to apply on 2 August 2026. In short: a user must know when they are interacting with an AI system, and generated content must be marked in a machine-readable way. The Digital Omnibus package pushed the high-risk timeline back (Annex III to 2 December 2027, Annex I to 2 August 2028), but it left the transparency article alone. Systems already on the market before 2 August 2026 were given until 2 December 2026 for the marking obligation.

Turkey has no dedicated AI statute in force. Several bills are pending in parliament, a parliamentary research commission published its report in March 2026, and the 2026-2030 national AI action plan took effect through a presidential circular dated 17 August 2026. Existing law already applies in the meantime, from consumer protection to misrepresentation to data rules, so there is nothing to wait for. If you sell into the EU, the Article 50 clock is already running for you.

A first step that fits in this week

Pick one task. Routing inbound support requests to the right team, say. Pull 40 real examples from the last three months and write the correct answer for each by hand. Measure a baseline without any model first: how many of those 40 do plain keyword rules get right? Then run the same set through the model.

The gap between the two is the real value of the feature, and it is now a number. Put it next to the cost per request and the correction workload and the decision makes itself. If the gap turns out to be small, that is good news too. You learned it in a few days, cheaply.