Building With AI, Without the Guesswork.

The first Field Guide, AI, Without the Hype¹, covered what Artificial Intelligence (AI) is and how it behaves. This one picks up where that left off and covers what it takes to build something with it. Nine explainers sit behind these eight chapters, and they arrived in the order the work itself arrives: first you wire the thing into your tools, then you give it a job, then you argue about which model, and somewhere near the end someone asks whether it is safe to put a client's name on the output. Read it as a build manual. Each chapter names the decision, shows what the evidence supports, says what it means when you are scoping and quoting, and points back at the explainer it came from when you need the detail.

1. Wiring it up

A model on its own knows nothing about your client. Two wires change that. The first reaches your tools³: the Model Context Protocol (MCP) is an open standard for connecting an assistant to software, and Anthropic describes it as a universal, open standard for connecting AI systems with data sources¹². OpenAI supports it too, which is what turned it from one vendor's idea into a port worth building against. The second wire reaches your knowledge⁴: text becomes coordinates, and the system retrieves the passages nearest the question. Both wires do the same job. They put the right context in front of the model at the moment it answers.

• •Retrieval is measurably better than keyword matching for this. A purpose-built dense retriever beat a strong keyword baseline by 9% to 19% absolute on top-20 passage retrieval accuracy¹³. That gap is why “search our documents” became a retrieval project rather than a search-box project.
• •A protocol is a connector, and it carries no judgment about what should be connected. The same port that reaches a calendar reaches a production database. OWASP files that risk as excessive agency¹⁵, and the fix is scoping the permissions rather than trusting the model to be careful.

2. Giving it a job

An agent is a model handed a goal and the means to act on it. The demos are spectacular and the failure mode is boring: a system that works in a scripted run and wanders on the real thing. Anthropic's own guidance is unfashionably plain. The teams that succeed find the simplest solution possible, and only increase complexity when needed¹⁴, and many jobs never need the full agent loop. A shipped agent² usually turns out to be a narrow task, a small set of tools, and a person who signs the result.

• •Scope the permissions before the capability. OWASP's excessive agency¹⁵ entry covers exactly this: too much functionality, too many permissions, too much autonomy. Every tool you connect is a thing the agent can do at three in the morning without asking.
• •Governance is a control requirement. The NIST AI Risk Management Framework¹⁶ treats measurement and oversight as part of the system rather than paperwork bolted on afterward. Write the oversight into the statement of work, not into the retrospective.
• •Decide what “done” means before you build. Without a test you agree on up front, “the agent works” becomes an argument about vibes, which is the argument evals¹¹ exist to settle.

3. Choosing the model

Two settings drive most of an AI feature's bill, and neither is the sticker price per token. The first is whether you reach for a reasoning model⁶, which works the problem out privately before it answers. That hidden work is billed: OpenAI is explicit that reasoning tokens are billed as output tokens¹⁷ even though you never see them. The second is whether you reuse your standing context. Prompt caching⁵ reads the unchanging part of a prompt once and resumes from it, and Anthropic prices a cache read at roughly a tenth of the base input rate.

• •Reasoning is a dial, and it can be turned up too far. Researchers have documented models spending far more computation than easy problems warrant, calling it overthinking¹⁸. Pointed at a simple lookup, a reasoning model costs more and takes longer for an answer that is no better.
• •The private chain is not a confession. Anthropic found that reasoning models don't always say what they think¹⁹, so a visible chain of steps is a useful artifact rather than an audit trail. Do not sell it to a client as one.
• •Caching pays when a large standing prompt gets reused inside a short window. Structure the prompt with the stable material first, and the discount is available. Retrofit it later and you are rewriting the feature.

4. Reading the claims

Every vendor arrives with a chart. A benchmark⁹ is a fixed exam, and the score says how a model did on that exam under one set of choices. Read it the way you would read a single test score for a candidate: narrow, and easy to over-interpret. Stanford's HELM makes the honest version of this visible by reporting accuracy, robustness, fairness, efficiency²⁰ and more alongside each other, rather than collapsing a model into one number.

• •The gap at the top is often inside the noise. Anthropic's own evaluation work argues for adding error bars to evals²¹ and treating scores as statistical estimates. Draw the interval every score deserves and a one-point lead can vanish.
• •The exam can leak into the study material. A survey of benchmark data contamination²² documents test items turning up in training data, which inflates a score without improving the model.
• •Leaderboards have their own dynamics. The Leaderboard Illusion²³ paper describes how selective reporting and unequal access distort arena rankings. Treat a rank as a shortlist, then run your own small test on the client's real task.

5. Safe to publish

Asked whether an AI feature can be guaranteed never to say anything off-brand, the honest answer is no, and the useful answer is layers⁸. Give it the tone and the refusals up front. Screen what comes in, ahead of the model. Hand it approved material to quote from. Scan the reply on the way out. Then put a person in front of anything carrying a client's name. Each control catches a share of what reaches it. Stacked, with different gaps, very little threads all of them.

• •Prompt injection is the standing risk and it has no complete fix. OWASP ranks prompt injection²⁴ first among risks to applications built on language models, which is why input screening and output filtering are separate layers rather than one setting.
• •A filter catches categories, and your brand is not a category. Toxicity classifiers do not know that a client never says “cheap.” That gap is what the human on the last mile is for.

6. Being found

Clients have started asking how they show up when someone puts the question to an assistant instead of a search box. The mechanism is less mysterious than the pitches suggest. Assistants answer by retrieving live pages and quoting them, and Google says its AI features rely on our core Search ranking systems to retrieve relevant, up-to-date web pages²⁵. So the route to being cited⁷ runs through being retrievable first and quotable second.

• •Credibility moves are the ones that measure. The paper that gave Generative Engine Optimization (GEO) its name measured nine tactics against live answers. Sourcing, quotation and hard numbers came out on top, lifting a page's visibility by up to 40%²⁶ in generative answers.
• •What worked on a search engine does not transfer. The same paper found keyword stuffing²⁶ gave little to no improvement. When a vendor pitches a secret setting that unlocks the assistants, the pitch is the product.

7. Making the work

Generative video arrived fast enough to reset the question every quarter. Google's Veo can generate cinematic video with audio²⁷, which makes it genuinely useful early in the process. It stays risky at the finish, because holding a subject steady across a longer take is the open research problem: the survey literature notes video generation demands not only high-quality individual frames but also strong temporal coherence²⁸. What holds up in practice is generative video for the storyboard¹⁰ and a human crew for the film that ships.

• •The tells cluster where an audience lingers. Coverage of Coca-Cola's all-AI holiday spot cataloged an uncanny gloss, diffuse lighting, rubbery facial features, frequent cuts, and a lack of spoken dialogue²⁹. Those are the artifacts of a long take rather than a short one.
• •The honest nuance cuts both ways. In the same coverage, viewers who were not told the ad was AI-made rated the ad similarly to the original 1995 spot²⁹. The reputational risk lives closer to the disclosure than to the pixels.

8. The agency playbook

Seven chapters of decisions collapse into one working order. Establish what the client wants the system to do, and whether a person signs the result. Wire it to the tools and the documents it needs. Choose the model against the job rather than the leaderboard, and structure the prompt so the standing context is reused. Put the layers in before anything is published, and price the human review as a line item rather than absorbing it. The sequence matters more than any single choice in it, because each step constrains the next.

• •Scope the narrow task, not the category. “An agent for the account” is unquotable. “Draft the weekly performance summary from these three reports, reviewed before it sends” is a piece of work with an edge you can price.
• •Put the review in the estimate. The automated layers cost little; the one that catches what slips past them is somebody's booked hours. Budgeting for it is what makes the rest of the promise honest.
• •Bring a test, not an opinion. Run the client's real task against two candidates and show the result. It ends the leaderboard argument in a meeting rather than a memo.
• •Say what you do not know. The disclosure that ages best is the one that named its limits while the work was being scoped, which is the whole reason the first Field Guide¹ exists.