Where AI Tools Quietly Get Things Wrong (and What to Check by Hand)

The AI failures that actually cost time aren't the ones where the output is obviously nonsense — those get caught immediately and discarded. The expensive ones are confidently wrong in a way that looks exactly like a correct answer, which means you only catch them if you already know to check a specific thing. This isn't a "don't trust AI" piece — I use these tools daily and they save real time. It's a list of the specific spots where they've quietly failed on me or people I work with, organized so you know what to check by hand before you ship, publish, or send whatever came out.

Confidently wrong facts and citations

The clearest version of this: asking for a statistic, a source, or a specific claim about something, and getting back a number or citation that sounds exactly like the kind of thing that would be true, phrased with zero hedging, and doesn't hold up when checked. This happens more with narrow or recent topics than with well-established facts, because the model is filling a gap with the statistically likely shape of an answer rather than a memorized fact.

What actually happened to me: I asked a model for the typical timeout default on a specific cloud provider's managed queue service, got back a specific number stated as fact, and used it in a design doc without checking. A reviewer caught it because the number didn't match their own experience with that service — the real default was different, and the doc had to be corrected before it shaped an actual timeout configuration downstream. The number wasn't a wild hallucination; it was close enough to plausible that it read as confident, correct output.

• Check by hand: any specific number, date, version, or named source — go to the primary documentation or source directly. Treat a fact stated with high confidence and zero hedging as no more trustworthy than one stated with hedging; confidence in the phrasing doesn't correlate with accuracy.
• Check by hand: if you ask for a citation or link and it's not something you can click and verify immediately, don't cite it further downstream until you have.

Outdated library and API knowledge in code suggestions

Code suggestions fail in a specific, recognizable way: the syntax is completely valid, it would have worked correctly at some point, and it's wrong for the version you're actually running. This is different from a bug — it compiles, it might even run, and it can silently use a deprecated pattern or a default that changed between versions.

A concrete example: asking for a Laravel validation rule and getting array-based syntax that was standard convention a couple of major versions back, rather than the class-based rule objects that are the current idiomatic approach. Nothing throws an error — the array syntax still works — but it's not what a reviewer looking at current code would expect, and it can miss newer validation features entirely.

The same pattern shows up with any fast-moving library: a package's configuration format changes across major versions, a cloud SDK renames a method, a framework changes its default behavior for something security-relevant. The model's training data includes a mix of versions, and unless you specify one, you get whatever's statistically most common in what it saw — which may or may not be current.

• Check by hand: always state your exact framework/library version in the prompt if it matters (it usually does), and still verify against that version's current docs for anything security- or data-handling related.
• Check by hand: if a suggested method, config key, or default doesn't appear in your installed version's changelog or docs, don't assume it's a newer feature you haven't heard of — assume it might be an older one that changed.

Math and counting errors

Language models are not calculators, and even when they can technically compute correctly, they're prone to a specific class of error: getting the setup of a problem right and then making an arithmetic or counting slip partway through, especially in multi-step calculations or when counting items in a list they generated themselves.

A small but telling example: asking a model to count how many items in a list it just wrote met a certain condition. It's not unusual for the count in the summary sentence to not match the actual list above it — off by one, because it miscounted rather than because the underlying list logic was wrong. The list itself is often more reliable than any number the model states about the list.

• Check by hand: any arithmetic that matters (costs, capacity estimates, statistical claims) — recompute it yourself or with an actual calculator/spreadsheet, don't trust a stated total.
• Check by hand: if a model summarizes a count from its own output ("so that's 7 items total"), count the actual list yourself rather than trusting the summary number.

Subtle logic bugs that look plausible

This is the hardest category to catch, because the code reads well, follows sensible naming conventions, and does something reasonable — just not quite the thing you asked for. It's the difference between wrong code and code with the wrong edge case.

A real instance: asking for a function to check whether a discount code is still valid, given a start date and an end date. The generated function checked current_date >= start_date and current_date <= end_date — which looks completely correct, and is correct, right up until you consider a discount code with no end date (meant to be open-ended), which the function treated as always invalid because the comparison against a null end date evaluated as false rather than being treated as "no limit." The bug wasn't in the logic anyone would think to scrutinize; it was in the unstated edge case nobody's prompt had covered.

Here's roughly what that looked like escaped as generated:

function isDiscountValid(code) {
  const now = new Date();
  return now >= code.startDate && now <= code.endDate;
  // fails silently when endDate is null — comparison with null
  // does not mean "no limit," it just evaluates unexpectedly
}

• Check by hand: for any generated function handling dates, null/optional values, empty collections, or boundary conditions (zero, negative numbers, first/last item), explicitly test those cases yourself — they're exactly the inputs most likely to be silently mishandled, and least likely to show up in a quick read-through.
• Check by hand: read generated code for what it doesn't handle, not just what it does. Ask directly: "what inputs would break this function?" as a follow-up prompt — models are noticeably better at finding edge cases in code when asked explicitly than at avoiding them in the first draft.

A five-minute pass before you ship anything AI-assisted

None of the checks above need to turn into a formal process to be useful. What's worked for me is a short, specific pass I run before sending, publishing, or merging anything a model helped produce — closer to five minutes than an hour, because it's targeted at the known failure spots rather than a full re-derivation:

• Any number, date, or version claim: did I click through to a primary source, or am I trusting the model's phrasing?
• Any generated code touching dates, null values, or empty collections: did I actually test those specific inputs, not just the happy path?
• Any stated count or total ("that's 6 items", "this affects 3 files"): did I count the actual list myself instead of trusting the summary sentence? For text specifically, a quick pass through an actual word counter takes a few seconds and catches length or count claims that don't hold up — useful before you accept an AI-stated word count on a piece of content at face value.
• Any claim about "what changed": did I diff the before and after myself, rather than trusting the model's own summary of its edit?

The pattern across all four checks is the same one running through this whole post: verify the specific claim, not the general vibe of confidence around it. A model that sounds certain and a model that sounds hedged are not meaningfully different in actual accuracy — the hedging is a style choice, not a calibration signal — so the checklist has to do the work that tone can't.

A verification habit worth building

The common thread across all four categories is the same: the failures aren't random noise, they cluster around specific, predictable spots — unstated versions, unstated edge cases, self-reported counts, and claims without a source you've personally clicked. Once you know the shape of where things go wrong, verification stops being "re-check everything" and becomes a short, specific list you run through for the kind of output you're looking at.

This connects directly to how you prompt in the first place — asking a model to state its assumptions or show its work, as covered in five prompt engineering patterns that actually improve output quality, surfaces a lot of these issues before you even get to the verification step, because a stated assumption is something you can catch, while a silent one usually isn't. And if you're comparing which model to trust for which kind of task in the first place, the tradeoffs in how ChatGPT, Claude, and Gemini actually differ on coding tasks line up closely with the failure patterns here — the models that hedge more also tend to be the ones that flag their own uncertainty instead of quietly guessing.

None of this is an argument against using these tools for real work. It's an argument for knowing exactly where to look before you trust the output, the same way you'd know to check a junior teammate's edge-case handling without needing to re-derive their entire solution from scratch.