When Not to Use Machine Learning (and What to Build Instead)

A team I worked near once spent six weeks building a model to predict shipping cost for an online order. Six weeks of feature engineering, cross-validation, and hyperparameter tuning, to approximate something the carrier's own published rate table already computed exactly, for free, with zero training data and zero drift risk. The rate table won by every measure that mattered: accuracy, latency, explainability, and maintenance cost. This kind of thing happens more often than the "AI-first" framing of most tooling would have you believe, and it's worth naming the specific situations where reaching for ML is the wrong instinct, not just a suboptimal one.

The default instinct, and why it's often backwards

When a problem involves predicting something, the modern reflex is to frame it as a machine learning task before asking whether prediction is even the right frame. But plenty of problems that sound like prediction are actually computation ("what does the tax code say I owe"), lookup ("what's this customer's current plan"), or a decision small enough to write down as a handful of if/then rules ("flag orders over $500 shipping to a new address for manual review"). Machine learning earns its cost when the relationship between inputs and outputs is genuinely too complex, too high-dimensional, or too poorly understood to write down by hand. When it isn't, ML adds a training pipeline, a monitoring burden, and an explainability gap to a problem that didn't need any of them.

Case 1: You don't actually have enough data

Say a small support team wants to predict which incoming tickets will need engineering escalation, based on their last 40 resolved tickets. Forty labeled examples is not a training set for a machine learning model with any real generalization power — it's barely enough to eyeball a pattern manually. A model trained on 40 rows will either overfit hard to quirks specific to those 40 tickets, or need so much regularization that it collapses into predicting the majority class every time, similar to the do-nothing baseline described in the precision and recall post.

What to build instead: a short scoring checklist written by the two people who've actually triaged escalations for the last year — "customer is on an enterprise plan," "ticket mentions data loss or an outage," "similar wording to a past P1" — each worth a point, escalate above a threshold. It's auditable, it works on day one, and every escalation decision it makes has a reason a human wrote down and can defend. Once you've accumulated a few hundred or thousand labeled tickets from actually running that checklist, revisiting ML becomes a reasonable conversation again.

Case 2: The problem is actually deterministic

Shipping cost, tax owed, unit conversions, interest accrual, whether a coupon code applies to a given cart — these all have exact, specified answers computed from a known formula or a published table. There's no uncertainty to model, because there's no noise in the relationship between inputs and outputs; it's a calculation, not a prediction.

What to build instead: the actual formula, or the rate table, implemented directly. If the carrier gives you a rate table by weight and zone, store that table and look values up. It'll be exactly correct, instant to compute, and require zero retraining when nothing about the underlying rule has changed. The only time this shifts toward ML is when you're trying to predict something adjacent but genuinely uncertain, like how long a shipment will actually take door to door once real-world delays are involved — that's a forecasting problem with real variance, not a formula.

Case 3: You need to explain every decision, not just predict it well

Loan approvals, medical triage priority, and content moderation strikes are all situations where "the model said so" is not an acceptable answer to the person on the receiving end, and in some jurisdictions and industries it isn't a legally acceptable answer either. Even models that are technically explainable in principle (like a single decision tree) tend to get replaced in practice by ensembles or neural networks for a small accuracy bump, at the cost of anyone being able to say plainly why a specific person was declined.

What to build instead: a scorecard model, the same style banks used long before modern ML — a fixed set of weighted factors (income, existing debt, payment history) that sum to a score, with a documented threshold. It's less flexible than a gradient boosted tree, and it will leave some predictive accuracy on the table. In exchange, every decision comes with a clear, defensible answer: "your score was 580, driven mainly by X and Y," which a human can check, appeal, and reason about. If your specific situation genuinely doesn't require that level of explainability, this trade-off may not apply to you, but it's worth confirming that on purpose rather than by default.

Case 4: A lookup table or simple heuristic already covers 95% of it

I've seen teams build classifiers to detect whether a piece of text is written in English before running further processing on it, when a short list of common English stopwords and a length check would have caught the overwhelming majority of cases correctly, in microseconds, with no training data required at all. The same pattern shows up in "is this email address probably valid," "is this a business day," and "does this input look like a phone number" — problems people occasionally reach for ML on despite a regular expression or a small hardcoded rule already solving them almost entirely.

What to build instead: the heuristic, written down as normal code, with the edge cases it misses noted explicitly. If you're prototyping this kind of "when X happens, do Y" flow and don't want to hand-write the branching logic yourself, a rules and workflow tool like our AI Automation Builder can wire up that kind of conditional logic quickly, without needing a labeled training set to get started. Save the ML budget for the genuinely ambiguous 5% of cases the heuristic can't resolve, if that 5% turns out to matter enough to justify it.

Case 5: Someone needs the answer faster than a model can respond, at a cost you can't absorb

A real-time bidding system deciding whether to serve an ad has single-digit milliseconds to decide, at a volume of thousands of requests per second. A heavyweight model with meaningful inference latency, or one that requires a network round-trip to a hosted endpoint, can simply be the wrong shape for that constraint regardless of how accurate it is offline — the business requirement is speed and cost at volume, not marginal accuracy. This case gets missed because teams often benchmark a model's accuracy carefully and its latency and infrastructure cost almost as an afterthought, only discovering the mismatch once it's serving real traffic.

What to build instead: a lighter, faster heuristic or a much smaller model that trades some accuracy for speed, or precomputed lookups for the most common cases with the heavier logic reserved only for genuinely novel inputs. Measuring latency and cost per prediction alongside accuracy, from the start of a project rather than after deployment, catches this mismatch while it's still cheap and early enough in the project to change direction without throwing away finished work.

What actually justifies reaching for ML

None of this is an argument against machine learning generally — it's an argument against reaching for it as the default. ML earns its complexity when most of these are true at once:

• The input-output relationship is genuinely complex or high-dimensional enough that nobody can write it down as a formula or a short rule list.
• You have a meaningful amount of representative historical data, not a few dozen examples.
• Some wrong predictions are an acceptable cost of doing business — the domain tolerates probabilistic answers rather than requiring exact or fully explainable ones.
• Someone is going to own monitoring the model over time, because unlike a rate table, its accuracy can silently degrade as the world changes.
• The gain over a simpler baseline has actually been measured, not assumed — the honest baseline-first habit described in the tiny end-to-end pipeline post applies just as much to "should we use ML at all" as it does to "which model should we use."
• The cost of a wrong prediction is genuinely lower than the cost of the engineering and monitoring overhead required to keep a model healthy in production over time.

A quick check before you start training anything

1. Can I write the correct answer down as a formula or table? If yes, stop — you don't need a model.
2. Do I have at least a few hundred representative labeled examples, ideally more? If no, start with a heuristic and collect data.
3. Does someone need a plain-language reason for every individual decision? If yes, lean toward a scorecard or rules system, not a black-box model.
4. Would a simple rule genuinely get most of the way there? If yes, ship that first and measure the gap before building anything more complex.
5. Can the whole system tolerate the extra latency and infrastructure cost a model adds? If speed or budget is tight, that constraint alone can rule ML out regardless of accuracy.

The shipping-cost project I mentioned at the start eventually got replaced with the carrier's own rate table, and it took an afternoon instead of six weeks. The model wasn't bad engineering — the team that built it was good at their job, and the cross-validated accuracy numbers in their write-up were genuinely solid. It was simply the wrong job to be doing at all, and that's a more common outcome across this industry than most ML-first advice tends to admit out loud.