Can you trust an AI summary? What it gets right, and what it quietly drops

An AI summarizer is one of the genuinely useful things a language model does. Paste a long article, a dense email thread or a page of notes and get the gist back in seconds. It is also, in a specific and predictable way, unreliable, and the unreliability is easy to miss because a summary always reads clean and confident. This is an honest account of what a summary keeps, what it quietly drops, and how to use one so it saves you time without costing you the detail that mattered.

What it is genuinely good at

Start with the fair part, because the tool earns its place. A good model is reliable at the shape of a text: the main argument, the broad structure, the handful of points a piece is built around. That makes a summary excellent for triage, deciding whether a long article is worth your half hour, catching up on a thread you were cc’d into late, or turning a rambling set of notes into a few clean bullets. For getting oriented fast, it is hard to beat, and for low-stakes text where being roughly right is enough, a summary is often all you need.

What it quietly drops

A summary is compression, and compression is loss. The question is not whether it drops things, it always does, but what it drops, and here the pattern is consistent enough to plan around. Four kinds of detail go missing more than any others:

• Conditions and caveats. The claim survives, the clause that limits it does not. An "only if", an "except when" or a "provided that" is exactly the kind of qualifier a short summary sheds, and losing it can invert the meaning of the sentence it was attached to.
• Negations and hedging. A careful "we did not find", "this is not yet confirmed" or "results were mixed" can be flattened into something more definite than the source ever said, so the summary sounds more certain than the thing it summarizes.
• Attribution. Once the "according to" is gone, a single study, a quoted opinion or one person’s forecast can read as established fact. The summary tells you what was claimed and drops who claimed it and how strongly.
• The dissent. A summary reports the dominant thread and tends to drop the counterargument, the minority view or the caveat paragraph near the end, so a balanced piece can come back sounding one-sided.

There is also a rarer but sharper failure: a model can state something that was not in the source at all, usually a plausible-looking number, name or date. This is the effect commonly called hallucination, and it matters most precisely where you are least likely to catch it, in a specific figure you did not have in front of you. The safe assumption is that any concrete fact in a summary is unverified until you have seen it in the original.

The deeper problem: it cannot know what you needed

The failures above are about the text. The bigger one is about you. A summarizer optimizes for what is generally salient in a document, not for the single detail that matters to your particular question, and it has no way to know which detail that is. A model summarizing a lease will faithfully surface the rent and the term and may drop the one clause about pets or subletting that was the entire reason you were reading. The summary is not wrong. It is answering a more general question than the one you had.

That reframes how to use the tool. A blanket "summarize this" gets you the document’s priorities. A pointed question, "what does this say about cancellation and notice", "does this mention any exceptions to the refund policy", puts your priority inside the target, so the detail you care about is what the model is trying to keep rather than what it compresses away. When a summary keeps missing your point, the fix is usually a sharper question, not a longer summary.

When to rely on it, and when not to

The line is not about the tool being good or bad, it is about what a dropped detail costs. Lean on a summary freely when being roughly right is fine and you can sanity-check it against what you already know:

• Deciding whether a long piece is worth reading in full.
• Catching the gist of a thread, a set of notes or a report in a domain you know well enough to spot an off note.
• Getting a map of a long document so you know which sections to actually read.

Do not rely on a summary alone when one missing clause or one wrong number changes the decision:

• Contracts, terms of service, legal, medical or financial documents, where a single caveat or exact figure is the whole point.
• Anything you are going to quote, forward as fact, or act on with money or reputation attached.
• Anything where the omission itself is the danger, because the summary reads clean exactly by leaving out the clause that would have made you stop.

How to get the time saving without the risk

The point is not to distrust the tool, it is to use it for the job it is good at and verify the one thing it is not. A workflow that holds up: summarize to triage and to navigate, then read the passages that actually bear on your decision. Ask targeted questions instead of requesting a generic summary. Keep the summary next to the source, not as a replacement for it. And spot-check every specific, any number, name, date or negation, against the original before you rely on it, because those are exactly the elements a summary is most likely to bend.

The summarizer on this site runs on Claude and returns a clear bullet-point condensation of whatever you paste, and it says the same thing in plainer words: it is a best-effort condensation, so for anything critical, check it against the original. Used that way, as a fast way in rather than the last word, an AI summary is one of the better time savers you have. Used as a substitute for reading the part that mattered, it is a quiet way to miss it.