Limitations of AI Circa 2025

Typically produces architecturally sound code; plans multi-file changes, identifies interdependencies, and maintains long-term maintainability. In tests, it handles large refactors (e.g., across 15-20 files) with minimal regression, often generating a step-by-step plan before execution.

Often produces functional code quickly but risks architectural regression (e.g., inline database logic in handlers); better for single-file or routine tasks, but struggles with multi-file coherence without heavy user intervention.

Model generates plausible-looking citations with real author names and journal titles, but DOIs and page numbers are fabricated. 40-60% of generated citations don't exist when verified.

Performance degrades significantly for information in the "middle" of long contexts. Key facts placed at positions 30-70% into the context are retrieved with 20-40% lower accuracy than facts at the beginning or end.

Model confidently produces answer matching the pattern of similar puzzles in training data, missing the unique constraint. When the twist is highlighted, model can solve correctly - showing capability exists but isn't triggered automatically.

Model abandons correct answer and agrees with user's incorrect position, often fabricating justifications. Occurs in ~30% of adversarial challenges even when model was initially correct.

Model correctly reads 95% of printed text but consistently fails on: rotated text, text in colored backgrounds, handwriting, and numbers in dense tables. Errors often plausible-looking (similar characters swapped).

Model provides confident, detailed answers that were accurate for the training period but are now outdated. Rarely acknowledges uncertainty about temporal relevance. Can cause production bugs when outdated API patterns are suggested.