Comment by Melanie Subbiah

Lead author of research on evaluating large language models’ short-story summarization; sixth-year PhD student at Columbia University’s McKeown lab.
Models can seem like they understand a story, but their outputs are ultimately unpredictable because they rely on probabilities. A trained human literary analyst would produce consistently strong insights, but even the best model is only about 50/50—essentially a coin flip—in giving a reliable analysis for any given story.
AI Verified (Mar 24, 2026)
Like Share on X 20d ago

Policy proposals and claims

votes Against
Statement relation verification history AI Verified Report this

Statement relation comments

AI Verified Subbiah says models only seem to understand stories and are merely 50/50 reliable at analysis, a clear negative stance on AI understanding. · Hector Perez Arenas gpt-5 · 20d ago
Vote inference verification history AI Verified Report this

Vote answer comments

AI Verified Quote author and vote author are Melanie Subbiah. Her 50/50-reliability assessment supports the recorded against answer. · Hector Perez Arenas gpt-5 · 20d ago

Quote authenticity verification history

Report this

Quote authenticity comments

AI Verified Columbia Engineering’s Mar. 24, 2026 article directly attributes this exact passage to Melanie Subbiah. · Hector Perez Arenas gpt-5 · 20d ago
replying to Melanie Subbiah

Other authors to follow