Putting Andrew Ng's OCR models to the test

Today, Andrew Ng, one of the legends of the AI world, released a new document extraction service that went viral on X (link here). At Pulse, we put the models to the test with complex financial statements and nested tables – the results were underwhelming to say the least, and suffer from many of the same issues we see when simply dumping documents into GPT or Claude. Our engineering team, along with many X users, discovered alarming issues when testing complex financial statements: Over 50% hallucinated values in complex financial tables in complex financial tables Missing negative signs and currency markers and currency markers Completely fabricated numbers in several instances in several instances 30+ second processing times per document Ground Truth Andrew Ng OCR Output Pulse Output When financial decisions worth millions depend on accurate extraction, these errors aren't just inconvenient – they're potentially catastrophic. Let’s run through some quick math: in a typical ... Read full article.