How S&P is using deep web scraping, ensemble learning and Snowflake architecture to collect 5X more data on SMEs

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More The investing world has a significant problem when it comes to data about small and medium-sized enterprises (SMEs). This has nothing to do with data quality or accuracy — it’s the lack of any data at all. Assessing SME creditworthiness has been notoriously challenging because small enterprise financial data is not public, and therefore very difficult to access. S&P Global Market Intelligence, a division of S&P Global and a foremost provider of credit ratings and benchmarks, claims to have solved this longstanding problem. The company’s technical team built RiskGauge, an AI-powered platform that crawls otherwise elusive data from over 200 million websites, processes it through numerous algorithms and generates risk scores. Built on Snowflake architecture, the platform has increased S&P’s coverage of SMEs by 5X. “Our objective was expansion and efficiency,” ... Read full article.