New Apple AI model generates 3D scenes from just three images

Apple’s Machine Learning team, in collaboration with researchers from Nanjing University and The Hong Kong University of Science and Technology, has announced an interesting 3D AI model called Matrix3D. This so-called Large Photogrammetry Model is able to reconstruct 3D objects and scenes from just a few 2D photos, but with a big difference from current pipelines. Here’s why this is a big deal. First things first: photogrammetry. It uses photographs to make measurements in order to create 3D models or maps. Currently, this process involves using different models for steps like pose estimation and depth prediction, which can lead to inefficiencies and errors. Matrix3D simplifies this by doing it all in one go. It takes in images, camera parameters (such as angle and focal length), and depth data, and processes them using a unified architecture. This not only simplifies the workflow but also improves accuracy. Even more interesting is how the model was trained. Researchers used a mas ... Read full article.