Tech News
← Home  ·  All topics

Cross Entropy Loss

1 GoKawiil brief on this topic · RSS for cross entropy loss

OpenAI proposes perplexity as cross-industry AI progress metric

OpenAI published a blog post arguing that perplexity, or cross-entropy loss, should be adopted as a standard metric for comparing AI model developers and informing shared safety standards. As part of this effort, the company disclosed some of its own internal perplexity measurements from pretraining, a move it says required weighing competitive risk against the benefits of industry transparency. The post frames pretraining as the creation of a hard-to-predict 'intelligence substrate,' with later techniques like finetuning used to draw out specific capabilities.