A model alignment method that learns from preference pairs using a direct optimization objective.
Sign in to saveDirect Preference Optimization (DPO) uses relative preferences between candidate responses to adapt a language model. The paper derives an objective that avoids a separately trained reward model and the reinforcement-learning stage used in some alignment pipelines. Its effectiveness still depends on the quality and coverage of the preference data.
Research paper first submitted to arXiv on 2023-05-29.
Useful when comparing alignment training recipes. DPO changes model behavior through training; it is distinct from simply ranking answers at inference time. Catalog note: this is a source-verified reference entry, not a measured current-trend signal. No search-volume or growth claim is made.
Pro opens the same-day prioritize / watch / skip table, including names that are still too new to publish here.
This name has not appeared in an imported daily report yet.