alright, so this is the normal DPO equation used to know how far the model we train’s logits difere from the reference:

but can we
add these weights marked in red (between 0-1) to affect how much Yl and Yw contributes?
I know this is totally weird and all, I just want to know yall folks thoughts on this.
I am pretty sure there is already like 10 research papers about this, but I want to know if it will work.
I am 15 so I dont really have a good enough teacher to verify this with, and my ChatGPT doing the ChatGPT thing:
“errrm, you are absolutely right!”
I would really appreciate some insight on this 

I think it’s a good idea. I found something similar
:
One particularly relevant paper is DPO-Shift: Shifting the Distribution of Direct Preference Optimization (2025):
https://arxiv.org/abs/2502.07599
DPO-Shift modifies the standard DPO objective by reducing the contribution of the rejected response relative to the chosen response. The idea is to mitigate likelihood displacement, where even the probabilities of preferred responses can decrease during DPO training.
Your proposal is closely related, but allows separate weights for both sides.
Interestingly, if your two weights are fixed and the chosen weight is positive, you can factor out the chosen weight and absorb it into beta. The ratio between the rejected and chosen weights then determines the asymmetry. So, when the rejected weight is smaller, your formulation is essentially equivalent to a DPO-Shift-style objective with a rescaled beta.
One important distinction, though: setting those weights between 0 and 1 changes the optimization objective, but doesn’t impose a hard limit on how far the model’s logits or probabilities can move.
The paper also discusses a trade-off between preserving the probability of chosen responses and maintaining the preference reward margin.
So I think your idea is worth exploring, especially if you’re interested in how different weight combinations affect training behavior. DPO-Shift could be a useful starting point!
Thanks, mates!
I will definitely read the research paper, and maybe I could research a bit on my idea as well, no?
I feel like having direct control over a limit on how we can control training for DPO (for both accepted and rejected samples) would make a good impact.
Also, this would be a good EC for college as well! 
And once again, thank you so much for the research paper! I was trying to find a paper like this, but I am not good with my keywords.
Not trying to pressure you, but I would really appreciate if I can get some recommendations on similar papers to read!