-
Notifications
You must be signed in to change notification settings - Fork 203
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Could there be a bug in the FT implementation #173
Comments
Thanks for your advice, we will modify the training paradigm of |
I have updated the optimization target of FT, you can refer to the latest version of the code |
Hello @pengzju. I'll mark the issue as closed (but I haven't double-checked the code). If you ever rerun the experiments in the survey paper with the fix, I would be interested in knowing how it changes relative performance of fine-tuning. Thank you very much for your rapid response and bug fix. |
Thank you for your suggestion. |
Thank you for your suggestion, we have provided two implementations (
You can choose the appropriate optimization goal based on your experiment settings. Welcome you to try. 😊 |
Thank you very much for raising the issue. We actually encountered this problem in the early experiments last year, but to maintain consistency with previous work ROME, we didn't address it at the time. As @pengzju mentioned, our current approach involves splitting FT into two strategies.
We'll be planning to update the survey paper on Arxiv with new experimental results soon, and we've already noted this issue in the readme. Feeling like in the future, everyone can use this FT-M technique as a strong knowledge editing baseline. Best, EasyEdit Team |
I've found what I think might be a bug in the implementation of the fine-tuning baseline. If this is indeed the case, this bug would yield incorrect results when the unlearning target is longer than one token.
Using the VSCode debugger, I found that the code in
ft_main.py
doesn't carry-out backpropagation properly. The current version of the code passes the prompts without the targets to the model by callingmodel(**inputs)
. It then gathers the logits of all tokens in the target from the last tokens logits. This will maximise the probability of all tokens in the target immediately succedding the prompt. This is not the correct behaviour which should maximise the probability of the first token in the target being a continuation to the input and then maximising the probability of the second token in the target being a continuation to the first token in the target...I think this issue might be in the ROME repository, where the original code came from, where I've written an issue but they haven't responded. Thanks for any assistance you may offer.
The text was updated successfully, but these errors were encountered: