I am at very starting stage in replicating research paper, could you provide me some tips ?
1 Like
A lot to unpack here. Depends on the paper.
As someone who recently tried to replicate the CANINE paper(you can find it here) , you gotta be prepared to go down the rabbit hole of previous papers mentioned in the paper you’re trying to replicate. For example in the CANINE paper, they use something called hash embeddings. I had no idea what the hell that was. So I had to read that paper and replicate it. Rinse, repeat, and keep reading. You’re learning on top of learning.
Don’t discouraged if it takes over an hour, a day, or even a week. Might be a while. Remember, the people on the paper probably took a long while to write and think about it, so don’t expect it to come easy.
I recommend replicating it in torch since it’s the easiest. If you get stuck, try to look at the code repo if the paper has one. Don’t copy and paste their code unless you get really stuck, but it can help guide you in the right direction as to their thinking.
As a smaller tip, don’t shy away using unoptimized things as well to get it to work. If the paper does some fancy function vectorization, but you find a for loop works/is easier, do that. Just get it to train.
Most importantly, don’t give up! You got this. Don’t know if you need that encouragement, but I’ve been doing this for 6 years now and have a master’s degree and still feel like I don’t know what I’m doing lol. Learning this stuff should be fun and it’s the journey, not the end results.
Happy replicating!
1 Like