Roberta nsp prediction

scienceapptest · June 11, 2021, 9:59pm

Is it possible to train RoBERTA for next sentence prediction like it has been done for ALBERT?

NadirEM/nlp-notebooks/blob/master/Fine_tune_ALBERT_sentence_pair_classification.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Fine_tune_ALBERT_sentence_pair_classification.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyNlbkBhVp6bNo8ze7MJzkQI",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "6f797e5377fe4ab281003d537c15f279": {
          "model_module": "@jupyter-widgets/controls",

This file has been truncated. show original

Topic		Replies	Views
BERT Next Sentence Prediction: How to do predictions? Beginners	5	6737	September 29, 2022
Finetuning ALBERT with custom unlabeled dataset for next sentence prediction task Beginners	0	300	September 2, 2020
Pretraining ALBERT Intermediate	2	1302	February 16, 2022
BERT pre-training run_mlm_flax.py questions Beginners	0	250	November 3, 2021
Does anyone else observer RoBERTa fine-tuning instability? 🤗Transformers	8	2925	April 20, 2023

Roberta nsp prediction

Related topics