Roberta nsp prediction

scienceapptest · June 11, 2021, 9:59pm

Is it possible to train RoBERTA for next sentence prediction like it has been done for ALBERT?

NadirEM/nlp-notebooks/blob/master/Fine_tune_ALBERT_sentence_pair_classification.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Fine_tune_ALBERT_sentence_pair_classification.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyNlbkBhVp6bNo8ze7MJzkQI",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "6f797e5377fe4ab281003d537c15f279": {
          "model_module": "@jupyter-widgets/controls",

This file has been truncated. show original

Topic		Replies	Views
Finetuning ALBERT with custom unlabeled dataset for next sentence prediction task Beginners	0	279	September 2, 2020
Ask for help with prediction results of Named Entity Recognition Task 🤗Transformers	10	2805	May 21, 2021
Finetuning RoBERTa on SentiHood Dataset producing Random Outputs Models	0	339	December 24, 2021
Multilingual NER pretrained model fine tuning Models	0	185	December 9, 2023
Pretraining RoBERTa (or another 'flavor') on permuted word orders Flax/JAX Projects	3	361	July 4, 2021

Roberta nsp prediction

Related Topics