How to run t5-3b or t5-11b on Google Ai Notebook?

Models
1

Hey everyone,

I’m curious to try either the 3B or even the big 11B T5 model (preferably in the pipeline) for summarization.

  • Are these two T5 models usable via the pipeline? (my understanding is they are, but maybe I’m failing already at this point)

If they are, my guess is that my VM configuration must not be enough because I can only load “t5-large” on both a google Notebook (15GB vCPU, 1xTesla T4) or my datalore pro account (16GB RAM, 1xTesla T4)

  • Is this enough for the 3B model, or do I need at least 2 GPUs / 32vCPU’s?

Best,
Enrico

1 Like
2

Hi Enrico,
More information about using t5-3b and t5-11b is available on this notebook from the authors:

Looks like you’ll need to pay for a more performant system.

3

Thx!
I was aware of that notebook but could only see the comment about the Colab TPU not having enough memory to fine-tune the 11B model but I thought that there must be some RAM advice on how much you need to run at least the 3B model.
Actually just asked for more vCPU on GCP and got the 3B model running with 32 vCPU if anyone is interested.

1 Like
4

How much is your system RAM that couple with the 32 vCPU?

5

Just an AI Notebook with the N1 32 vCPU - no GPU - on GCP - I think its 120 GB RAM total