Hello! Because its quite slow to load the model to GPU vram I was wondering if there’s a way I can keep the model (falcon7B atm) loaded into the GPU and then access that from a different python program? Different from the one that uploaded it? I basically want to keep the model ready to receive bu…

Can I keep model in GPU vram and then iterate and change my program without re-uploading?

fredrum February 21, 2024, 3:12am 3

Hi and thank you so much for the tip!
I will try this out!

Cheers!

1 Like