How to disable caching in .from_pretrained()

skumar1998 · September 1, 2024, 4:46pm

I’m using multiple models inside my docker image and I don’t wanna cache it as it always be downloaded while starting the containers.
Is there a way to ignore this or I have to write manual deletion logic?
Thanks for any help and suggestions!

John6666 · September 1, 2024, 5:02pm

It takes 3 minutes at the earliest to start up, 30 minutes or more at the worst to abort just to generate beautiful sunset, right?
I suffered a lot too.
I only use Gradio, not Docker, for now, but maybe it will help.

Relatively legitimate way

cache_examples=False,

Tweaking that looks like something on the edge

github.com

gradio-app/gradio/blob/main/gradio/external.py

"""This module should not be used directly as its API is subject to change. Instead,
use the `gr.Blocks.load()` or `gr.load()` functions."""

from __future__ import annotations

import json
import os
import re
import tempfile
import warnings
from pathlib import Path
from typing import TYPE_CHECKING, Callable, Literal

import httpx
import huggingface_hub
from gradio_client import Client
from gradio_client.client import Endpoint
from gradio_client.documentation import document
from packaging import version

This file has been truncated. show original

↓

Even in Docker, there should be a line of logic somewhere that specifies the generation of Examples. Just kill that one line.

mikehemberger · September 2, 2024, 6:48am

Hey @skumar1998 ,
You can set
force_download=True

To re-Download the model.
Best,
M

skumar1998 · September 2, 2024, 5:28pm

Hi @mikehemberger
Thanks for your quick reply. But it still stores model weights on cache memory.

Topic		Replies	Views
Cache large models on GPU instances between reboots Spaces	3	814	February 14, 2023
Using gpt-j-6B in a CPU space without the InferenceAPI Spaces	0	2265	January 28, 2022
Manually Downloading Models in docker build with snapshot_download 🤗Transformers	2	14274	December 5, 2022
Is there an API to re-download a model? Beginners	5	1432	January 11, 2024
Sometimes arbitrarily, I get an inference error with some of the examples in my gradio app 🔒 Gradio	2	1162	August 15, 2022

How to disable caching in .from_pretrained()

Relatively legitimate way

Tweaking that looks like something on the edge

Related topics