Why is use_cache incompatible with gradient checkpointing?

ratishsp · June 7, 2022, 10:25am

Hi,
I see the below snippet in modeling_t5.py. I wanted to understand why use_cache is incompatible with gradient checkpointing.

github.com

huggingface/transformers/blob/9aa230aa2fb17e4a7da4c61fccaddafe410f7ed1/src/transformers/models/t5/modeling_t5.py#L1008-L1011

      
        
            if use_cache:
                logger.warning(
                    "`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`..."
                )

Thanks.

lianghsun · June 8, 2022, 5:46am

Hi, there. I face the same problem in run_clm.py when I set --gradient_checkpointing true. However I do not find any config that I can set on run_clm.py, does anyone know?

luisarmando · October 27, 2022, 9:11pm

Anybody knows how to fix this?

rs2992 · March 24, 2023, 4:33pm

Honestly, I’ve just ignored it. It automatically disables the use_cache function. I’m about to remove the warning altogether.

use_cache=True is incompatible with gradient checkpointing. Setting use_cache=False