How to run on CPU?

lucadini · April 1, 2022, 9:03am

Hi everyone,

I’m currently trying to modify the token classification script. The machine where I’m running the script has a GPU that is currently fully utilized by another process, so I’d like to run my classification script on the CPU (I’m just editing things, not actually running the training) and only switch to the GPU when I’m done editing. I’ve noticed that other scripts in examples have the --no_cuda parameter, but this one doesn’t. How can I make this script run in my CPU?

Thanks!
Irene

BramVanroy · April 1, 2022, 9:32am

You’ll have to force the accelerator to run on CPU.

github.com

huggingface/transformers/blob/9de70f213eb234522095cc9af7b2fac53afc2d87/examples/pytorch/token-classification/run_ner_no_trainer.py#L230

      
        
                if args.push_to_hub:
                    assert args.output_dir is not None, "Need an `output_dir` to create a repo when `--push_to_hub` is passed."
            
            
    return args
            
            

            
def main():
                args = parse_args()
            
            
    # Initialize the accelerator. We will let the accelerator handle device placement for us in this example.
                accelerator = Accelerator()
                # Make one log on every process with the configuration for debugging.
                logging.basicConfig(
                    format="%(asctime)s - %(levelname)s - %(name)s - %(message)s",
                    datefmt="%m/%d/%Y %H:%M:%S",
                    level=logging.INFO,
                )
                logger.info(accelerator.state)
            
            
    # Setup logging, we only want one process per machine to log things on the screen.
                # accelerator.is_local_main_process is only True for one process per machine.

accelerator = Accelerator(cpu=True)

should do the trick.

Topic		Replies	Views
Pytorch NLP model doesn’t use GPU when making inference 🤗Transformers	5	12613	January 5, 2024
BLOOM models don't run on my GPU using Transformers 🤗Transformers	1	1500	September 18, 2022
Is Transformers using GPU by default? Beginners	6	120509	December 11, 2023
Run pre-trained LLM model on CPU - ValueError: Expected a cuda device, but got: cpu Beginners	0	106	April 17, 2024
Using GPU with transformers Beginners	4	7668	November 3, 2020

How to run on CPU?

Related Topics