Oooh I think I figured it out.
I think the correct way to use pipeline() is to do something like the following -
model = AutoModelForCausalLM.from_pretrained("bigscience/bloom", device_map="balanced_low_0", torch_dtype=torch.float16,cache_dir=<path to driec>)
followed by
token = AutoTokenizer.from_pretrained("bigscience/bloom", cache_dir="/home/racball/opt175b_tokeniser")
Then,
generator = pipeline("text-generation", model=model,tokenizer=token)