There are two versions of the OpenOrca dataset - one with GPT4 completions and another with GPT 3.5. You can load the former with load_dataset("Open-Orca/OpenOrca", data_dir="001-1M-GPT4-Augmented")
and the latter with load_dataset("Open-Orca/OpenOrca", data_dir="002-3_5M-GPT3_5-Augmented")
.