When does transformers support pipeline parallelism?

Will transformers support pipeline parallelism?