Hi @vblagoje,
That is correct that the integration with HuggingFace considers that a typical run is linked to one process. However the intent is to allow full flexibility for custom cases with environmental variables.
You can for example resume a previous run from a different process (see resuming docs).
Would you be able to make a simple colab/notebook so I understand better you use case?