Mlflow tracking with accelerate

🤗Accelerate
1

Hello,

I have a question regarding mlflow tracking with accelerate library. Writing this code block will error: “INVALID_PARAMETER_VALUE: The experiment was created with a bad artifact location: … It should have a scheme like dbfs:/ or s3:// and that scheme should not be file:/”

accelerator = Accelerator(log_with="mlflow", logging_dir=".")
if accelerator.is_main_process:
    accelerator.init_trackers("accelerate_distributed_run")

Where and how should I set artifact location. When using Trainer class it automatically reads my environment variables for using mlflow on databricks. Here I have no idea which parameters should I put and to what. I apologize, but I didn’t find many answers in the documentation of tracking with Accelerate.

Best regards

2

Update:
I managed to find out the parameters and their values my updated code is below:

accelerator = Accelerator(
        log_with="mlflow",
        logging_dir="***", #  actual databricks url
    )

if accelerator.is_main_process:
    accelerator.init_trackers(
        "experiment-name"
    )

But now here is the catch, I get following error: “RESOURCE_ALREADY_EXISTS: Node named ‘named-entity-recognition-v3’ already exists”.

This experiment truly exists in my databricks workspace and I wish to create a new run under this experiment. Hope there is a way to use this with existing experiments.

Does anyone have the answer to that?

Or should I implement CustomTracker?