I/O error calling ToenizersLibrary.createTokenizer in container

🤗Tokenizers
1

I’m using code like:

    static String DJL_MODEL = "intfloat/multilingual-e5-base";
    static String DJL_PATH = "djl://ai.djl.huggingface.pytorch/" + DJL_MODEL;
    static private HuggingFaceTokenizer huggingFaceTokenizer;
...
    static private HuggingFaceTokenizer getHuggingFaceTokenizer() {
        if (huggingFaceTokenizer == null) {
            huggingFaceTokenizer = HuggingFaceTokenizer.newInstance(DJL_MODEL,
                getDJLConfig());
        }
        return huggingFaceTokenizer;
    }

and it works fine running from the command line. However, when executing the same code in a Docker container I am getting an error:

I/O error Permission denied (os error 13)
RuntimeException: I/O error Permission denied (os error 13)
ai.djl.huggingface.tokenizers.jni.TokenizersLibrary.createTokenizer(Native Method)
ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.newInstance(HuggingFaceTokenizer.java: 109)

I’m sure the issue has to do with permissions in the container, but I have no idea what happens when loading the instance so I don’t know which directory is needed to have its attributes changed. Unfortunately, there method resolves to a native library so I can’t figure out what is causing this error.

Any help is greatly appreciated.

2

Set the HF_HOME environment variable went starting the container to a directory that can be written to by the default user in the container.