Custom Inference handler.py: FileNotFoundError

Inference Endpoints on the Hub
1

Hi all,

I am trying to deploy my own custom model using handler.py. Here is my code so far:

class EndpointHandler():

    def __init__(self, path=""):

        """ Initialize the model and required parameters."""

        # model_path = "SRx4_EDTB_Div2kFlickr2K.pth"

        config_root = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'configs')
        config_files = [
            os.path.splitext(os.path.basename(v))[0] for v in scandir(config_root)
            if v.endswith('.py')
        ]
        config_file = os.path.basename("./configs/SRx4_EDTB_Div2kFlickr2K.py").split('.')[0]
        assert config_file in config_files, 'Illegal config!'
        module_reg = importlib.import_module(f'configs.{config_file}')
        self.config = getattr(module_reg, 'Config', None)

        self.model = Network(self.config)
        if torch.cuda.is_available():
            self.device = torch.device('cuda')
        else:
            self.device = torch.device('cpu')
        self.model = self.model.to(self.device)

        weights = "SRx4_EDTB_Div2kFlickr2K.pth"

        load_model_filter_list(self.model, weights, filter_list=[])
        self.model.eval()


    def __call__(self, data: Dict[str, Any]) -> List[Dict[str, Any]]:

        """
        Data Args:
            inputs (:obj: `str` | `PIL.Image` | `np.array`)
            kwargs
        Return:
            A :obj:`list` | `dict`: will be serialized and returned
        """

        input_data = data.pop("inputs", data)

        img = torch.from_numpy(np.array(input_data))
        img = img.permute(2, 0, 1).unsqueeze(0) / 255.0
        img = img.to(self.device)
        # if model.parameters().dtype == 'fp16':
        #     input_data = input_data.half()

        scales = []
        for s in self.config.VAL.SCALES:
            scales.append(s)

        lqs, _, _ = preprocess_images(img, scales[0], self.config)

        with torch.no_grad():

            start_time = time.time()
            preds = self.model(lqs)
            torch.cuda.synchronize()
            end_time = time.time()
            print("Inference time: ", end_time - start_time, "seconds")

            result = np.array(preds[0][0,:,:,:].cpu())

        return  {"result" : np.transpose(result, (1,2,0))}

Which gives me this error when I initialize the endpoint:

FileNotFoundError: [Errno 2] No such file or directory: 'SRx4_EDTB_Div2kFlickr2K.pth'

It works on my PC in a container, the file exists, I tried many things already and it doesn’t see my model. What is going on? I don’t know how to solve this issue at this point.

2

Hi,
You are not using the path parameter of the __init__ function of the handler.
It tells you where your model repository is pulled.
So you probably want to construct the path to your weights using this path argument.

3

Tried. Doesn’t work. Works with /repository/your_model.pth, but doesn’t work with ONNX models. It just doesn’t load it, it is not there even though it exists in the repo.

4

Yes I have the same problem. In the logs there is a line that indicates that some extensions are ignored when fetching the repository.
INFO | Ignore regex pattern for files, which are not downloaded: onnx , tar.gz, rust , *mlmodel, tflite, tf , openvino , ckpt, flax , *safetensors
I don’t know why these file extensions are ignored…
One potential workaround is maybe to save the weights file under another extension in the repo, and then at the start of __init__ change it back to its original extension but haven’t tried it yet.

5

I am encountering the same issue as I have a single model.safetensors in my repo that I am trying to load using from_single_file and although everything works perfectly fine testing the EndpointHandler locally once I deploy it I get an error that the provided path is not a valid model file or repository.

6

Facing some problem with onnx model, anyone have a solution?

7

Same here. Lost a day before pointing out that pulling ignoring my onnx files from a huggingface model. What we should do?

8

I don’t know guys. I quit and became a dancer. :joy:

1 Like
9

For anyone struggling with this issue, it seems to come from the “Framework” setting in the “Advanced configuration” panel when creating an inference endpoint. By default, it will select “PyTorch”, which will ignore the mentioned file formats. If you don’t want these file formats to be ignored, just select “Custom” for this field.