As long as it can be incorporated into a Batch object, the existing HF framework can take care of it to some extent, but if the preprocessor part is not common, it’s tough…
However, Python has strict restrictions on parallel processing in the parts that can be written in Python, so I think it is better to look for a method that relies on some external library anyway.