understanding 1, 2, 3
Maybe true.
batching
Great! I didn’t know it…
Rate limits
This seems to change depending on the current situation, so there is no clear information, but my personal impression is that it is relatively strict for the Free Plan. Even with the Pro Plan, it does not seem to be unlimited.
If you want unlimited usage, you will probably have to consider a Dedicated Endpoint.
Machine cost per second
Could this be it…?
I have never seen any information that seems to be definitively correct on this matter.
When the Inference Provider is HF, is it okay to assume that it is fluid as to which machine a given model will actually be hosted on? @meganariley