Azure inference endpoint SSE response

Volker-ep January 25, 2024, 2:03am 1

I believe the Azure pods have response caching turned on, so any SSE service will not behave properly. This hasn’t been an issue on the AWS pods

Topic		Replies	Views
Is it possible to have streaming responses from inference endpoints? Inference Endpoints on the Hub	6	1384	July 24, 2023
Inference Endpoint not starting on HTTP request Inference Endpoints on the Hub	2	144	March 6, 2024
Inference endpoint + Dataset Inference Endpoints on the Hub	0	323	March 7, 2023
Inference Endpoint not stable Inference Endpoints on the Hub	3	203	March 18, 2024
Inference endpoints cost Beginners	1	421	November 1, 2022