I’m trying to update my zephyr 7B beta inference endpoint to have a larger max-input-size. I believe this model has a max input size of 4k-8k from this discussion, but anytime I try to upgrade it past 2048, the upgrade fails with this message:
Update failed. Endpoint is still available and is using the previously defined configuration. [Server message]Endpoint failed to start. Endpoint failed. Is there any way to introspect into the error to see why the update failed? It also may be likely that I need a more powerful machine - the instance type is on AWS with: GPU · Nvidia A10G · 1x GPU · 24 GB. Any response appreciated. Thanks!