When serving an MLflow Python model with the "pyfunc" backend (), how can I set a custom gunicorn worker timeout? The default timeout of 60 seconds may be insufficient when serving large models that take a long time to load.
2 Answers
As of MLflow 1.2, you can set a custom gunicorn timeout by specifying the GUNICORN_CMD_ARGS environment variable. The following example serves a model with a worker timeout of 120 seconds
GUNICORN_CMD_ARGS="--timeout 120" mlflow models serve --model-uri /path/to/model
mlflow allows setting these options from the cli:
Example: mlflow models serve ... --timeout 180
Official documentation (mlflow models serve --help):
-t, --timeout TEXT Timeout in seconds to serve a request (default: 60).
