-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
Describe the current behavior
Currently health check is only supported for the main API server services and workers. When setting up a scalable prefect server, the API service is split from the background service. The background services does not have a health check functionality to monitor the status of the service.
One work around to get the health of the background service it to use pgrep -f "prefect server services". However pgrep is not available in the prefecthq/prefect:3-latest image and so the health check will fail. Alternatively, we can run the background service with prefect server services start --background to spawn a new process and try to find the process via ps -p $(cat ~/.prefect/services.pid). However running a background service in docker probably isn't the best.
Describe the proposed behavior
A health check feature for the background service. This could be spawning a lightweight server similar to the --with-healtcheck flag when starting a worker. This could be an additional service PREFECT_SERVER_SERVICES_HEALTHCHECK_ENABLED. Or a quick implementation would be to add pgrep into the prefect docker image.
Example Use
a sample of the docker compose
services:
prefect-background:
image: prefecthq/prefect:3-latest
command: prefect server services start
environment:
PREFECT_SERVER_SERVICES_HEALTHCHECK_ENABLED: 1
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request as u; u.urlopen('http://localhost:8080/health', timeout=1)"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
Additional context
No response