Skip to content

Healthcheck for prefect server services #18997

@hanhwanglim

Description

@hanhwanglim

Describe the current behavior

Currently health check is only supported for the main API server services and workers. When setting up a scalable prefect server, the API service is split from the background service. The background services does not have a health check functionality to monitor the status of the service.

One work around to get the health of the background service it to use pgrep -f "prefect server services". However pgrep is not available in the prefecthq/prefect:3-latest image and so the health check will fail. Alternatively, we can run the background service with prefect server services start --background to spawn a new process and try to find the process via ps -p $(cat ~/.prefect/services.pid). However running a background service in docker probably isn't the best.

Describe the proposed behavior

A health check feature for the background service. This could be spawning a lightweight server similar to the --with-healtcheck flag when starting a worker. This could be an additional service PREFECT_SERVER_SERVICES_HEALTHCHECK_ENABLED. Or a quick implementation would be to add pgrep into the prefect docker image.

Example Use

a sample of the docker compose

services:
  prefect-background:
    image: prefecthq/prefect:3-latest
    command: prefect server services start
    environment:
        PREFECT_SERVER_SERVICES_HEALTHCHECK_ENABLED: 1
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request as u; u.urlopen('http://localhost:8080/health', timeout=1)"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAn improvement of an existing feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions