Skip to content

Conversation

@gingerXue
Copy link
Contributor

@gingerXue gingerXue commented Jan 7, 2026

Overview

this PR introduces mthreads-ml-py as an explicit dependency and refacts the related detection code accordingly.

  • Please refer the mthreads-ml-py pypi url

What’s Changed

  • Added mthreads-ml-py to pyproject.toml as the official dependency for accessing MThreads runtime information.
  • Refactored the detector structure to make platform-specific logic clearer and more modular.

Future Work

  • Extend the MThreads detector method to provide more comprehensive device information by refering to nvidia's implementations.

Test Result

$ gpustack-runtime detect
+-------------------------------------------------------------+
| MTHREADS Driver Version: 3.3.2-server  Runtime Version: N/A |
|-------------------------------------------------------------|
| GPU |    Name   |    Memory-Usage   | GPU-Util | Temp |  CC |
|-----|-----------|-------------------|----------|------|-----|
| 0   | MTT S5000 | 201MiB / 81920MiB | 0%       | 38C  | N/A |
| 1   | MTT S5000 | 5MiB / 81920MiB   | 0%       | 44C  | N/A |
| 2   | MTT S5000 | 5MiB / 81920MiB   | 0%       | 42C  | N/A |
| 3   | MTT S5000 | 5MiB / 81920MiB   | 0%       | 39C  | N/A |
| 4   | MTT S5000 | 5MiB / 81920MiB   | 0%       | 39C  | N/A |
| 5   | MTT S5000 | 5MiB / 81920MiB   | 0%       | 43C  | N/A |
| 6   | MTT S5000 | 5MiB / 81920MiB   | 0%       | 42C  | N/A |
| 7   | MTT S5000 | 5MiB / 81920MiB   | 0%       | 39C  | N/A |
+-------------------------------------------------------------+

@yeahdongcn
Copy link
Contributor

@thxCode Please take a look. Thanks.

thxCode
thxCode previously approved these changes Jan 7, 2026
Copy link
Contributor

@thxCode thxCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thxCode
Copy link
Contributor

thxCode commented Jan 7, 2026

@gingerXue please execute make lint on your laptop before committing.

image

@thxCode thxCode self-requested a review January 7, 2026 13:12
@thxCode
Copy link
Contributor

thxCode commented Jan 7, 2026

@gingerXue One more thing, can you also check the output of gpustack-runtime topo and gpustack-runtime topo --format json? Thanks.

@gingerXue
Copy link
Contributor Author

$ gpustack-runtime topo
+---------------------------------------------------------------------------------------------------------------------------------+
| MTHREADS | Device 0 | Device 1 | Device 2 | Device 3 | Device 4 | Device 5 | Device 6 | Device 7 | CPU Affinity | NUMA Affinity |
|----------|----------|----------|----------|----------|----------|----------|----------|----------|--------------|---------------|
| Device 0 | X        | NODE     | NODE     | NODE     | SYS      | SYS      | SYS      | SYS      | 0-31,64-95   | 0             |
| Device 1 | NODE     | X        | NODE     | NODE     | SYS      | SYS      | SYS      | SYS      | 0-31,64-95   | 0             |
| Device 2 | NODE     | NODE     | X        | NODE     | SYS      | SYS      | SYS      | SYS      | 0-31,64-95   | 0             |
| Device 3 | NODE     | NODE     | NODE     | X        | SYS      | SYS      | SYS      | SYS      | 0-31,64-95   | 0             |
| Device 4 | SYS      | SYS      | SYS      | SYS      | X        | NODE     | NODE     | NODE     | 32-63,96-127 | 1             |
| Device 5 | SYS      | SYS      | SYS      | SYS      | NODE     | X        | NODE     | NODE     | 32-63,96-127 | 1             |
| Device 6 | SYS      | SYS      | SYS      | SYS      | NODE     | NODE     | X        | NODE     | 32-63,96-127 | 1             |
| Device 7 | SYS      | SYS      | SYS      | SYS      | NODE     | NODE     | NODE     | X        | 32-63,96-127 | 1             |
+---------------------------------------------------------------------------------------------------------------------------------+

Legend (from nearest to farthest):
  X    = Self
  LINK = Connection traversing with High-Speed Link (e.g., AMD XGMI, Ascend HCCS, NVIDIA NVLink)
  PIX  = Connection traversing at most a single PCIe bridge
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  NODE = Connection traversing PCIe and the interconnect between NUMA nodes
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  N/A  = Unknown or unavailable information

$ gpustack-runtime topo --format json
[
  {
    "manufacturer": "mthreads",
    "devices_distances": [
      [
        0,
        40,
        40,
        40,
        50,
        50,
        50,
        50
      ],
      [
        40,
        0,
        40,
        40,
        50,
        50,
        50,
        50
      ],
      [
        40,
        40,
        0,
        40,
        50,
        50,
        50,
        50
      ],
      [
        40,
        40,
        40,
        0,
        50,
        50,
        50,
        50
      ],
      [
        50,
        50,
        50,
        50,
        0,
        40,
        40,
        40
      ],
      [
        50,
        50,
        50,
        50,
        40,
        0,
        40,
        40
      ],
      [
        50,
        50,
        50,
        50,
        40,
        40,
        0,
        40
      ],
      [
        50,
        50,
        50,
        50,
        40,
        40,
        40,
        0
      ]
    ],
    "devices_cpu_affinities": [
      "0-31,64-95",
      "0-31,64-95",
      "0-31,64-95",
      "0-31,64-95",
      "32-63,96-127",
      "32-63,96-127",
      "32-63,96-127",
      "32-63,96-127"
    ],
    "devices_numa_affinities": [
      "0",
      "0",
      "0",
      "0",
      "1",
      "1",
      "1",
      "1"
    ]
  }
]

Copy link
Contributor

@thxCode thxCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thxCode thxCode merged commit 18b9bab into gpustack:main Jan 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants