[ET-VK][refactor] Introduce PackedDimInfo struct for packed dimension metadata #16403

SS-JIA · 2025-12-27T01:40:03Z

Stack from ghstack (oldest at bottom):

Context

With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by vTensor used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the GPUMemoryLayout which would be needed to compute storage descriptors such as image extents.

This diff addresses the problem by introducing the PackedDimInfo struct to vTensor, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents.

`PackedDimInfo`

Introduced the PackedDimInfo struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit.

The PackedDimInfo struct contains:

packed_dim: Which dimension is tightly packed (WHCN index) / contiguous in memory
packed_dim_padded: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores
outer_packed_dim: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to packed_dim
outer_packed_dim_padded: Whether outer packed dim is padded (tiled only)

Changes

Added PackedDimInfo struct with helper function calculate_packed_dim_info()
Replaced packed_dim_ member with packed_dim_info_ in vTensor class
Updated function signatures to accept PackedDimInfo& instead of packed_dim_:
- create_hashed_layout
- calculate_dim_order
- calculate_padded_sizes
- calculate_logical_limits
- TextureMetadata constructor/update
- vTensorStorage constructor
Added packed_dim_info() accessor to vTensor and ComputeGraph classes
Store an additional padded_sizes_ member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using sizes_ directly
Introduce memory layouts for kInt8x4 type that only use a single level of packing

Differential Revision: D89832382

… metadata ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) [ghstack-poisoned]

pytorch-bot · 2025-12-27T01:40:07Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16403

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

B200 runners are down due to network issues

❌ 2 New Failures, 1 Unrelated Failure

As of commit 1ad1bef with merge base 0dfda64 ():

NEW FAILURES - The following jobs have failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t 18092e8f4a8d71a4712b018cd0c0b0d6fdb1674d5b56379dbd3459aea5b3c222 /exec failed with exit code 1
pull / test-samsung-quantmodels-linux / linux-job (gh)
RuntimeError: Command docker exec -t d5ff5e83e0ebdee101d15600ecca6d98abd848ba5c1e4e81e747135deb670d01 /exec failed with exit code 1

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

pull / android / run-emulator (gh) (#16137)
Timeout waiting for emulator to boot.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

… metadata ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) ghstack-source-id: 331218513 Pull Request resolved: #16403

github-actions · 2025-12-27T01:40:55Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

backends/vulkan/runtime/api/containers/Tensor.h

backends/vulkan/runtime/api/containers/Tensor.cpp

backends/vulkan/test/vulkan_compute_api_test.cpp

…d dimension metadata" ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) [ghstack-poisoned]

… metadata Pull Request resolved: #16403 ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) ghstack-source-id: 331346332

…d dimension metadata" ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) [ghstack-poisoned]

… metadata Pull Request resolved: #16403 ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing ghstack-source-id: 331360027 Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/)

SS-JIA · 2025-12-29T21:59:19Z

Addressed comments (cc: @mergennachin)

Introduced better enforcement of rules in PackedDimInfo
- Also introduced utility field is_block_packed to make code more readable
Fixed logic of virtual_transpose() - virtual transposition is not allowed if either packed dim is the one being transposed
Introduced comprehensive testing for the metadata of constructed vTensor instances in vulkan_compute_api_test
calculate_*() functions are no longer public - the only callsites were in testing code anyway

mergennachin

There are a few unused variables.

What are your thoughts on adding stricter flag (e.g., -Wunused-parameter) in backends/vulkan/CMakeLists.txt?

backends/vulkan/runtime/api/containers/Tensor.cpp

SS-JIA · 2025-12-29T22:31:18Z

@mergennachin yes, let's do that. Is there a list of additional flags you recommend in addition to -Wunused-parameter? I'll put up a follow up PR to add these flags + relevant fixes

…d dimension metadata" ## Context With the introduction of block-packed memory layouts for quantized tensors, the metadata stored by `vTensor` used to describe the data layout within a texture/buffer was no longer sufficient to form a complete description of the data layout. This created an awkward pattern of needing to estimate the `GPUMemoryLayout` which would be needed to compute storage descriptors such as image extents. This diff addresses the problem by introducing the `PackedDimInfo` struct to `vTensor`, which provides a complete description of how data in a tensor may be organized in the GPU buffer/texture used to store the tensor data, and allows simplification of the functions used to compute buffer numel or texture extents. ## `PackedDimInfo` Introduced the `PackedDimInfo` struct that encapsulates all information about packed dimensions in GPU tensors. This improves code organization and makes the relationship between related metadata fields explicit. The `PackedDimInfo` struct contains: - `packed_dim`: Which dimension is tightly packed (WHCN index) / contiguous in memory - `packed_dim_padded`: Whether the packed dimension is padded to multiple of 4; some layouts will do this to accomodate vectorized load/stores - `outer_packed_dim`: Second-level packing for block-packed layouts (4W4C, 4H4W); for layouts with only a single level of packing, will be equal to `packed_dim` - `outer_packed_dim_padded`: Whether outer packed dim is padded (tiled only) ## Changes - Added PackedDimInfo struct with helper function calculate_packed_dim_info() - Replaced packed_dim_ member with packed_dim_info_ in vTensor class - Updated function signatures to accept PackedDimInfo& instead of packed_dim_: * create_hashed_layout * calculate_dim_order * calculate_padded_sizes * calculate_logical_limits * TextureMetadata constructor/update * vTensorStorage constructor - Added packed_dim_info() accessor to vTensor and ComputeGraph classes - Store an additional `padded_sizes_` member in vTensor which is now used to for strides/image extents/GPU buffer numel computation, instead of using `sizes_` directly - Introduce memory layouts for kInt8x4 type that only use a single level of packing Differential Revision: [D89832382](https://our.internmc.facebook.com/intern/diff/D89832382/) [ghstack-poisoned]

SS-JIA · 2025-12-30T03:20:14Z

Changes after accept: small change to Concat.cpp so that it doesn't need to call calculate_padded_sizes; this allows all calculate_*() utility functions to be made private.

Also resolved unused parameters issues.

mergennachin · 2026-01-02T16:44:32Z

Is there a list of additional flags you recommend in addition to -Wunused-parameter

@SS-JIA What about just -Wall and -Wextra (which will include unused-parameter)? arm, samsung and qualcomm backends have these flags enabled fwiw

https://github.com/pytorch/executorch/blob/main/backends/arm/CMakeLists.txt
https://github.com/pytorch/executorch/blob/main/backends/samsung/CMakeLists.txt
https://github.com/pytorch/executorch/blob/main/backends/qualcomm/CMakeLists.txt

SS-JIA mentioned this pull request Dec 27, 2025

[ET-VK][testing] Fix qdq8ta_conv2d_activations test failures via configurable shader filtering #16402

Merged

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 27, 2025

meta-codesync bot added fb-exported meta-exported labels Dec 27, 2025

mergennachin requested changes Dec 29, 2025

View reviewed changes

SS-JIA requested a review from mergennachin December 29, 2025 22:00

mergennachin approved these changes Dec 29, 2025

View reviewed changes

backends/vulkan/runtime/api/containers/Tensor.cpp Outdated Show resolved Hide resolved

backends/vulkan/runtime/api/containers/Tensor.cpp Outdated Show resolved Hide resolved

SS-JIA mentioned this pull request Dec 30, 2025

[ET-VK][testing] Fix on-device tests #16412

Merged

meta-codesync bot merged commit d1a6c5d into gh/SS-JIA/388/base Dec 30, 2025
162 of 166 checks passed

meta-codesync bot deleted the gh/SS-JIA/388/head branch December 30, 2025 04:04

meta-codesync bot temporarily deployed to cherry-pick-bot December 30, 2025 04:04 Inactive

pytorchbot mentioned this pull request Dec 30, 2025

[ET-VK][refactor] Introduce PackedDimInfo struct for packed dimension metadata #16414

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ET-VK][refactor] Introduce PackedDimInfo struct for packed dimension metadata #16403

[ET-VK][refactor] Introduce PackedDimInfo struct for packed dimension metadata #16403

Uh oh!

SS-JIA commented Dec 27, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Dec 27, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Dec 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SS-JIA commented Dec 29, 2025

Uh oh!

mergennachin left a comment

Uh oh!

Uh oh!

Uh oh!

SS-JIA commented Dec 29, 2025

Uh oh!

SS-JIA commented Dec 30, 2025

Uh oh!

Uh oh!

mergennachin commented Jan 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[ET-VK][refactor] Introduce PackedDimInfo struct for packed dimension metadata #16403

[ET-VK][refactor] Introduce PackedDimInfo struct for packed dimension metadata #16403

Uh oh!

Conversation

SS-JIA commented Dec 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

PackedDimInfo

Changes

Uh oh!

pytorch-bot bot commented Dec 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16403

❗ 1 Active SEVs

❌ 2 New Failures, 1 Unrelated Failure

Uh oh!

github-actions bot commented Dec 27, 2025

This PR needs a release notes: label

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SS-JIA commented Dec 29, 2025

Uh oh!

mergennachin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

SS-JIA commented Dec 29, 2025

Uh oh!

SS-JIA commented Dec 30, 2025

Uh oh!

Uh oh!

mergennachin commented Jan 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SS-JIA commented Dec 27, 2025 •

edited

Loading

`PackedDimInfo`

pytorch-bot bot commented Dec 27, 2025 •

edited

Loading

This PR needs a `release notes:` label