Skip to content

Conversation

@Yu-Jack
Copy link
Collaborator

@Yu-Jack Yu-Jack commented Jan 21, 2026

Problem:

Solution:

Related Issue(s):

Test plan:

Additional documentation or context

@Yu-Jack Yu-Jack self-assigned this Jan 21, 2026
@github-actions
Copy link

github-actions bot commented Jan 21, 2026

Name Link
🔨 Latest commit bc03698
😎 Deploy Preview https://697184e802b930a8c677dc2c--harvester-preview.netlify.app

@Yu-Jack Yu-Jack marked this pull request as ready for review January 21, 2026 06:54
Copilot AI review requested due to automatic review settings January 21, 2026 06:54
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds documentation for a known upgrade issue related to the cpu-feature.node.kubevirt.io/ipred-ctrl=true CPU feature that appears during upgrades from v1.5.x to v1.6.x on specific Intel Xeon CPU models. The documentation provides workarounds for users before and after the upgrade.

Changes:

  • Added a new section (§8) documenting the ipred-ctrl CPU feature issue during upgrades
  • Updated subsequent section numbering (previous §8 becomes §9)
  • Provided specific CPU models affected and multiple resolution options

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
docs/upgrade/v1-5-x-to-v1-6-x.md Added new section 8 documenting ipred-ctrl CPU feature issue and renumbered subsequent sections
versioned_docs/version-v1.6/upgrade/v1-5-x-to-v1-6-x.md Added identical documentation for v1.6 versioned docs
versioned_docs/version-v1.7/upgrade/v1-5-x-to-v1-6-x.md Added identical documentation for v1.7 versioned docs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Signed-off-by: Jack Yu <jack.yu@suse.com>
**After the upgrade:**
- Reboot the virtual machines.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if node-labeller was disabled, then VM's do not need to rebooted after the upgrade as selector labels matching cpu features will not have changed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right, done!

Yu-Jack and others added 2 commits January 22, 2026 09:45
Co-authored-by: Gaurav Mehta <gaurav.mehta@suse.com>
Signed-off-by: Jack Yu <jack.yu@suse.com>
Signed-off-by: Jack Yu <jack.yu@suse.com>
Copy link
Contributor

@jillian-maroket jillian-maroket left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review done

### 8. Change in Default VLAN Behavior for Secondary Pod Interfaces
### 8. `cpu-feature.node.kubevirt.io/ipred-ctrl=true` CPU Feature Appears During Upgrade
When upgrading from v1.5.x to v1.6.x, Harvester live migrates virtual machines to another node to complete the node upgrade. During this process, if your CPU model is in the following list, the `cpu-feature.node.kubevirt.io/ipred-ctrl=true` feature temporarily appears until the upgrade is complete.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When upgrading from v1.5.x to v1.6.x, Harvester live migrates virtual machines to another node to complete the node upgrade. During this process, if your CPU model is in the following list, the `cpu-feature.node.kubevirt.io/ipred-ctrl=true` feature temporarily appears until the upgrade is complete.
Harvester live migrates virtual machines to ensure zero downtime during node upgrades. If your cluster uses any of the following CPU models, you may notice a temporary feature flag (`cpu-feature.node.kubevirt.io/ipred-ctrl=true`) appear while the upgrade is in progress.

- Intel(R) Xeon(R) Gold 5418Y
- Intel(R) Xeon(R) Silver 4509Y
Because the `cpu-feature.node.kubevirt.io/ipred-ctrl=true` feature is no longer available after the upgrade, these virtual machines cannot be live migrated because of the mismatched node selector.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Because the `cpu-feature.node.kubevirt.io/ipred-ctrl=true` feature is no longer available after the upgrade, these virtual machines cannot be live migrated because of the mismatched node selector.
While this feature flag is automatically removed from nodes after the upgrade, the corresponding node selector is retained in the virtual machine configuration. This mismatch between the virtual machine's requirements and the node's labels causes subsequent live migrations to fail.

Comment on lines +274 to +279
To resolve this issue, use one of the following options:
**Before the upgrade:**
- **Option 1:** Configure a CPU model for the virtual machines using [this guideline](https://harvesterhci.io/kb/setup_common_cpu_model_for_vm_live_migration).
- **Option 2:** Stop the KubeVirt labeller by [adding the `node-labeller.kubevirt.io/skip-node` annotation to the nodes](https://kubevirt.io/user-guide/compute/virtual_hardware/#labeling-nodes-with-cpu-models-cpu-features-and-machine-types), and then remove the annotation after the upgrade completes. This option is more complex but useful if the virtual machines cannot be rebooted. For more information, see the [knowledge base article](https://harvesterhci.io/kb/troubleshooting_vm_scheduling_issues_nodeselector).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To resolve this issue, use one of the following options:
**Before the upgrade:**
- **Option 1:** Configure a CPU model for the virtual machines using [this guideline](https://harvesterhci.io/kb/setup_common_cpu_model_for_vm_live_migration).
- **Option 2:** Stop the KubeVirt labeller by [adding the `node-labeller.kubevirt.io/skip-node` annotation to the nodes](https://kubevirt.io/user-guide/compute/virtual_hardware/#labeling-nodes-with-cpu-models-cpu-features-and-machine-types), and then remove the annotation after the upgrade completes. This option is more complex but useful if the virtual machines cannot be rebooted. For more information, see the [knowledge base article](https://harvesterhci.io/kb/troubleshooting_vm_scheduling_issues_nodeselector).
To resolve this issue, perform either of the following actions _before starting the upgrade_:
- **Option 1:** [Configure a common CPU model](https://harvesterhci.io/kb/setup_common_cpu_model_for_vm_live_migration) for the virtual machines.
- **Option 2:** Stop the KubeVirt labeller by [adding the `node-labeller.kubevirt.io/skip-node` annotation to the nodes](https://kubevirt.io/user-guide/compute/virtual_hardware/#labeling-nodes-with-cpu-models-cpu-features-and-machine-types), and then remove the annotation after the upgrade. While more complex, this option is useful if the virtual machines cannot be rebooted. For more information, see [Troubleshooting VM Live Migration Issues Caused by Node Selectors](https://harvesterhci.io/kb/troubleshooting_vm_scheduling_issues_nodeselector).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants