Skip to content

Conversation

@Damans227
Copy link
Contributor

@Damans227 Damans227 commented Jan 13, 2026

Description

When registering a CKS data ISO for all zones, if any zone lacks secondary storage, the registration fails but leaves an orphaned ISO in zones that do have storage. This requires manual cleanup.

Fixes: #10994

This PR adds upfront validation to check that secondary storage exists before attempting to register the ISO:

  • If a specific zone is provided, validates that zone has secondary storage
  • If no zone is provided (registering for all zones), validates all zones have secondary storage and reports which zones are missing it.

Fixes: #10994

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

N/A

How Has This Been Tested?

  • Added unit tests for validateImageStoreForZone method.
  • Manual testing against KVM based 4.20 environment.

@Damans227 Damans227 changed the title Add image store validation for Kubernetes version registration CKS: Add image store validation for Kubernetes version registration Jan 13, 2026
Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@codecov
Copy link

codecov bot commented Jan 14, 2026

Codecov Report

❌ Patch coverage is 95.23810% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 16.24%. Comparing base (6a324da) to head (fae86a4).
⚠️ Report is 9 commits behind head on 4.20.

Files with missing lines Patch % Lines
...bernetes/version/KubernetesVersionManagerImpl.java 95.23% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##               4.20   #12418   +/-   ##
=========================================
  Coverage     16.23%   16.24%           
- Complexity    13382    13387    +5     
=========================================
  Files          5657     5657           
  Lines        498999   499045   +46     
  Branches      60566    60568    +2     
=========================================
+ Hits          81035    81053   +18     
- Misses       408928   408956   +28     
  Partials       9036     9036           
Flag Coverage Δ
uitests 4.03% <ø> (ø)
unittests 17.09% <95.23%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vishesh92 vishesh92 linked an issue Jan 15, 2026 that may be closed by this pull request
@apache apache deleted a comment from blueorangutan Jan 15, 2026
@apache apache deleted a comment from blueorangutan Jan 15, 2026
@DaanHoogland
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16374

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16384

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@Damans227 Damans227 requested a review from vishesh92 January 15, 2026 17:38
@vishesh92
Copy link
Member

@blueorangutan package

Copy link
Member

@vishesh92 vishesh92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@blueorangutan
Copy link

[SF] Trillian test result (tid-15193)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 50270 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12418-t15193-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

@DaanHoogland DaanHoogland added this to the 4.20.3 milestone Jan 16, 2026
@RosiKyu RosiKyu self-assigned this Jan 19, 2026
Copy link
Collaborator

@RosiKyu RosiKyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Test Results Summary

Test Case Description Expected Result Status
TC1 Register K8s version in zone WITH secondary storage Registration succeeds, ISO created PASS
TC2 Register K8s version in zone WITHOUT secondary storage Registration fails with clear error, NO orphaned ISO PASS
TC4 Register K8s version for ALL zones (mixed storage) - Original bug scenario Registration fails upfront, NO orphaned ISO in zones with storage PASS
TC5 Register K8s version with directdownload=true in zone without storage Image store validation bypassed PASS

Key Findings

  1. Original bug #10994 is fixed: When registering for all zones with mixed storage, the validation now fails before creating any ISO, preventing orphaned artifacts.

  2. Error messages are clear and informative:

    • Single zone: Unable to register Kubernetes version ISO. No image store available in zone: TestZone-NoStorage
    • All zones: Unable to register Kubernetes version ISO for all zones. The following zones have no image store: TestZone-NoStorage
  3. Direct download correctly bypasses validation: The directdownload=true parameter skips image store validation as expected.

Conclusion

  • All test cases passed. The PR correctly implements upfront
    validation to prevent orphaned ISOs when secondary storage is missing.

Detailed Test Report:

Test Case 1: Register Kubernetes Version ISO in Zone WITH Secondary Storage

Objective: Verify that Kubernetes version ISO registration succeeds when the specified zone has secondary storage configured.

Environment:

  • CloudStack Management Server: ref-trl-10655-k-Mol9-rositsa-kyuchukova-mgmt1
  • CloudStack Version: 4.20 (with PR #12418)
  • Zone: ref-trl-10655-k-Mol9-rositsa-kyuchukova (with NFS secondary storage)
  • Account: admin (Root Admin)

Preconditions:

  • Zone has secondary storage (image store) configured
  • Valid Kubernetes ISO URL available

Test Steps:

  1. Verify no existing Kubernetes supported versions
  2. Register Kubernetes version in zone with storage
  3. Verify registration succeeds and ISO downloads successfully

Expected Result: Registration succeeds, Kubernetes version entry created, ISO status shows "Successfully Installed".

Evidence:

  1. Baseline - no existing Kubernetes versions:
(localcloud) 🐱 > list kubernetessupportedversions
(localcloud) 🐱 >
  1. Register Kubernetes version:
(localcloud) 🐱 > add kubernetessupportedversion semanticversion=1.31.0 name="k8s-1.31.0-TC1" zoneid=3bc00eb5-3419-4cd5-a022-b131b5a58868 url="http://10.0.3.122/kirantemplates/cks-1.31-calico-3.29.0.iso" mincpunumber=2 minmemory=2048
{
  "kubernetessupportedversion": {
    "arch": "x86_64",
    "created": "2026-01-19T22:33:02+0000",
    "directdownload": false,
    "id": "b3a1fbc2-c373-4bd3-b378-e451db1440ec",
    "isoid": "70928a85-180e-406d-bb80-e7acbc1f90f7",
    "isoname": "k8s-1.31.0-TC1-Kubernetes-Binaries-ISO",
    "isostate": "Creating",
    "mincpunumber": 2,
    "minmemory": 2048,
    "name": "k8s-1.31.0-TC1",
    "semanticversion": "1.31.0",
    "state": "Enabled",
    "supportsautoscaling": true,
    "supportsha": true,
    "zoneid": "3bc00eb5-3419-4cd5-a022-b131b5a58868",
    "zonename": "ref-trl-10655-k-Mol9-rositsa-kyuchukova"
  }
}
  1. Verify ISO successfully installed:
(localcloud) 🐱 > list isos isofilter=all keyword="k8s-1.31.0-TC1" filter=id,name,zonename,status
{
  "count": 1,
  "iso": [
    {
      "id": "70928a85-180e-406d-bb80-e7acbc1f90f7",
      "name": "k8s-1.31.0-TC1-Kubernetes-Binaries-ISO",
      "status": "Successfully Installed",
      "zonename": "ref-trl-10655-k-Mol9-rositsa-kyuchukova"
    }
  ]
}

Test Result: PASS - Kubernetes version registered successfully in zone with secondary storage.

Test Case 2: Register Kubernetes Version ISO in Zone WITHOUT Secondary Storage

Objective: Verify that Kubernetes version ISO registration fails with a clear error message when the specified zone lacks secondary storage, and NO orphaned ISO is created.

Environment:

  • CloudStack Management Server: ref-trl-10655-k-Mol9-rositsa-kyuchukova-mgmt1
  • CloudStack Version: 4.20 (with PR #12418)
  • Zone: TestZone-NoStorage (NO secondary storage)
  • Account: admin (Root Admin)

Preconditions:

  • Zone exists without secondary storage configured
  • Valid Kubernetes ISO URL available

Test Steps:

  1. Attempt to register Kubernetes version in zone without storage
  2. Verify operation fails with appropriate error message
  3. Verify NO orphaned ISO was created
  4. Verify NO Kubernetes version entry was created

Expected Result: Registration fails with clear error message. No orphaned ISO or Kubernetes version entry created.

Evidence:

  1. Attempt registration in zone without storage:
(localcloud) 🐱 > add kubernetessupportedversion semanticversion=1.31.1 name="k8s-1.31.1-TC2" zoneid=3128f684-8b61-484b-8f55-a3ca86b74c66 url="http://10.0.3.122/kirantemplates/cks-1.31-calico-3.29.0.iso" mincpunumber=2 minmemory=2048
🙈 Error: (HTTP 530, error code 9999) Unable to register Kubernetes version ISO. No image store available in zone: TestZone-NoStorage
  1. Verify NO orphaned ISO created:
(localcloud) 🐱 > list isos isofilter=all keyword="k8s-1.31.1-TC2"
(localcloud) 🐱 >
  1. Verify NO Kubernetes version entry created:
(localcloud) 🐱 > list kubernetessupportedversions keyword="1.31.1"
(localcloud) 🐱 >

Test Result: PASS - Registration failed with clear error message identifying the zone without storage. No orphaned ISO or Kubernetes version entry was created.

Test Case 4: Register Kubernetes Version ISO for ALL Zones - Mixed Storage (Original Bug Scenario)

Objective: Verify that registering a Kubernetes version ISO for all zones fails upfront with a clear error when ANY zone lacks secondary storage, and NO orphaned ISOs are created in zones that DO have storage. This is the original bug scenario from issue #10994.

Environment:

  • CloudStack Management Server: ref-trl-10655-k-Mol9-rositsa-kyuchukova-mgmt1
  • CloudStack Version: 4.20 (with PR #12418)
  • Zones:
    • ref-trl-10655-k-Mol9-rositsa-kyuchukova (WITH storage)
    • TestZone-NoStorage (WITHOUT storage)
  • Account: admin (Root Admin)

Preconditions:

  • Multiple zones exist, at least one without secondary storage
  • Valid Kubernetes ISO URL available

Test Steps:

  1. Attempt to register Kubernetes version for all zones (no zoneid parameter)
  2. Verify operation fails with appropriate error listing zones without storage
  3. Verify NO orphaned ISO was created in zones that have storage
  4. Verify NO Kubernetes version entry was created

Expected Result: Registration fails upfront before creating any ISO. Error message lists all zones missing storage. No orphaned artifacts.

Evidence:

  1. Attempt registration for all zones:
(localcloud) 🐱 > add kubernetessupportedversion semanticversion=1.31.2 name="k8s-1.31.2-TC4-allzones" url="http://10.0.3.122/kirantemplates/cks-1.31-calico-3.29.0.iso" mincpunumber=2 minmemory=2048
🙈 Error: (HTTP 530, error code 9999) Unable to register Kubernetes version ISO for all zones. The following zones have no image store: TestZone-NoStorage
  1. Verify NO orphaned ISO created:
(localcloud) 🐱 > list isos isofilter=all keyword="k8s-1.31.2-TC4"
(localcloud) 🐱 >
  1. Verify NO Kubernetes version entry created:
(localcloud) 🐱 > list kubernetessupportedversions keyword="1.31.2"
(localcloud) 🐱 >

Test Result: PASS - Registration failed upfront with clear error message listing the zone without storage. CRITICAL: No orphaned ISO was created in the zone that has storage - this confirms the original bug #10994 is fixed.

Test Case 5: Direct Download Bypasses Image Store Validation

Objective: Verify that the image store validation is bypassed when using direct download (directdownload=true), as direct download stores ISOs on KVM hosts, not secondary storage.

Environment:

  • CloudStack Management Server: ref-trl-10655-k-Mol9-rositsa-kyuchukova-mgmt1
  • CloudStack Version: 4.20 (with PR #12418)
  • Zone: TestZone-NoStorage (NO secondary storage, NO hosts)
  • Account: admin (Root Admin)

Preconditions:

  • Zone exists without secondary storage
  • Zone has no KVM hosts configured

Test Steps:

  1. Attempt to register Kubernetes version with directdownload=true in zone without storage
  2. Verify the operation does NOT fail with image store validation error
  3. Confirm failure occurs at a later stage (host validation)

Expected Result: Registration should bypass image store validation and fail at host validation stage instead.

Evidence:

  1. Attempt registration with direct download:
(localcloud) 🐱 > add kubernetessupportedversion semanticversion=1.31.3 name="k8s-1.31.3-TC5-directdownload" zoneid=3128f684-8b61-484b-8f55-a3ca86b74c66 url="http://10.0.3.122/kirantemplates/cks-1.31-calico-3.29.0.iso" mincpunumber=2 minmemory=2048 directdownload=true
🙈 Error: (HTTP 530, error code 9999) Couldn't find a host to validate URL http://10.0.3.122/kirantemplates/cks-1.31-calico-3.29.0.iso

Test Result: PASS - The image store validation was correctly bypassed for direct download. The operation failed at host validation stage (Couldn't find a host) instead of the image store validation, confirming the directdownload=true parameter correctly skips the secondary storage check.

@DaanHoogland DaanHoogland merged commit da518e9 into apache:4.20 Jan 20, 2026
24 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CKS Data ISO registration fails but ISO is registered

5 participants