Skip to content

feat: always include files from diff in overlay changed files#3554

Open
sam-robson wants to merge 1 commit intomainfrom
sam-robson/overlay-include-diff
Open

feat: always include files from diff in overlay changed files#3554
sam-robson wants to merge 1 commit intomainfrom
sam-robson/overlay-include-diff

Conversation

@sam-robson
Copy link
Contributor

Ensure that overlay changes always include all PR diff files used for diff-informed analysis.

Risk assessment

For internal use only. Please select the risk level of this change:

  • Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Workflow types:

  • Managed - Impacts users with dynamic workflows (Default Setup, Code Quality, ...).

Products:

  • Code Scanning - The changes impact analyses when analysis-kinds: code-scanning.
  • Code Quality - The changes impact analyses when analysis-kinds: code-quality.

Environments:

  • Dotcom - Impacts CodeQL workflows on github.com and/or GitHub Enterprise Cloud with Data Residency.
  • GHES - Impacts CodeQL workflows on GitHub Enterprise Server.

How did/will you validate this change?

  • Unit tests - I am depending on unit test coverage (i.e. tests in .test.ts files).
  • End-to-end tests - I am depending on PR checks (i.e. tests in pr-checks).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

  • Feature flags - All new or changed code paths can be fully disabled with corresponding feature flags.

How will you know if something goes wrong after this change is released?

  • Telemetry - I rely on existing telemetry or have made changes to the telemetry.
    • Dashboards - I will watch relevant dashboards for issues after the release. Consider whether this requires this change to be released at a particular time rather than as part of a regular release.
    • Alerts - New or existing monitors will trip if something goes wrong with this change.

Are there any special considerations for merging or releasing this change?

  • No special considerations - This change can be merged at any time.
  • Special considerations - This change should only be merged once certain preconditions are met. Please provide details of those or link to this PR from an internal issue.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change.
  • Confirm the readme and docs have been updated if necessary.

@github-actions github-actions bot added the size/M Should be of average difficulty to review label Mar 6, 2026
@sam-robson sam-robson force-pushed the sam-robson/overlay-include-diff branch from ecde788 to d135b9c Compare March 6, 2026 16:11
Base automatically changed from kaspersv/move-diff-range-absolute-path-conversion to main March 18, 2026 12:02
@sam-robson sam-robson force-pushed the sam-robson/overlay-include-diff branch from d135b9c to 0b55f65 Compare March 24, 2026 10:23
@sam-robson sam-robson marked this pull request as ready for review March 24, 2026 10:44
@sam-robson sam-robson requested a review from a team as a code owner March 24, 2026 10:44
Copilot AI review requested due to automatic review settings March 24, 2026 10:44
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures overlay analysis always includes all files present in the PR diff by persisting PR diff-range data during init, reusing it during analyze, and merging diff-derived paths into the overlay “changed files” list to cover cases like revert PRs.

Changes:

  • Persist PR diff ranges to pr-diff-range.json during init when diff-informed analysis is enabled.
  • Update analyze to read precomputed diff ranges from disk instead of recomputing via API calls.
  • Merge diff-derived file paths into overlay changed files and add unit tests covering diff-only/revert-like cases.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/overlay/index.ts Merges OID-changed files with PR diff-derived file paths for overlay changes.
src/overlay/index.test.ts Adds tests verifying diff-derived files are merged/deduplicated and behavior is unchanged when the diff file is absent.
src/init-action.ts Computes and persists PR diff ranges during init (non-fatal on failure).
src/analyze.ts Switches diff-informed setup to read precomputed diff ranges from disk.
src/analyze-action.ts Simplifies diff-informed setup call to rely on precomputed diff file presence.
lib/* Generated JavaScript output corresponding to the TypeScript changes.
Comments suppressed due to low confidence (1)

src/overlay/index.ts:185

  • getDiffRangeFilePaths will throw if pr-diff-range.json is missing required fields, malformed JSON, or not an array. Since this file is optional and lives in the temp directory, it would be safer for overlay mode to treat parse/shape errors as non-fatal (log a warning/debug and return []) so overlay database creation doesn’t fail due to a stale/corrupted temp file.
function getDiffRangeFilePaths(logger: Logger): string[] {
  const jsonFilePath = path.join(getTemporaryDirectory(), "pr-diff-range.json");
  if (!fs.existsSync(jsonFilePath)) {
    return [];
  }
  const diffRanges = JSON.parse(
    fs.readFileSync(jsonFilePath, "utf8"),
  ) as Array<{ path: string }>;
  logger.debug(
    `Read ${diffRanges.length} diff range(s) from ${jsonFilePath} for overlay changes.`,
  );
  return [...new Set(diffRanges.map((r) => r.path))];

Comment on lines +138 to +143
// Merge in any file paths from precomputed PR diff ranges to ensure the
// overlay always includes all files from the PR diff, even in edge cases
// like revert PRs where OID comparison shows no change.
const diffRangeFiles = getDiffRangeFilePaths(logger);
const changedFiles = [...new Set([...oidChangedFiles, ...diffRangeFiles])];

Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pr-diff-range.json paths are repo-root relative (from the GitHub compare API), but computeChangedFiles returns paths relative to sourceRoot (because getFileOidsUnderPath(sourceRoot) returns paths relative to basePath). Merging these lists as-is will produce incorrect overlay paths when source-root is not the repository root, and may cause the CLI to treat files as missing/outside the source root.

Consider converting diff-range paths to be relative to sourceRoot (and filtering out files not under sourceRoot) before merging, e.g. by deriving the repo root/checkout_path and stripping the sourceRoot prefix in a platform-independent way.

This issue also appears on line 174 of the same file.

Copilot uses AI. Check for mistakes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☝️ this seems like a real bug to me

Comment on lines +240 to +246
const diffRanges = readDiffRangesJsonFile(logger);
if (diffRanges === undefined) {
logger.info(
"No precomputed diff ranges found; skipping diff-informed analysis stage.",
);
return undefined;
}
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setupDiffInformedQueryRun now relies on readDiffRangesJsonFile, which does an unconditional JSON.parse. If the temp file exists but is corrupted/partially written, this will throw and abort the analyze step. Since diff-informed analysis is an optional optimization, consider catching parse errors here (or making readDiffRangesJsonFile return undefined on parse failure) and falling back to full analysis with a warning.

Copilot uses AI. Check for mistakes.
@sam-robson sam-robson force-pushed the sam-robson/overlay-include-diff branch from 0b55f65 to 335dcf5 Compare March 24, 2026 14:17
@github-actions github-actions bot added size/L May be hard to review and removed size/M Should be of average difficulty to review labels Mar 24, 2026
@sam-robson sam-robson force-pushed the sam-robson/overlay-include-diff branch from 335dcf5 to 35dd583 Compare March 24, 2026 14:40
@sam-robson sam-robson force-pushed the sam-robson/overlay-include-diff branch from 0fc9e98 to cfa00b1 Compare March 24, 2026 15:09
sourceRoot: string,
logger: Logger,
): Promise<string[]> {
const jsonFilePath = path.join(getTemporaryDirectory(), "pr-diff-range.json");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should probably be extracted to a getPrDiffRangeFile function to avoid repeating the magic constant file name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L May be hard to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants