From 2113ce2f93a065779db02aedc2e6091c0cb0bce4 Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Sun, 8 Mar 2026 18:10:00 +0100 Subject: [PATCH 1/9] Refactor Copilot synchronization and auditing processes - Enhanced the workflow for syncing Copilot configurations by adding source auditing steps to detect legacy aliases, operational overlaps, and redundant asset references. - Updated AGENTS.md and SKILL.md templates to improve clarity and maintainability, including a new naming policy and decision priority for agent behavior. - Improved validation processes by ensuring legacy aliases and conflicts are flagged during synchronization, preventing the creation of duplicate configurations. - Added comprehensive tests to validate the new auditing features and ensure proper handling of legacy configurations and agent overlaps. - Removed redundant sections from AGENTS.md and streamlined the available skills and prompts documentation. --- .github/CHANGELOG.md | 9 + .../tech-ai-customization-auditor.agent.md | 3 + .../tech-ai-sync-copilot-configs.agent.md | 16 +- .../tech-ai-sync-copilot-configs.prompt.md | 10 +- .../scripts/tech-ai-sync-copilot-configs.py | 701 ++++++++++++++++-- .../tech-ai-sync-copilot-configs/SKILL.md | 27 +- .github/templates/AGENTS.template.md | 62 +- AGENTS.md | 182 ++--- tests/test_tech_ai_sync_copilot_configs.py | 214 +++++- ...tech_ai_validate_copilot_customizations.py | 2 + 10 files changed, 1025 insertions(+), 201 deletions(-) diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index 28633da..691c77b 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -6,6 +6,15 @@ Use this format for new updates: - One bullet per meaningful change. - Include file/path scope when useful. +## 2026-03-08 +- Added source-side redundancy auditing to `scripts/tech-ai-sync-copilot-configs.py`, including canonical asset inventory, legacy alias detection, triad role-overlap checks, and `AGENTS.md` inventory-repeat detection in both markdown and JSON reports. +- Refactored `agents/tech-ai-sync-copilot-configs.agent.md`, `skills/tech-ai-sync-copilot-configs/SKILL.md`, and `prompts/tech-ai-sync-copilot-configs.prompt.md` so workflow detail lives in the skill while the agent and prompt stay thin. +- Simplified root `AGENTS.md` and `.github/templates/AGENTS.template.md` to keep asset paths in the inventory section only and remove descriptive prompt or skill catalogs. +- Expanded sync and validator tests to cover source audit behavior, slimmer AGENTS structure, and JSON report sections. +- Updated `agents/tech-ai-sync-copilot-configs.agent.md`, `skills/tech-ai-sync-copilot-configs/SKILL.md`, and `prompts/tech-ai-sync-copilot-configs.prompt.md` so the sync workflow explicitly detects redundant legacy aliases before apply. +- Updated `scripts/tech-ai-sync-copilot-configs.py` to recognize legacy `cs-*`, unprefixed prompt names, and legacy agent or skill aliases, report them as redundant target assets, and raise sync conflicts instead of creating duplicate canonical `tech-ai-*` assets. +- Updated `tests/test_tech_ai_sync_copilot_configs.py` to cover duplicate-alias detection and conflict behavior during sync planning. + ## 2026-03-07 - Added repo-only global customization agents `TechAIGlobalCustomizationBuilder` and `TechAIGlobalCustomizationAuditor` for standards-authoring and final quality gates in this repository. - Marked `TechAICustomizationAuditor` as a deprecated compatibility alias that now points to `TechAIGlobalCustomizationAuditor`. diff --git a/.github/agents/tech-ai-customization-auditor.agent.md b/.github/agents/tech-ai-customization-auditor.agent.md index 9873ae6..e34447c 100644 --- a/.github/agents/tech-ai-customization-auditor.agent.md +++ b/.github/agents/tech-ai-customization-auditor.agent.md @@ -6,6 +6,9 @@ tools: ["search", "problems", "fetch"] # TechAI Customization Auditor Agent +## Status +Deprecated compatibility alias for `TechAIGlobalCustomizationAuditor`. Use the global auditor for new customization work in this repository. + ## Objective Keep this repository portable and coherent by checking that customization assets are generic, internally consistent, and validator-compliant. diff --git a/.github/agents/tech-ai-sync-copilot-configs.agent.md b/.github/agents/tech-ai-sync-copilot-configs.agent.md index 83f3663..06aa8f8 100644 --- a/.github/agents/tech-ai-sync-copilot-configs.agent.md +++ b/.github/agents/tech-ai-sync-copilot-configs.agent.md @@ -15,16 +15,16 @@ Analyze a local target repository, select the minimum Copilot customization asse - Do not overwrite unmanaged divergent files. - Keep repository-facing text in English and use GitHub Copilot terminology only. -## Workflow -1. Inspect the target repository layout, manifests, `.github` contents, `AGENTS.md` location, and local git state. -2. Classify the target repository against `repo-profiles.yml` and extend with stack-specific rules only when necessary. -3. Select the minimum Copilot core asset set from the source repository. -4. Render target-specific content, especially `AGENTS.md`. -5. Run `.github/scripts/tech-ai-sync-copilot-configs.py` in `plan` mode first and use `apply` only when requested and conflict-safe. -6. Produce a final report with applied, skipped, unchanged, and conflicted items plus source-repository recommendations. +## Routing +- Use this agent only for cross-repository Copilot-core alignment work. +- Treat `.github/skills/tech-ai-sync-copilot-configs/SKILL.md` as the single workflow definition. +- Treat `.github/scripts/tech-ai-sync-copilot-configs.py` as the deterministic execution path. +- Start with `plan` mode and move to `apply` only on explicit request and only when the plan is conflict-safe. -## Output format +## Output Contract - `Target analysis`: repo shape, selected profile, stacks, git state, and AGENTS location. +- `Source audit`: canonical assets, legacy aliases, role overlaps, AGENTS.md repeats, and source-side recommendations. - `Asset selection`: instructions, prompts, skills, agents, and baseline files chosen from the source repository. +- `Redundant target assets`: canonical assets that would duplicate legacy aliases or already coexist with them in the target repository. - `File actions`: create, update, adopt, unchanged, and conflict results. - `Recommendations`: categorized source-repository improvements. diff --git a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md index af26e48..fba1178 100644 --- a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md +++ b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md @@ -20,15 +20,17 @@ Use this prompt to analyze a local repository, select the minimum Copilot custom ## Instructions 1. Use `.github/skills/tech-ai-sync-copilot-configs/SKILL.md` as the workflow definition. 2. Use `.github/scripts/tech-ai-sync-copilot-configs.py` for deterministic execution. -3. Keep scope limited to Copilot core assets only. -4. Preserve unmanaged target files and report conflicts instead of overwriting them. -5. Render the target `AGENTS.md` with GitHub Copilot wording only. -6. Keep recommendation categories stable across runs. +3. Start with `mode=plan`; use `mode=apply` only when explicitly requested and only after a conflict-safe plan. +4. Keep scope limited to Copilot core assets only. +5. Preserve unmanaged target files and report conflicts instead of overwriting them. +6. Report source-side audit findings separately from target-side redundant assets and file actions. ## Minimal example - Input: `target_repo=/workspace/consumer-repo mode=plan report_format=md` - Expected output: - Target analysis summary with detected profile and stacks. + - Source configuration audit for canonical assets, legacy aliases, role overlaps, and AGENTS.md repeats. + - Redundant target assets that would duplicate canonical sync output. - Conservative file action plan for Copilot core assets only. - Recommendations for improving the source standards repository. diff --git a/.github/scripts/tech-ai-sync-copilot-configs.py b/.github/scripts/tech-ai-sync-copilot-configs.py index 4605278..f6b2092 100644 --- a/.github/scripts/tech-ai-sync-copilot-configs.py +++ b/.github/scripts/tech-ai-sync-copilot-configs.py @@ -13,8 +13,10 @@ import datetime as dt import hashlib import json +import re import subprocess import sys +from collections import defaultdict from dataclasses import dataclass, field from pathlib import Path @@ -71,6 +73,20 @@ } STACK_PRIORITY = ("terraform", "python", "nodejs", "java", "bash") PROMPT_SKILL_REFERENCE_PREFIX = ".github/" +EXTRA_LEGACY_ALIAS_PATHS = { + ".github/prompts/tech-ai-bash-script.prompt.md": ( + ".github/prompts/script-bash.prompt.md", + ), + ".github/prompts/tech-ai-python-script.prompt.md": ( + ".github/prompts/script-python.prompt.md", + ), + ".github/prompts/tech-ai-github-action.prompt.md": ( + ".github/prompts/cicd-workflow.prompt.md", + ), +} +ROLE_OVERLAP_SECTION_PREFIXES = ("workflow", "instructions", "validation") +ROLE_OVERLAP_LINE_THRESHOLD = 3 +AGENTS_INVENTORY_CATEGORIES = ("instructions", "prompts", "skills", "agents") def log_info(message: str) -> None: @@ -172,6 +188,8 @@ class SyncPlan: analysis: TargetAnalysis selection: AssetSelection actions: list[FileAction] + redundant_assets: list["RedundantAsset"] + source_audit: "SourceAudit" recommendations: dict[str, list[str]] manifest_relative_path: str @@ -183,6 +201,71 @@ def action_counts(self) -> dict[str, int]: return counts +@dataclass +class RedundantAsset: + category: str + canonical_target_path: str + existing_target_paths: list[str] + issue_type: str + selected_for_sync: bool + + @property + def reason(self) -> str: + listed = ", ".join(self.existing_target_paths) + if self.issue_type == "sync_would_duplicate": + return ( + "Equivalent legacy asset(s) already exist in target: " + f"{listed}. Syncing `{self.canonical_target_path}` would create redundant configuration." + ) + + return ( + "Equivalent assets from the same capability family already coexist in target: " + f"{listed}. Consolidate to one canonical asset before continuing." + ) + + +@dataclass +class CanonicalAssetGroup: + category: str + family: str + paths: list[str] + + @property + def has_physical_duplicates(self) -> bool: + return len(self.paths) > 1 + + +@dataclass +class LegacyAlias: + category: str + canonical_path: str + alias_paths: list[str] + + +@dataclass +class RoleOverlap: + family: str + asset_paths: list[str] + shared_instruction_count: int + examples: list[str] + + +@dataclass +class AgentsMdRepeat: + reference: str + sections: list[str] + count: int + + +@dataclass +class SourceAudit: + canonical_assets: list[CanonicalAssetGroup] + legacy_aliases: list[LegacyAlias] + role_overlaps: list[RoleOverlap] + agents_md_repeats: list[AgentsMdRepeat] + recommendations: list[str] + + def sha256_text(value: str) -> str: return hashlib.sha256(value.encode("utf-8")).hexdigest() @@ -502,8 +585,36 @@ def detect_git_state(repo_root: Path) -> tuple[bool, list[str]]: return bool(lines), lines +def legacy_alias_paths_for_canonical_path(canonical_relative_path: str) -> list[str]: + aliases = set(EXTRA_LEGACY_ALIAS_PATHS.get(canonical_relative_path, ())) + path = Path(canonical_relative_path) + parent = path.parent.as_posix() + name = path.name + + if canonical_relative_path.endswith(".prompt.md") and name.startswith("tech-ai-"): + remainder = name[len("tech-ai-") :] + aliases.add(f"{parent}/{remainder}") + aliases.add(f"{parent}/cs-{remainder}") + elif name == "SKILL.md" and path.parent.name.startswith("tech-ai-"): + legacy_dir = path.parent.name[len("tech-ai-") :] + aliases.add(f"{path.parent.parent.as_posix()}/{legacy_dir}/SKILL.md") + elif canonical_relative_path.endswith(".agent.md") and name.startswith("tech-ai-"): + aliases.add(f"{parent}/{name[len('tech-ai-') :]}") + + return sorted(aliases) + + +def known_legacy_alias_paths(source_root: Path) -> set[str]: + aliases: set[str] = set() + for path in scan_repo_files(source_root): + relative_path = str(path.relative_to(source_root)) + aliases.update(legacy_alias_paths_for_canonical_path(relative_path)) + return aliases + + def detect_target_only_assets(source_root: Path, target_root: Path) -> dict[str, list[str]]: result: dict[str, list[str]] = {"prompts": [], "skills": [], "agents": []} + known_aliases = known_legacy_alias_paths(source_root) for category, pattern in (("prompts", "*.prompt.md"), ("skills", "SKILL.md"), ("agents", "*.agent.md")): target_dir = target_root / ".github" / category source_dir = source_root / ".github" / category @@ -512,11 +623,298 @@ def detect_target_only_assets(source_root: Path, target_root: Path) -> dict[str, source_names = {path.name for path in source_dir.rglob(pattern)} if source_dir.is_dir() else set() for path in sorted(target_dir.rglob(pattern)): + relative_path = str(path.relative_to(target_root)) + if relative_path in known_aliases: + continue if path.name not in source_names: - result[category].append(str(path.relative_to(target_root))) + result[category].append(relative_path) return result +def is_canonical_source_asset(relative_path: str) -> bool: + category = asset_category(relative_path) + path = Path(relative_path) + if category == "instructions": + return path.name.endswith(".instructions.md") + if category == "prompts": + return path.name.startswith("tech-ai-") and path.name.endswith(".prompt.md") + if category == "agents": + return path.name.startswith("tech-ai-") and path.name.endswith(".agent.md") + if category == "skills": + return path.name == "SKILL.md" and path.parent.name.startswith("tech-ai-") + return False + + +def canonical_family_name(relative_path: str) -> str: + category = asset_category(relative_path) + path = Path(relative_path) + if category == "instructions": + name = path.name[: -len(".instructions.md")] + elif category == "prompts": + name = path.name[: -len(".prompt.md")] + elif category == "agents": + name = path.name[: -len(".agent.md")] + elif category == "skills": + name = path.parent.name + else: + name = path.stem + + if name.startswith("tech-ai-"): + return name[len("tech-ai-") :] + return name + + +def detect_canonical_asset_groups(source_root: Path) -> list[CanonicalAssetGroup]: + grouped_paths: dict[tuple[str, str], list[str]] = defaultdict(list) + for path in scan_repo_files(source_root): + relative_path = str(path.relative_to(source_root)) + if not is_canonical_source_asset(relative_path): + continue + key = (asset_category(relative_path), canonical_family_name(relative_path)) + grouped_paths[key].append(relative_path) + + groups = [ + CanonicalAssetGroup(category=category, family=family, paths=sorted(paths)) + for (category, family), paths in sorted(grouped_paths.items()) + ] + return groups + + +def detect_source_legacy_aliases(source_root: Path, canonical_assets: list[CanonicalAssetGroup]) -> list[LegacyAlias]: + aliases: list[LegacyAlias] = [] + for group in canonical_assets: + if group.category not in {"prompts", "skills", "agents"}: + continue + canonical_path = group.paths[0] + present_aliases = [ + alias_path + for alias_path in legacy_alias_paths_for_canonical_path(canonical_path) + if (source_root / alias_path).is_file() + ] + if not present_aliases: + continue + aliases.append( + LegacyAlias( + category=group.category, + canonical_path=canonical_path, + alias_paths=sorted(present_aliases), + ) + ) + return aliases + + +def detect_role_overlaps(source_root: Path, canonical_assets: list[CanonicalAssetGroup]) -> list[RoleOverlap]: + family_paths: dict[str, list[str]] = defaultdict(list) + for group in canonical_assets: + if group.category not in {"prompts", "skills", "agents"}: + continue + family_paths[group.family].extend(group.paths) + + overlaps: list[RoleOverlap] = [] + for family, relative_paths in sorted(family_paths.items()): + if len(relative_paths) < 2: + continue + + shared_lines: dict[str, set[str]] = defaultdict(set) + examples_by_line: dict[str, str] = {} + for relative_path in sorted(relative_paths): + for normalized, original in extract_operational_lines(source_root / relative_path).items(): + shared_lines[normalized].add(relative_path) + examples_by_line.setdefault(normalized, original) + + examples = [ + examples_by_line[normalized] + for normalized, paths in sorted(shared_lines.items()) + if len(paths) >= 2 + ] + if len(examples) < ROLE_OVERLAP_LINE_THRESHOLD: + continue + + overlaps.append( + RoleOverlap( + family=family, + asset_paths=sorted(relative_paths), + shared_instruction_count=len(examples), + examples=examples[:5], + ) + ) + + return overlaps + + +def extract_operational_lines(path: Path) -> dict[str, str]: + lines: dict[str, str] = {} + inside_frontmatter = False + current_heading = "" + for raw_line in path.read_text(encoding="utf-8").splitlines(): + stripped = raw_line.strip() + if stripped == "---": + inside_frontmatter = not inside_frontmatter + continue + if inside_frontmatter: + continue + if stripped.startswith("#"): + current_heading = stripped.lstrip("#").strip().lower() + continue + if not should_audit_operational_section(current_heading): + continue + + item_text = extract_markdown_list_text(stripped) + if not item_text: + continue + + normalized = normalize_instruction_text(item_text) + if len(normalized.split()) < 4: + continue + lines.setdefault(normalized, item_text) + return lines + + +def should_audit_operational_section(heading: str) -> bool: + return any(heading.startswith(prefix) for prefix in ROLE_OVERLAP_SECTION_PREFIXES) + + +def extract_markdown_list_text(line: str) -> str: + if not line: + return "" + if line.startswith("- ") or line.startswith("* "): + return line[2:].strip() + + match = re.match(r"^\d+\.\s+(.*)$", line) + if match: + return match.group(1).strip() + + return "" + + +def normalize_instruction_text(value: str) -> str: + lowered = value.lower().replace("`", "") + lowered = re.sub(r"[^a-z0-9]+", " ", lowered) + return re.sub(r"\s+", " ", lowered).strip() + + +def detect_agents_markdown_repeats(source_root: Path) -> list[AgentsMdRepeat]: + agents_path = source_root / "AGENTS.md" + if not agents_path.is_file(): + return [] + + current_h2 = "Introduction" + current_section = current_h2 + references: dict[str, list[str]] = defaultdict(list) + + for raw_line in agents_path.read_text(encoding="utf-8").splitlines(): + stripped = raw_line.strip() + if stripped.startswith("## "): + current_h2 = stripped[3:].strip() + current_section = current_h2 + elif stripped.startswith("### "): + current_section = f"{current_h2} / {stripped[4:].strip()}" + + for token in re.findall(r"`([^`]+)`", stripped): + normalized = normalize_agents_inventory_reference(token) + if not normalized: + continue + references[normalized].append(current_section) + + repeats: list[AgentsMdRepeat] = [] + for reference, sections in sorted(references.items()): + unique_sections = list(dict.fromkeys(sections)) + if len(unique_sections) < 2: + continue + repeats.append( + AgentsMdRepeat( + reference=reference, + sections=unique_sections, + count=len(sections), + ) + ) + return repeats + + +def normalize_agents_inventory_reference(value: str) -> str | None: + cleaned = value.strip() + if cleaned.startswith("./"): + cleaned = cleaned[2:] + if cleaned.startswith(".github/"): + normalized = cleaned + elif any(cleaned.startswith(f"{category}/") for category in AGENTS_INVENTORY_CATEGORIES): + normalized = f".github/{cleaned}" + else: + return None + + if not any(f"/{category}/" in normalized for category in AGENTS_INVENTORY_CATEGORIES): + return None + return normalized + + +def build_source_audit_recommendations( + canonical_assets: list[CanonicalAssetGroup], + legacy_aliases: list[LegacyAlias], + role_overlaps: list[RoleOverlap], + agents_md_repeats: list[AgentsMdRepeat], +) -> list[str]: + recommendations: list[str] = [] + + duplicate_families = [group for group in canonical_assets if group.has_physical_duplicates] + if duplicate_families: + summary = ", ".join( + f"{group.category}:{group.family}" for group in duplicate_families[:5] + ) + recommendations.append( + "Consolidate physical duplicate canonical asset families so each capability resolves to one real file: " + f"{summary}." + ) + + if legacy_aliases: + summary = ", ".join(alias.canonical_path for alias in legacy_aliases[:5]) + recommendations.append( + "Remove or clearly deprecate source-side legacy aliases so the standards repository ships one canonical " + f"`tech-ai-*` family per capability: {summary}." + ) + + if role_overlaps: + summary = ", ".join(overlap.family for overlap in role_overlaps[:5]) + recommendations.append( + "Keep detailed workflow and validation steps in the matching skill only; agent and prompt files should " + f"remain thin entrypoints for: {summary}." + ) + + if agents_md_repeats: + summary = ", ".join(repeat.reference for repeat in agents_md_repeats[:5]) + recommendations.append( + "Keep asset path references in `AGENTS.md` inventory only and use capability names elsewhere to avoid " + f"documentation repeats: {summary}." + ) + + if not recommendations: + recommendations.append( + "No source-side redundancy detected in canonical assets, legacy aliases, triad roles, or AGENTS.md " + "inventory references." + ) + + return recommendations + + +def audit_source_configuration(source_root: Path) -> SourceAudit: + canonical_assets = detect_canonical_asset_groups(source_root) + legacy_aliases = detect_source_legacy_aliases(source_root, canonical_assets) + role_overlaps = detect_role_overlaps(source_root, canonical_assets) + agents_md_repeats = detect_agents_markdown_repeats(source_root) + recommendations = build_source_audit_recommendations( + canonical_assets, + legacy_aliases, + role_overlaps, + agents_md_repeats, + ) + return SourceAudit( + canonical_assets=canonical_assets, + legacy_aliases=legacy_aliases, + role_overlaps=role_overlaps, + agents_md_repeats=agents_md_repeats, + recommendations=recommendations, + ) + + def build_analysis(source_root: Path, target_root: Path, profiles: dict[str, RepoProfile]) -> TargetAnalysis: files = scan_repo_files(target_root) stacks, unsupported_stacks, extension_counts = detect_stacks(target_root, files) @@ -750,6 +1148,94 @@ def build_planned_files( return sorted(planned_files, key=lambda item: item.target_relative_path) +def detect_redundant_assets(target_root: Path, selection: AssetSelection) -> list[RedundantAsset]: + selected_paths = set(selection.managed_source_paths) + redundant_assets: list[RedundantAsset] = [] + + canonical_candidates = { + path + for path in selected_paths + if asset_category(path) in {"prompts", "skills", "agents"} + } + canonical_candidates.update( + str(path.relative_to(target_root)) + for path in scan_repo_files(target_root) + if asset_category(str(path.relative_to(target_root))) in {"prompts", "skills", "agents"} + ) + + for canonical_relative_path in sorted(canonical_candidates): + legacy_aliases = legacy_alias_paths_for_canonical_path(canonical_relative_path) + if not legacy_aliases: + continue + + canonical_exists = (target_root / canonical_relative_path).is_file() + present_aliases = [path for path in legacy_aliases if (target_root / path).is_file()] + present_variants = ([canonical_relative_path] if canonical_exists else []) + present_aliases + selected_for_sync = canonical_relative_path in selected_paths + + if len(present_variants) >= 2: + redundant_assets.append( + RedundantAsset( + category=asset_category(canonical_relative_path), + canonical_target_path=canonical_relative_path, + existing_target_paths=present_variants, + issue_type="existing_redundancy", + selected_for_sync=selected_for_sync, + ) + ) + continue + + if selected_for_sync and present_aliases and not canonical_exists: + redundant_assets.append( + RedundantAsset( + category=asset_category(canonical_relative_path), + canonical_target_path=canonical_relative_path, + existing_target_paths=present_aliases, + issue_type="sync_would_duplicate", + selected_for_sync=True, + ) + ) + + return redundant_assets + + +def apply_redundancy_conflicts( + actions: list[FileAction], + redundant_assets: list[RedundantAsset], + agents_relative_path: str, +) -> list[FileAction]: + action_by_target_path = {action.target_relative_path: action for action in actions} + blocks_agents_inventory = False + + for redundant_asset in redundant_assets: + if not redundant_asset.selected_for_sync: + continue + + action = action_by_target_path.get(redundant_asset.canonical_target_path) + if action is None: + continue + + if action.status == "conflict": + action.reason = f"{action.reason} Also, {redundant_asset.reason}" + else: + action.status = "conflict" + action.reason = redundant_asset.reason + blocks_agents_inventory = True + + if not blocks_agents_inventory: + return actions + + agents_action = action_by_target_path.get(agents_relative_path) + if agents_action is not None and agents_action.status != "conflict": + agents_action.status = "conflict" + agents_action.reason = ( + "Redundant legacy prompt, skill, or agent aliases were detected in the target. Resolve duplicate " + "configuration families before regenerating `AGENTS.md` inventory." + ) + + return actions + + def asset_category(relative_path: str) -> str: for category in ("instructions", "prompts", "skills", "agents", "scripts"): if f"/{category}/" in relative_path: @@ -877,9 +1363,8 @@ def plan_actions(target_root: Path, planned_files: list[PlannedFile], manifest: def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, source_root: Path) -> str: instructions_apply_to = build_instruction_rules(source_root, selection.instructions) - prompts_list = describe_assets(source_root, selection.prompts, "prompt") - skills_list = describe_assets(source_root, selection.skills, "skill") - agents_list = describe_assets(source_root, selection.agents, "agent") + preferred_prompts = preferred_asset_lines(source_root, selection.preferred_prompts) + preferred_skills = preferred_asset_lines(source_root, selection.preferred_skills) governance_references = [ ".github/security-baseline.md", ".github/DEPRECATION.md", @@ -906,21 +1391,10 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, "6. Apply implementation details from referenced `skills/*/SKILL.md`.", "7. If no agent is explicitly selected, default to `TechAIImplementer`.", "", - "## Stack Resolution Rules", - "- The agent role is behavioral, not language-specific.", - "- Resolve stack from target files and explicit prompt inputs.", - "- Primary `applyTo` rules (one instruction per file type):", + "## Agent Routing", + "", + "### When to use each agent", ] - lines.extend(f" - `{rule}`" for rule in instructions_apply_to) - lines.extend( - [ - "- Overlay instructions never conflict with primary instructions - they add cross-cutting standards.", - "", - "## Agent Routing", - "", - "### When to use each agent", - ] - ) lines.extend(agent_routing_lines(selection.agents)) lines.extend( [ @@ -929,13 +1403,9 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, "- For changes spanning multiple specialist domains, run each relevant specialist and aggregate findings.", "- The standard chain for non-trivial work is: `TechAIPlanner` -> `TechAIImplementer` -> `TechAIReviewer` or a matching specialist.", "", - "## Available Skills", + "## Governance References", ] ) - lines.extend(skills_list) - lines.extend(["", "## Available Prompts"]) - lines.extend(prompts_list) - lines.extend(["", "## Governance References"]) lines.extend(f"- `{path}`" for path in governance_references) lines.extend( [ @@ -952,16 +1422,17 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, f"- Primary focus: {analysis.focus}", f"- Profile hint: `{selection.profile.name}`", "- AGENTS.md is the external bridge for assistant behavior and naming; keep runtime references abstract.", + "- Resolve stack from target files and explicit prompt inputs; the agent role remains behavioral, not language-specific.", "- Prioritize these paths:", ] ) lines.extend(f" - `{path}`" for path in analysis.priority_paths) lines.extend(["", "### Default instruction routing"]) - lines.extend(f"- `{strip_github_prefix(path)}`" for path in selection.instructions) + lines.extend(f"- `{rule}`" for rule in instructions_apply_to) lines.extend(["", "### Preferred prompts"]) - lines.extend(f"- `{strip_github_prefix(path)}`" for path in selection.preferred_prompts) + lines.extend(preferred_prompts) lines.extend(["", "### Preferred skills"]) - lines.extend(f"- `{strip_github_prefix(path)}`" for path in selection.preferred_skills) + lines.extend(preferred_skills) lines.extend(["", "### Required validations before PR"]) lines.extend(f"- `{command}`" for command in selection.validation_commands) lines.extend( @@ -979,8 +1450,6 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, lines.extend(f"- `{path}`" for path in selection.skills) lines.extend(["", "### Agents"]) lines.extend(f"- `{path}`" for path in selection.agents) - lines.extend(["", "## Agents"]) - lines.extend(agents_list) return "\n".join(lines) + "\n" @@ -997,25 +1466,28 @@ def build_instruction_rules(source_root: Path, instruction_paths: list[str]) -> apply_to = frontmatter_value(source_root / instruction_path, "applyTo") if not apply_to: continue - label = strip_github_prefix(instruction_path) + label = Path(strip_github_prefix(instruction_path)).name rules.append(f"{apply_to} -> `{label}`") return rules -def describe_assets(source_root: Path, relative_paths: list[str], asset_type: str) -> list[str]: - described: list[str] = [] +def preferred_asset_lines(source_root: Path, relative_paths: list[str]) -> list[str]: + lines: list[str] = [] for relative_path in relative_paths: - asset_path = source_root / relative_path - name = frontmatter_value(asset_path, "name") or asset_path.stem - description = frontmatter_value(asset_path, "description") or "No description available." - label = relative_path[len(".github/") :] - if asset_type == "prompt": - described.append(f"- `{name}` (`{label}`): {description}") - elif asset_type == "skill": - described.append(f"- `{name}` (`{label}`): {description}") - else: - described.append(f"- `{name}` (`{label}`): {description}") - return described + name = asset_display_name(source_root, relative_path) + description = frontmatter_value(source_root / relative_path, "description") or "No description available." + lines.append(f"- `{name}`: {description}") + return lines + + +def asset_display_name(source_root: Path, relative_path: str) -> str: + asset_path = source_root / relative_path + category = asset_category(relative_path) + if category in {"prompts", "agents"}: + return frontmatter_value(asset_path, "name") or asset_path.stem + if category == "skills": + return frontmatter_value(asset_path, "name") or asset_path.parent.name + return Path(strip_github_prefix(relative_path)).name def agent_routing_lines(agent_paths: list[str]) -> list[str]: @@ -1044,6 +1516,7 @@ def build_recommendations( analysis: TargetAnalysis, selection: AssetSelection, actions: list[FileAction], + redundant_assets: list[RedundantAsset], source_root: Path, ) -> dict[str, list[str]]: recommendations: dict[str, list[str]] = { @@ -1083,6 +1556,12 @@ def build_recommendations( "merge conflicts." ) + if any(asset.selected_for_sync for asset in redundant_assets): + recommendations["weak conflict-handling rules"].append( + "Keep the sync alias map current when canonical `tech-ai-*` assets replace legacy `cs-*` or " + "unprefixed consumer assets, so sync can stop before creating redundant configuration families." + ) + if not (analysis.repo_root / "AGENTS.md").exists() and (analysis.repo_root / ".github" / "AGENTS.md").exists(): recommendations["missing consumer-facing validation or onboarding guidance"].append( "Move legacy `.github/AGENTS.md` to root `AGENTS.md`; root is the canonical location for project " @@ -1172,8 +1651,30 @@ def render_markdown_report(plan: SyncPlan) -> str: f"- Skills: {', '.join(plan.selection.skills)}", f"- Agents: {', '.join(plan.selection.agents)}", "", - "## Planned or applied actions", ] + lines.extend(render_source_audit_markdown(plan.source_audit)) + lines.extend( + [ + "", + "## Redundant target assets", + ] + ) + if not plan.redundant_assets: + lines.append("- None") + else: + for redundant_asset in plan.redundant_assets: + lines.append( + f"- `{redundant_asset.category}` canonical `{redundant_asset.canonical_target_path}` overlaps with " + f"{', '.join(f'`{path}`' for path in redundant_asset.existing_target_paths)}" + f" ({redundant_asset.issue_type})" + ) + + lines.extend( + [ + "", + "## Planned or applied actions", + ] + ) for status in ("create", "update", "adopt", "unchanged", "conflict"): matching = [action for action in plan.actions if action.status == status] lines.append(f"### {status.title()}") @@ -1187,7 +1688,7 @@ def render_markdown_report(plan: SyncPlan) -> str: for command in plan.selection.validation_commands: lines.append(f"- `{command}`") - lines.extend(["", "## Recommendations for improving the source repo"]) + lines.extend(["", "## Target-driven recommendations for improving the source repo"]) for category, items in plan.recommendations.items(): lines.append(f"### {category.title()}") for item in items: @@ -1196,6 +1697,64 @@ def render_markdown_report(plan: SyncPlan) -> str: return "\n".join(lines) + "\n" +def render_source_audit_markdown(source_audit: SourceAudit) -> list[str]: + lines = ["## Source configuration audit", "", "### canonical_assets"] + grouped: dict[str, list[CanonicalAssetGroup]] = defaultdict(list) + for asset in source_audit.canonical_assets: + grouped[asset.category].append(asset) + + for category in AGENTS_INVENTORY_CATEGORIES: + assets = grouped.get(category, []) + if not assets: + continue + duplicate_families = [asset for asset in assets if asset.has_physical_duplicates] + if duplicate_families: + duplicate_summary = "; ".join( + f"`{asset.family}` -> {', '.join(f'`{path}`' for path in asset.paths)}" + for asset in duplicate_families + ) + lines.append( + f"- `{category}`: {len(assets)} canonical families; physical duplicates: {duplicate_summary}" + ) + else: + lines.append(f"- `{category}`: {len(assets)} canonical families; no physical duplicates.") + + lines.extend(["", "### legacy_aliases"]) + if not source_audit.legacy_aliases: + lines.append("- None") + else: + for alias in source_audit.legacy_aliases: + lines.append( + f"- `{alias.canonical_path}` has legacy aliases {', '.join(f'`{path}`' for path in alias.alias_paths)}" + ) + + lines.extend(["", "### role_overlaps"]) + if not source_audit.role_overlaps: + lines.append("- None") + else: + for overlap in source_audit.role_overlaps: + lines.append( + f"- `{overlap.family}` shares {overlap.shared_instruction_count} operational lines across " + f"{', '.join(f'`{path}`' for path in overlap.asset_paths)}" + ) + for example in overlap.examples: + lines.append(f"- Example for `{overlap.family}`: `{example}`") + + lines.extend(["", "### agents_md_repeats"]) + if not source_audit.agents_md_repeats: + lines.append("- None") + else: + for repeat in source_audit.agents_md_repeats: + lines.append( + f"- `{repeat.reference}` appears in {', '.join(f'`{section}`' for section in repeat.sections)}" + ) + + lines.extend(["", "### recommendations"]) + for item in source_audit.recommendations: + lines.append(f"- {item}") + return lines + + def render_json_report(plan: SyncPlan) -> str: payload = { "tool": SCRIPT_NAME, @@ -1210,6 +1769,53 @@ def render_json_report(plan: SyncPlan) -> str: "priority_paths": plan.analysis.priority_paths, "top_extension_counts": plan.analysis.top_extension_counts, "target_only_assets": plan.analysis.target_only_assets, + "redundant_assets": [ + { + "category": asset.category, + "canonical_target_path": asset.canonical_target_path, + "existing_target_paths": asset.existing_target_paths, + "issue_type": asset.issue_type, + "selected_for_sync": asset.selected_for_sync, + } + for asset in plan.redundant_assets + ], + }, + "source_audit": { + "canonical_assets": [ + { + "category": asset.category, + "family": asset.family, + "paths": asset.paths, + "has_physical_duplicates": asset.has_physical_duplicates, + } + for asset in plan.source_audit.canonical_assets + ], + "legacy_aliases": [ + { + "category": alias.category, + "canonical_path": alias.canonical_path, + "alias_paths": alias.alias_paths, + } + for alias in plan.source_audit.legacy_aliases + ], + "role_overlaps": [ + { + "family": overlap.family, + "asset_paths": overlap.asset_paths, + "shared_instruction_count": overlap.shared_instruction_count, + "examples": overlap.examples, + } + for overlap in plan.source_audit.role_overlaps + ], + "agents_md_repeats": [ + { + "reference": repeat.reference, + "sections": repeat.sections, + "count": repeat.count, + } + for repeat in plan.source_audit.agents_md_repeats + ], + "recommendations": plan.source_audit.recommendations, }, "selection": { "baseline_files": plan.selection.baseline_files, @@ -1242,12 +1848,17 @@ def build_plan(source_root: Path, target_root: Path) -> tuple[SyncPlan, list[Pla manifest = load_manifest(target_root) planned_files = build_planned_files(source_root, target_root, analysis, selection) actions = plan_actions(target_root, planned_files, manifest) - recommendations = build_recommendations(analysis, selection, actions, source_root) + redundant_assets = detect_redundant_assets(target_root, selection) + source_audit = audit_source_configuration(source_root) + actions = apply_redundancy_conflicts(actions, redundant_assets, analysis.agents_relative_path) + recommendations = build_recommendations(analysis, selection, actions, redundant_assets, source_root) return ( SyncPlan( analysis=analysis, selection=selection, actions=actions, + redundant_assets=redundant_assets, + source_audit=source_audit, recommendations=recommendations, manifest_relative_path=MANIFEST_RELATIVE_PATH, ), diff --git a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md index 6f35ae6..6ee56ce 100644 --- a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md +++ b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md @@ -12,20 +12,31 @@ description: Analyze a local repository, select the minimum Copilot customizatio ## Workflow 1. Inspect the target repository layout, manifests, `.github` contents, `AGENTS.md` location, and git state. -2. Classify the repository against `repo-profiles.yml`, then extend the profile with stack-specific rules only when needed. -3. Select the minimum Copilot core assets that the target repository actually needs. -4. Prefer canonical prompt families when multiple prompts cover the same workflow so consumer repositories keep the same capability with fewer tokens. -5. Render a target-specific `AGENTS.md` that uses GitHub Copilot wording only. -6. Apply conservative merge rules through the manifest file and never overwrite unmanaged divergent files. -7. Produce a final report with target actions and source-repository improvement recommendations. +2. Audit the source standards repository before syncing: + - classify canonical instructions, prompts, skills, and agents; + - detect source-side legacy aliases such as `cs-*` and unprefixed equivalents; + - detect operational overlap across prompt/skill/agent triads; + - detect repeated asset references across `AGENTS.md` sections. +3. Classify the repository against `repo-profiles.yml`, then extend the profile with stack-specific rules only when needed. +4. Select the minimum Copilot core assets that the target repository actually needs. +5. Prefer canonical prompt families when multiple prompts cover the same workflow so consumer repositories keep the same capability with fewer tokens. +6. Detect redundant legacy aliases for selected canonical assets, especially `cs-*`, unprefixed prompt names, and legacy agent or skill filenames. +7. Treat redundant canonical-vs-legacy overlaps as conflicts instead of silently creating duplicate configuration families. +8. Render a target-specific `AGENTS.md` only when the selected inventory is conflict-safe: + - keep `Preferred prompts` and `Preferred skills` as a curated shortlist; + - keep asset path references in `Repository Inventory (Auto-generated)` only; + - avoid descriptive prompt/skill catalogs that duplicate the inventory. +9. Apply conservative merge rules through the manifest file and never overwrite unmanaged divergent files. +10. Produce a final report that separates source-side redundancy from target-side conflicts and actions. ## Scope rules - Manage Copilot core assets only. - Exclude README, changelog, templates, workflows, bootstrap helpers, and source-only review/audit agents from consumer sync. - Prefer an existing root `AGENTS.md` over creating a second managed AGENTS file under `.github/`. - Keep recommendation categories fixed and comparable across runs. +- Keep legacy alias logic in code and tests instead of repeating the same rules across prompt, skill, and agent prose. ## Validation -- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict`. -- Run `python -m compileall .github/scripts/tech-ai-sync-copilot-configs.py tests`. +- Run `python -m compileall .github/scripts tests`. - Run `pytest` for the `TechAISyncCopilotConfigs` test suite. +- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict`. diff --git a/.github/templates/AGENTS.template.md b/.github/templates/AGENTS.template.md index 6c88f33..321b31c 100644 --- a/.github/templates/AGENTS.template.md +++ b/.github/templates/AGENTS.template.md @@ -1,25 +1,53 @@ # AGENTS.md - -This file is for AI assistants and GitHub Copilot working in this repository. +This file is for GitHub Copilot and AI assistants working in this repository. -## Main Instructions -- Read `.github/copilot-instructions.md` first. -- Then apply path-specific files under `.github/instructions/`. +## Naming Policy +- Use GitHub Copilot terminology in repository-facing content. +- Keep repository-facing text in English. +- Treat prompt frontmatter `name:` as the canonical command identifier. -## Configuration Files -- `.github/copilot-instructions.md` -- `.github/copilot-code-review-instructions.md` -- `.github/copilot-commit-message-instructions.md` -- Relevant `.github/instructions/*.instructions.md` +## Decision Priority +1. Apply repository non-negotiables from `.github/copilot-instructions.md`. +2. Apply explicit user requirements for the current task. +3. Apply the selected agent behavior. +4. Apply matching `.github/instructions/*.instructions.md` files. +5. Apply selected prompt constraints from `.github/prompts/*.prompt.md`. +6. Apply implementation details from referenced `.github/skills/*/SKILL.md`. -## Available Skills -- List local skill files and their purpose. +## Agent Routing +- Keep agent guidance short and behavioral. +- Document only the preferred agents for this repository and when to use them. +- Keep file path references in `Repository Inventory (Auto-generated)` only. -## Available Prompts -- List local prompt files and when to use them. +## Repository Defaults +- Primary focus: +- Priority paths: + - `` + - `` -## Conventions -- Language, naming, testing, and validation commands. +### Default instruction routing +- `` -> `` -## Prohibitions -- Explicit unsafe actions to avoid for this repository. +### Preferred prompts +- ``: + +### Preferred skills +- ``: + +### Required validations before PR +- `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` + +## Repository Inventory (Auto-generated) + +### Instructions +- `.github/instructions/.instructions.md` + +### Prompts +- `.github/prompts/.prompt.md` + +### Skills +- `.github/skills//SKILL.md` + +### Agents +- `.github/agents/.agent.md` diff --git a/AGENTS.md b/AGENTS.md index 4df6487..2cb4cf8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,119 +17,56 @@ This file is for GitHub Copilot and AI assistants working in this repository. 1. Apply repository non-negotiables from `copilot-instructions.md`. 2. Apply explicit user requirements for the current task. -3. Apply the selected agent behavior (agent-first routing). +3. Apply the selected agent behavior. 4. Apply matching files under `instructions/*.instructions.md` using `applyTo`. 5. Apply selected prompt constraints from `prompts/*.prompt.md`. 6. Apply implementation details from referenced `skills/*/SKILL.md`. 7. If no agent is explicitly selected, default to `TechAIImplementer`. -## Stack Resolution Rules - -- The agent role is behavioral, not language-specific. -- Resolve stack from target files and explicit prompt inputs. -- Primary `applyTo` rules (one instruction per file type): - - `**/*.py` -> `instructions/python.instructions.md` - - `**/*.sh` -> `instructions/bash.instructions.md` - - `**/*.tf` -> `instructions/terraform.instructions.md` - - `**/*.java` -> `instructions/java.instructions.md` - - `**/*.js,**/*.cjs,**/*.mjs,**/*.ts,**/*.tsx` -> `instructions/nodejs.instructions.md` - - `**/*lambda*.tf,**/*lambda*.py,**/*lambda*.js,**/*lambda*.ts` -> `instructions/lambda.instructions.md` - - `**/*.yml,**/*.yaml` -> `instructions/yaml.instructions.md` - - `**/*.md` -> `instructions/markdown.instructions.md` - - `**/Makefile,**/*.mk` -> `instructions/makefile.instructions.md` - - `**/workflows/**` -> `instructions/github-actions.instructions.md` - - `**/actions/**/action.y*ml` -> `instructions/github-action-composite.instructions.md` - - `**/authorizations/**/*.json,**/organization/**/*.json,**/src/**/*.json,**/data/**/*.json` -> `instructions/json.instructions.md` -- Overlay rules (additive - apply alongside the primary instruction above): - - `**/*.sh,**/scripts/**/*.py,**/scripts/**/*.sh` -> `instructions/scripts.instructions.md` -- If a change spans multiple stacks, apply all relevant instruction files. -- Overlay instructions never conflict with primary instructions - they add cross-cutting standards. - ## Agent Routing ### When to use each agent - Use `TechAIPlanner` for ambiguous scope, tradeoff analysis, or multi-step design. -- Use `TechAIImplementer` for direct code/config changes and validations. -- Use `TechAIReviewer` for quality gates and defect/regression findings. -- Use `TechAIScriptReviewer` for exhaustive, nit-level code reviews on Python, Bash, and Terraform with per-language anti-pattern catalogs. +- Use `TechAIImplementer` for direct code or configuration changes and validation-first delivery. +- Use `TechAIReviewer` for quality gates and defect or regression findings. +- Use `TechAIScriptReviewer` for exhaustive, nit-level reviews on Python, Bash, and Terraform. - Use `TechAIGlobalCustomizationBuilder` as the default specialist for creating or updating GitHub Copilot customization assets in this repository. - Use `TechAIGlobalCustomizationAuditor` as the final quality gate for GitHub Copilot customization changes in this repository. - Use `TechAICustomizationAuditor` only as a deprecated compatibility alias while older references are migrated. -- Use `TechAISyncCopilotConfigs` to analyze a local consumer repository and conservatively align the minimum Copilot customization assets from this standards repository. +- Use `TechAISyncCopilotConfigs` for cross-repository Copilot-core alignment and source or target redundancy audits. - Use specialist agents (`TechAIWorkflowSupplyChain`, `TechAISecurityReviewer`, `TechAITerraformGuardrails`, `TechAIIAMLeastPrivilege`, `TechAIPRWriter`) only when their domain matches the task. - The `TechAIGlobalCustomizationBuilder` and `TechAIGlobalCustomizationAuditor` agents are repo-only and must not be synced to consumer repositories. -### When NOT to use (anti-patterns) - -- Do not use `TechAIPlanner` for trivial single-file changes with clear requirements - go directly to `TechAIImplementer`. -- Do not use `TechAIImplementer` when requirements are ambiguous or scope is unclear - use `TechAIPlanner` first. -- Do not use `TechAIImplementer` as the primary authoring agent for GitHub Copilot customization assets in this repository - use `TechAIGlobalCustomizationBuilder`. -- Do not use generic `TechAIReviewer` when the change is purely Terraform, IAM, workflows, or security - use the matching specialist instead. -- Do not use generic `TechAIReviewer` when you need exhaustive per-language nit-level review - use `TechAIScriptReviewer` instead. -- Do not use `TechAICustomizationAuditor` for new work - use `TechAIGlobalCustomizationAuditor`. -- Do not use `TechAIImplementer` alone when the task is cross-repository Copilot configuration alignment - use `TechAISyncCopilotConfigs`. - -### Agent composition +### Anti-patterns -- For changes spanning multiple specialist domains (for example Terraform + IAM), run each relevant specialist and aggregate findings. -- The standard chain for non-trivial work is: `TechAIPlanner` -> `TechAIImplementer` -> `TechAIReviewer` (or specialist reviewer). -- For GitHub Copilot customization changes (for example `.github/prompts`, `.github/skills`, `.github/agents`, `.github/scripts`, `copilot-*.md`, `repo-profiles.yml`, and validator/workflow assets), use `TechAIGlobalCustomizationBuilder` first and `TechAIGlobalCustomizationAuditor` before final handoff. +- Do not use `TechAIPlanner` for trivial single-file changes with clear requirements; go directly to `TechAIImplementer`. +- Do not use `TechAIImplementer` when requirements are ambiguous or scope is unclear; use `TechAIPlanner` first. +- Do not use `TechAIImplementer` as the primary authoring agent for GitHub Copilot customization assets in this repository; use `TechAIGlobalCustomizationBuilder`. +- Do not use generic `TechAIReviewer` when the change is purely Terraform, IAM, workflows, or security; use the matching specialist instead. +- Do not use generic `TechAIReviewer` when you need exhaustive per-language nit-level review; use `TechAIScriptReviewer` instead. +- Do not use `TechAICustomizationAuditor` for new work; use `TechAIGlobalCustomizationAuditor`. +- Do not use `TechAIImplementer` alone when the task is cross-repository Copilot configuration alignment; use `TechAISyncCopilotConfigs`. -### Handoff protocol +### Composition and Handoffs -- `TechAIPlanner` output (implementation plan) is input context for `TechAIImplementer`. -- `TechAIImplementer` output (list of changed files + validation results) is input context for `TechAIReviewer`. +- For changes spanning multiple specialist domains, run each relevant specialist and aggregate findings. +- The standard chain for non-trivial work is `TechAIPlanner` -> `TechAIImplementer` -> `TechAIReviewer` or a matching specialist. +- For GitHub Copilot customization changes in this repository, use `TechAIGlobalCustomizationBuilder` first and `TechAIGlobalCustomizationAuditor` before final handoff. +- `TechAIPlanner` output is input context for `TechAIImplementer`. +- `TechAIImplementer` output is input context for `TechAIReviewer`. - `TechAIReviewer` findings flagged as `Critical` or `Major` route back to `TechAIImplementer` for remediation. -- `TechAIGlobalCustomizationBuilder` output (changed assets + validation results) is input context for `TechAIGlobalCustomizationAuditor`. +- `TechAIGlobalCustomizationBuilder` output is input context for `TechAIGlobalCustomizationAuditor`. - `TechAIGlobalCustomizationAuditor` findings flagged as `Critical` or `Major` route back to `TechAIGlobalCustomizationBuilder` for remediation. -## Available Skills - -- `TechAICICDWorkflow` (`skills/tech-ai-cicd-workflow/SKILL.md`): GitHub Actions workflow design and CI/CD patterns. -- `TechAICloudPolicy` (`skills/tech-ai-cloud-policy/SKILL.md`): AWS SCP, Azure Policy, and GCP Org Policy governance. -- `TechAICodeReview` (`skills/tech-ai-code-review/SKILL.md`): Exhaustive per-language anti-pattern catalogs for strict code reviews. -- `TechAICompositeAction` (`skills/tech-ai-composite-action/SKILL.md`): GitHub composite action implementation patterns. -- `TechAIDataRegistry` (`skills/tech-ai-data-registry/SKILL.md`): Data registry schema and governance automation patterns. -- `TechAIPRWriting` (`skills/tech-ai-pr-writing/SKILL.md`): PR title/body generation from template and diff. -- `TechAIProjectJava` (`skills/tech-ai-project-java/SKILL.md`): Java project code with DDD and deterministic tests. -- `TechAIProjectNodejs` (`skills/tech-ai-project-nodejs/SKILL.md`): Node.js project code with module boundaries and tests. -- `TechAIProjectPython` (`skills/tech-ai-project-python/SKILL.md`): Python project code with DDD, pytest, and type hints. -- `TechAIScriptBash` (`skills/tech-ai-script-bash/SKILL.md`): Bash utility scripts with strict mode and shellcheck. -- `TechAIScriptPython` (`skills/tech-ai-script-python/SKILL.md`): Python utility scripts with argparse and tests. -- `TechAISyncCopilotConfigs` (`skills/tech-ai-sync-copilot-configs/SKILL.md`): Conservative local repository alignment for minimal Copilot customization assets with reporting. -- `TechAITerraformFeature` (`skills/tech-ai-terraform-feature/SKILL.md`): Terraform feature implementation patterns. -- `TechAITerraformModule` (`skills/tech-ai-terraform-module/SKILL.md`): Terraform reusable module design. - -## Available Prompts - -- `TechAIAddPlatform` (`prompts/tech-ai-add-platform.prompt.md`): Add a new supported platform/profile pattern in a generic way. -- `TechAIAddReportScript` (`prompts/tech-ai-add-report-script.prompt.md`): Add a reusable reporting/analysis automation script. -- `TechAICICDWorkflow` (`prompts/tech-ai-cicd-workflow.prompt.md`): Create or modify GitHub Actions workflows. -- `TechAIAddUnitTests` (`prompts/tech-ai-add-unit-tests.prompt.md`): Add or improve unit tests. -- `TechAICloudPolicy` (`prompts/tech-ai-cloud-policy.prompt.md`): Create cloud governance policies. -- `TechAICodeReview` (`prompts/tech-ai-code-review.prompt.md`): Perform exhaustive, nit-level code reviews. -- `TechAIDataRegistry` (`prompts/tech-ai-data-registry.prompt.md`): Create or update data registry assets. -- `TechAIJava` (`prompts/tech-ai-java.prompt.md`): Generate Java project code. -- `TechAINodejs` (`prompts/tech-ai-nodejs.prompt.md`): Generate Node.js project code. -- `TechAIPython` (`prompts/tech-ai-python.prompt.md`): Generate Python project code. -- `TechAITerraform` (`prompts/tech-ai-terraform.prompt.md`): Create Terraform modules and features. -- `TechAIGitHubAction` (`prompts/tech-ai-github-action.prompt.md`): Create GitHub Actions workflows. -- `TechAICompositeAction` (`prompts/tech-ai-github-composite-action.prompt.md`): Create GitHub composite actions. -- `TechAIPRDescription` (`prompts/tech-ai-github-pr-description.prompt.md`): Generate PR descriptions. -- `TechAIBashScript` (`prompts/tech-ai-bash-script.prompt.md`): Create or modify Bash scripts with the canonical low-duplication prompt. -- `TechAIPythonScript` (`prompts/tech-ai-python-script.prompt.md`): Create or modify Python utility scripts with the canonical low-duplication prompt. -- `TechAISyncCopilotConfigs` (`prompts/tech-ai-sync-copilot-configs.prompt.md`): Analyze and conservatively align a local repository with the minimum Copilot customization assets from this standards repo. -- `TechAITerraformModule` (`prompts/tech-ai-terraform-module.prompt.md`): Create or modify Terraform modules. - ## Governance References -- `security-baseline.md`: Portable security controls baseline for all Copilot customization. -- `DEPRECATION.md`: Lifecycle and deprecation policy for all customization assets. -- `repo-profiles.yml`: Advisory profile catalog for different repository types. -- `.github/scripts/validate-copilot-customizations.sh`: Validation gate for customization changes. -- `templates/AGENTS.template.md`: Template for onboarding new repositories. -- `templates/copilot-quickstart.md`: Quick start guide for new teams. +- `security-baseline.md`: portable security controls baseline for all Copilot customization. +- `DEPRECATION.md`: lifecycle and deprecation policy for all customization assets. +- `repo-profiles.yml`: advisory profile catalog for different repository types. +- `.github/scripts/validate-copilot-customizations.sh`: validation gate for customization changes. +- `.github/templates/AGENTS.template.md`: slimmer onboarding template that keeps asset paths in the inventory section only. +- `.github/templates/copilot-quickstart.md`: quick start guide for new teams. ## Template Placeholders @@ -149,7 +86,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. ## PR and Workflow Conventions - PR content must follow `PULL_REQUEST_TEMPLATE.md` in exact section order. -- For GitHub Actions pinning, each full SHA must include an adjacent comment with release/tag reference. +- For GitHub Actions pinning, each full SHA must include an adjacent comment with release or tag reference. ## Backlog Triggers @@ -160,6 +97,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Primary focus: reusable, repository-agnostic Copilot customization standards. - Profile hint: `minimal` - AGENTS.md is the external bridge for assistant behavior and naming; keep runtime references abstract. +- Resolve stack from target files and explicit prompt inputs; the agent role remains behavioral, not language-specific. - Prioritize these paths: - `.github/instructions` - `.github/prompts` @@ -169,36 +107,44 @@ This file is for GitHub Copilot and AI assistants working in this repository. ### Default instruction routing -- `instructions/markdown.instructions.md` -- `instructions/yaml.instructions.md` -- `instructions/json.instructions.md` -- `instructions/github-actions.instructions.md` -- `instructions/github-action-composite.instructions.md` +- `**/*.py` -> `python.instructions.md` +- `**/*.sh` -> `bash.instructions.md` +- `**/*.tf` -> `terraform.instructions.md` +- `**/*.java` -> `java.instructions.md` +- `**/*.js,**/*.cjs,**/*.mjs,**/*.ts,**/*.tsx` -> `nodejs.instructions.md` +- `**/*lambda*.tf,**/*lambda*.py,**/*lambda*.js,**/*lambda*.ts` -> `lambda.instructions.md` +- `**/*.yml,**/*.yaml` -> `yaml.instructions.md` +- `**/*.md` -> `markdown.instructions.md` +- `**/Makefile,**/*.mk` -> `makefile.instructions.md` +- `**/workflows/**` -> `github-actions.instructions.md` +- `**/actions/**/action.y*ml` -> `github-action-composite.instructions.md` +- `**/authorizations/**/*.json,**/organization/**/*.json,**/src/**/*.json,**/data/**/*.json` -> `json.instructions.md` +- `**/*.sh,**/scripts/**/*.py,**/scripts/**/*.sh` -> `scripts.instructions.md` as an overlay ### Preferred prompts -- `prompts/tech-ai-code-review.prompt.md` -- `prompts/tech-ai-github-action.prompt.md` -- `prompts/tech-ai-sync-copilot-configs.prompt.md` -- `prompts/tech-ai-github-pr-description.prompt.md` -- `prompts/tech-ai-add-unit-tests.prompt.md` -- `prompts/tech-ai-terraform.prompt.md` +- `TechAICodeReview`: exhaustive, nit-level code review. +- `TechAIGitHubAction`: GitHub Actions workflow authoring. +- `TechAISyncCopilotConfigs`: cross-repository alignment and redundancy analysis. +- `TechAIPRDescription`: pull request body generation. +- `TechAIAddUnitTests`: test authoring and improvement. +- `TechAITerraform`: Terraform feature or module authoring. ### Preferred skills -- `skills/tech-ai-code-review/SKILL.md` -- `skills/tech-ai-cicd-workflow/SKILL.md` -- `skills/tech-ai-sync-copilot-configs/SKILL.md` -- `skills/tech-ai-pr-writing/SKILL.md` -- `skills/tech-ai-cloud-policy/SKILL.md` -- `skills/tech-ai-terraform-module/SKILL.md` +- `TechAICodeReview`: strict review workflow and anti-pattern catalog. +- `TechAICICDWorkflow`: CI or CD workflow design patterns. +- `TechAISyncCopilotConfigs`: deterministic sync planning and reporting. +- `TechAIPRWriting`: PR writing conventions aligned to the repository template. +- `TechAICloudPolicy`: reusable cloud policy authoring patterns. +- `TechAITerraformModule`: reusable Terraform module design. ### Required validations before PR - `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` -- `bash -n` and `shellcheck -s bash` for changed Bash scripts (when available) -- `python -m compileall ` and relevant `pytest` checks for Python changes -- `terraform fmt` and `terraform validate` for Terraform changes +- `bash -n` and `shellcheck -s bash` for changed Bash scripts when available. +- `python -m compileall ` and relevant `pytest` checks for Python changes. +- `terraform fmt` and `terraform validate` for Terraform changes. ## Repository Inventory (Auto-generated) @@ -222,22 +168,22 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/prompts/tech-ai-add-platform.prompt.md` - `.github/prompts/tech-ai-add-report-script.prompt.md` -- `.github/prompts/tech-ai-cicd-workflow.prompt.md` -- `.github/prompts/tech-ai-github-action.prompt.md` -- `.github/prompts/tech-ai-github-composite-action.prompt.md` -- `.github/prompts/tech-ai-github-pr-description.prompt.md` - `.github/prompts/tech-ai-add-unit-tests.prompt.md` - `.github/prompts/tech-ai-bash-script.prompt.md` +- `.github/prompts/tech-ai-cicd-workflow.prompt.md` - `.github/prompts/tech-ai-cloud-policy.prompt.md` - `.github/prompts/tech-ai-code-review.prompt.md` - `.github/prompts/tech-ai-data-registry.prompt.md` +- `.github/prompts/tech-ai-github-action.prompt.md` +- `.github/prompts/tech-ai-github-composite-action.prompt.md` +- `.github/prompts/tech-ai-github-pr-description.prompt.md` - `.github/prompts/tech-ai-java.prompt.md` - `.github/prompts/tech-ai-nodejs.prompt.md` - `.github/prompts/tech-ai-python-script.prompt.md` - `.github/prompts/tech-ai-python.prompt.md` - `.github/prompts/tech-ai-sync-copilot-configs.prompt.md` -- `.github/prompts/tech-ai-terraform.prompt.md` - `.github/prompts/tech-ai-terraform-module.prompt.md` +- `.github/prompts/tech-ai-terraform.prompt.md` ### Skills @@ -259,15 +205,15 @@ This file is for GitHub Copilot and AI assistants working in this repository. ### Agents - `.github/agents/tech-ai-customization-auditor.agent.md` -- `.github/agents/tech-ai-global-customization-auditor.agent.md` -- `.github/agents/tech-ai-global-customization-builder.agent.md` - `.github/agents/tech-ai-github-pr-writer.agent.md` - `.github/agents/tech-ai-github-workflow-supply-chain.agent.md` +- `.github/agents/tech-ai-global-customization-auditor.agent.md` +- `.github/agents/tech-ai-global-customization-builder.agent.md` - `.github/agents/tech-ai-iam-least-privilege.agent.md` - `.github/agents/tech-ai-implementer.agent.md` - `.github/agents/tech-ai-planner.agent.md` - `.github/agents/tech-ai-reviewer.agent.md` +- `.github/agents/tech-ai-script-reviewer.agent.md` - `.github/agents/tech-ai-security-reviewer.agent.md` - `.github/agents/tech-ai-sync-copilot-configs.agent.md` -- `.github/agents/tech-ai-script-reviewer.agent.md` - `.github/agents/tech-ai-terraform-guardrails.agent.md` diff --git a/tests/test_tech_ai_sync_copilot_configs.py b/tests/test_tech_ai_sync_copilot_configs.py index 27a07df..77420ae 100644 --- a/tests/test_tech_ai_sync_copilot_configs.py +++ b/tests/test_tech_ai_sync_copilot_configs.py @@ -53,6 +53,117 @@ def build_script_automation_target(path: Path) -> None: write_file(path / "src" / "scripts" / "report.py", 'def main() -> None:\n print("report")\n') +def build_source_audit_fixture(path: Path) -> None: + write_file( + path / "AGENTS.md", + "\n".join( + [ + "# AGENTS.md - fixture", + "", + "## Preferred prompts", + "- `prompts/tech-ai-python.prompt.md`", + "", + "## Repository Inventory (Auto-generated)", + "", + "### Prompts", + "- `.github/prompts/tech-ai-python.prompt.md`", + "", + ] + ), + ) + write_file( + path / ".github" / "prompts" / "tech-ai-python.prompt.md", + "\n".join( + [ + "---", + "name: TechAIPython", + "description: canonical", + "agent: agent", + "argument-hint: target=python", + "---", + "", + "# Python", + "", + ] + ), + ) + write_file( + path / ".github" / "prompts" / "cs-python.prompt.md", + "\n".join( + [ + "---", + "name: cs-python", + "description: legacy", + "agent: agent", + "argument-hint: target=python", + "---", + "", + "# Legacy Python", + "", + ] + ), + ) + shared_steps = [ + "1. Inspect the target repository layout and current Copilot assets.", + "2. Run the sync script in plan mode before any apply step.", + "3. Report redundant aliases before rendering AGENTS inventory.", + ] + write_file( + path / ".github" / "agents" / "tech-ai-sync-copilot-configs.agent.md", + "\n".join( + [ + "---", + "name: TechAISyncCopilotConfigs", + "description: sync agent", + 'tools: ["search"]', + "---", + "", + "# Agent", + "", + "## Workflow", + *shared_steps, + "", + ] + ), + ) + write_file( + path / ".github" / "skills" / "tech-ai-sync-copilot-configs" / "SKILL.md", + "\n".join( + [ + "---", + "name: TechAISyncCopilotConfigs", + "description: sync skill", + "---", + "", + "# Skill", + "", + "## Workflow", + *shared_steps, + "", + ] + ), + ) + write_file( + path / ".github" / "prompts" / "tech-ai-sync-copilot-configs.prompt.md", + "\n".join( + [ + "---", + "name: TechAISyncCopilotConfigs", + "description: sync prompt", + "agent: agent", + "argument-hint: target_repo=", + "---", + "", + "# Prompt", + "", + "## Instructions", + *shared_steps, + "", + ] + ), + ) + + def test_build_plan_detects_infrastructure_heavy_and_root_agents_conflict(tmp_path: Path) -> None: target_root = tmp_path / "eng-like" build_eng_like_target(target_root) @@ -128,11 +239,17 @@ def test_rendered_agents_markdown_keeps_github_copilot_wording(tmp_path: Path) - write_file(target_root / ".github" / "PULL_REQUEST_TEMPLATE.md", "# PR template\n") write_file(target_root / "infra" / "main.tf", 'resource "null_resource" "infra" {}\n') - _plan, planned_files = MODULE.build_plan(REPO_ROOT, target_root) + plan, planned_files = MODULE.build_plan(REPO_ROOT, target_root) agents_file = next(item for item in planned_files if item.target_relative_path == "AGENTS.md") + expected_prompt_path = plan.selection.prompts[0] + expected_skill_path = plan.selection.skills[0] assert "GitHub Copilot" in agents_file.desired_content assert "Codex" not in agents_file.desired_content + assert "## Available Skills" not in agents_file.desired_content + assert "## Available Prompts" not in agents_file.desired_content + assert agents_file.desired_content.count(expected_prompt_path) == 1 + assert agents_file.desired_content.count(expected_skill_path) == 1 def test_main_supports_targets_without_existing_github_directory(tmp_path: Path, capsys) -> None: @@ -184,6 +301,85 @@ def test_build_plan_prefers_tech_ai_script_prompts_to_reduce_prompt_duplication( assert ".github/prompts/script-python.prompt.md" not in plan.selection.prompts +def test_build_plan_flags_legacy_prompt_aliases_before_creating_canonical_duplicates(tmp_path: Path) -> None: + target_root = tmp_path / "legacy-prompt-aliases" + build_python_service_target(target_root) + write_file( + target_root / ".github" / "prompts" / "cs-python.prompt.md", + "---\nname: cs-python\ndescription: legacy\nagent: agent\nargument-hint: test=true\n---\n", + ) + + plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + prompt_action = next( + action for action in plan.actions if action.target_relative_path == ".github/prompts/tech-ai-python.prompt.md" + ) + agents_action = next(action for action in plan.actions if action.target_relative_path == "AGENTS.md") + + assert prompt_action.status == "conflict" + assert "redundant configuration" in prompt_action.reason + assert agents_action.status == "conflict" + assert ".github/prompts/cs-python.prompt.md" not in plan.analysis.target_only_assets["prompts"] + assert any( + asset.canonical_target_path == ".github/prompts/tech-ai-python.prompt.md" + and asset.issue_type == "sync_would_duplicate" + for asset in plan.redundant_assets + ) + + +def test_build_plan_flags_existing_canonical_and_legacy_agent_aliases_as_redundant(tmp_path: Path) -> None: + target_root = tmp_path / "duplicate-agents" + build_python_service_target(target_root) + write_file( + target_root / ".github" / "agents" / "tech-ai-planner.agent.md", + (REPO_ROOT / ".github" / "agents" / "tech-ai-planner.agent.md").read_text(encoding="utf-8"), + ) + write_file( + target_root / ".github" / "agents" / "planner.agent.md", + "---\nname: planner\ndescription: legacy\ntools: []\n---\n# Planner\n\n## Objective\nlegacy\n\n## Restrictions\nlegacy\n", + ) + + plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + action = next( + action for action in plan.actions if action.target_relative_path == ".github/agents/tech-ai-planner.agent.md" + ) + + assert action.status == "conflict" + assert ".github/agents/planner.agent.md" in action.reason + assert any( + asset.canonical_target_path == ".github/agents/tech-ai-planner.agent.md" + and asset.issue_type == "existing_redundancy" + for asset in plan.redundant_assets + ) + + +def test_audit_source_configuration_detects_legacy_aliases_role_overlaps_and_agents_repeats(tmp_path: Path) -> None: + source_root = tmp_path / "source-audit" + build_source_audit_fixture(source_root) + + audit = MODULE.audit_source_configuration(source_root) + + assert any(alias.canonical_path == ".github/prompts/tech-ai-python.prompt.md" for alias in audit.legacy_aliases) + assert any(overlap.family == "sync-copilot-configs" for overlap in audit.role_overlaps) + assert any(repeat.reference == ".github/prompts/tech-ai-python.prompt.md" for repeat in audit.agents_md_repeats) + + +def test_audit_source_configuration_does_not_flag_canonical_only_assets(tmp_path: Path) -> None: + source_root = tmp_path / "canonical-only" + write_file(source_root / "AGENTS.md", "# AGENTS.md - fixture\n") + write_file( + source_root / ".github" / "prompts" / "tech-ai-python.prompt.md", + "---\nname: TechAIPython\ndescription: canonical\nagent: agent\nargument-hint: target=python\n---\n", + ) + + audit = MODULE.audit_source_configuration(source_root) + + assert not audit.legacy_aliases + assert not audit.role_overlaps + assert not audit.agents_md_repeats + + def test_build_plan_excludes_repo_only_global_customization_agents_from_consumer_selection(tmp_path: Path) -> None: target_root = tmp_path / "python-service" build_python_service_target(target_root) @@ -239,4 +435,20 @@ def test_main_writes_json_report_with_selection_and_actions(tmp_path: Path) -> N assert payload["tool"] == "TechAISyncCopilotConfigs" assert payload["analysis"]["profile"] == "backend-python" assert ".github/prompts/tech-ai-python.prompt.md" in payload["selection"]["prompts"] + assert "redundant_assets" in payload["analysis"] + assert sorted(payload["source_audit"].keys()) == [ + "agents_md_repeats", + "canonical_assets", + "legacy_aliases", + "recommendations", + "role_overlaps", + ] assert any(action["status"] == "create" for action in payload["actions"]) + + +def test_current_source_repo_audit_has_no_sync_role_overlap_or_agents_inventory_repeats() -> None: + audit = MODULE.audit_source_configuration(REPO_ROOT) + + assert not audit.legacy_aliases + assert not any(overlap.family == "sync-copilot-configs" for overlap in audit.role_overlaps) + assert not audit.agents_md_repeats diff --git a/tests/test_tech_ai_validate_copilot_customizations.py b/tests/test_tech_ai_validate_copilot_customizations.py index 4337f42..a6f1bc1 100644 --- a/tests/test_tech_ai_validate_copilot_customizations.py +++ b/tests/test_tech_ai_validate_copilot_customizations.py @@ -108,6 +108,8 @@ def test_root_agents_routes_customization_work_to_global_agents() -> None: assert "TechAIGlobalCustomizationBuilder" in agents_text assert "TechAIGlobalCustomizationAuditor" in agents_text assert "repo-only" in agents_text + assert "## Available Skills" not in agents_text + assert "## Available Prompts" not in agents_text def test_global_builder_maps_consolidated_rules_and_legacy_auditor_is_deprecated() -> None: From 09b73e4ecdb40976e7e858a957e52162d70176bd Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Sun, 8 Mar 2026 19:13:05 +0100 Subject: [PATCH 2/9] feat: enhance Copilot synchronization and validation processes with unmanaged asset auditing --- .github/CHANGELOG.md | 4 + .../tech-ai-sync-copilot-configs.agent.md | 5 +- .../tech-ai-sync-copilot-configs.prompt.md | 7 +- .../scripts/tech-ai-sync-copilot-configs.py | 391 ++++++++++++++++-- .../tech-ai-sync-copilot-configs/SKILL.md | 16 +- .github/templates/AGENTS.template.md | 1 + tests/test_tech_ai_sync_copilot_configs.py | 134 ++++++ 7 files changed, 516 insertions(+), 42 deletions(-) diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index 691c77b..94292b8 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -7,6 +7,10 @@ Use this format for new updates: - Include file/path scope when useful. ## 2026-03-08 +- Updated `scripts/tech-ai-sync-copilot-configs.py` so target-only skill detection compares full relative paths instead of the shared `SKILL.md` filename, fixing missed unmanaged skill assets in consumer repositories. +- Expanded sync planning to audit unmanaged target-local instructions, prompts, skills, and agents for strict validation gaps and legacy alias drift, and added the new report section in both markdown and JSON outputs. +- Updated sync planning so legacy aliases such as `cs-*`, unprefixed prompt names, and legacy skill directories are reported even when the canonical family is outside the selected minimum baseline. +- Updated generated `AGENTS.md` inventory rendering and `.github/templates/AGENTS.template.md` so inventory reflects the desired managed baseline plus target-local Copilot assets already present in the consumer repository. - Added source-side redundancy auditing to `scripts/tech-ai-sync-copilot-configs.py`, including canonical asset inventory, legacy alias detection, triad role-overlap checks, and `AGENTS.md` inventory-repeat detection in both markdown and JSON reports. - Refactored `agents/tech-ai-sync-copilot-configs.agent.md`, `skills/tech-ai-sync-copilot-configs/SKILL.md`, and `prompts/tech-ai-sync-copilot-configs.prompt.md` so workflow detail lives in the skill while the agent and prompt stay thin. - Simplified root `AGENTS.md` and `.github/templates/AGENTS.template.md` to keep asset paths in the inventory section only and remove descriptive prompt or skill catalogs. diff --git a/.github/agents/tech-ai-sync-copilot-configs.agent.md b/.github/agents/tech-ai-sync-copilot-configs.agent.md index 06aa8f8..a01eae1 100644 --- a/.github/agents/tech-ai-sync-copilot-configs.agent.md +++ b/.github/agents/tech-ai-sync-copilot-configs.agent.md @@ -7,7 +7,7 @@ tools: ["search", "fetch", "editFiles", "runTerminal", "problems"] # TechAI Sync Copilot Configs Agent ## Objective -Analyze a local target repository, select the minimum Copilot customization assets from this standards repository, and align them with conservative merge rules plus a final report. +Analyze a local target repository, select the minimum Copilot customization assets from this standards repository, and align them with conservative merge rules plus a final report that also audits unmanaged target-local Copilot assets. ## Restrictions - Do not modify `README.md` files unless explicitly requested. @@ -25,6 +25,7 @@ Analyze a local target repository, select the minimum Copilot customization asse - `Target analysis`: repo shape, selected profile, stacks, git state, and AGENTS location. - `Source audit`: canonical assets, legacy aliases, role overlaps, AGENTS.md repeats, and source-side recommendations. - `Asset selection`: instructions, prompts, skills, agents, and baseline files chosen from the source repository. -- `Redundant target assets`: canonical assets that would duplicate legacy aliases or already coexist with them in the target repository. +- `Unmanaged target asset issues`: target-local instructions, prompts, skills, or agents outside the selected sync baseline, including strict validation gaps and legacy alias drift. +- `Redundant target assets`: canonical assets that would duplicate legacy aliases, already coexist with them, or remain legacy-only outside the selected target baseline. - `File actions`: create, update, adopt, unchanged, and conflict results. - `Recommendations`: categorized source-repository improvements. diff --git a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md index fba1178..7e9af9e 100644 --- a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md +++ b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md @@ -23,14 +23,17 @@ Use this prompt to analyze a local repository, select the minimum Copilot custom 3. Start with `mode=plan`; use `mode=apply` only when explicitly requested and only after a conflict-safe plan. 4. Keep scope limited to Copilot core assets only. 5. Preserve unmanaged target files and report conflicts instead of overwriting them. -6. Report source-side audit findings separately from target-side redundant assets and file actions. +6. Audit target-local instructions, prompts, skills, and agents that fall outside the selected sync baseline and report strict validation gaps or legacy alias drift. +7. Render `AGENTS.md` inventory from the desired managed baseline plus the Copilot assets already present in the target repository. +8. Report source-side audit findings separately from target-side unmanaged asset issues, redundant or legacy target assets, and file actions. ## Minimal example - Input: `target_repo=/workspace/consumer-repo mode=plan report_format=md` - Expected output: - Target analysis summary with detected profile and stacks. - Source configuration audit for canonical assets, legacy aliases, role overlaps, and AGENTS.md repeats. - - Redundant target assets that would duplicate canonical sync output. + - Unmanaged target asset issues for target-local files outside the selected baseline. + - Redundant or legacy target assets that would duplicate canonical sync output or remain alias-only. - Conservative file action plan for Copilot core assets only. - Recommendations for improving the source standards repository. diff --git a/.github/scripts/tech-ai-sync-copilot-configs.py b/.github/scripts/tech-ai-sync-copilot-configs.py index f6b2092..0c7fdb0 100644 --- a/.github/scripts/tech-ai-sync-copilot-configs.py +++ b/.github/scripts/tech-ai-sync-copilot-configs.py @@ -87,6 +87,14 @@ ROLE_OVERLAP_SECTION_PREFIXES = ("workflow", "instructions", "validation") ROLE_OVERLAP_LINE_THRESHOLD = 3 AGENTS_INVENTORY_CATEGORIES = ("instructions", "prompts", "skills", "agents") +PROMPT_NAME_OVERRIDES = { + "tech-ai-add-platform.prompt.md": "TechAIAddPlatform", + "tech-ai-add-report-script.prompt.md": "TechAIAddReportScript", + "tech-ai-cicd-workflow.prompt.md": "TechAICICDWorkflow", + "tech-ai-github-composite-action.prompt.md": "TechAICompositeAction", + "tech-ai-github-pr-description.prompt.md": "TechAIPRDescription", + "tech-ai-terraform-module.prompt.md": "TechAITerraformModule", +} def log_info(message: str) -> None: @@ -189,6 +197,7 @@ class SyncPlan: selection: AssetSelection actions: list[FileAction] redundant_assets: list["RedundantAsset"] + target_asset_issues: list["TargetAssetIssue"] source_audit: "SourceAudit" recommendations: dict[str, list[str]] manifest_relative_path: str @@ -217,6 +226,11 @@ def reason(self) -> str: "Equivalent legacy asset(s) already exist in target: " f"{listed}. Syncing `{self.canonical_target_path}` would create redundant configuration." ) + if self.issue_type == "legacy_alias_only": + return ( + "Legacy alias asset(s) exist in target without the canonical file: " + f"{listed}. Prefer `{self.canonical_target_path}` when aligning this capability family." + ) return ( "Equivalent assets from the same capability family already coexist in target: " @@ -224,6 +238,16 @@ def reason(self) -> str: ) +@dataclass +class TargetAssetIssue: + category: str + target_relative_path: str + issue_types: list[str] + details: list[str] + severity: str + canonical_source_path: str | None = None + + @dataclass class CanonicalAssetGroup: category: str @@ -383,20 +407,31 @@ def load_profiles(path: Path) -> dict[str, RepoProfile]: return profiles -def frontmatter_value(path: Path, key: str) -> str: +def parse_frontmatter(path: Path) -> dict[str, str]: + frontmatter: dict[str, str] = {} inside_frontmatter = False for raw_line in path.read_text(encoding="utf-8").splitlines(): - if raw_line.strip() == "---": - inside_frontmatter = not inside_frontmatter + stripped = raw_line.strip() + if stripped == "---": + if inside_frontmatter: + break + inside_frontmatter = True continue - if not inside_frontmatter: + if not inside_frontmatter or ":" not in raw_line: continue - if raw_line.startswith(f"{key}:"): - return raw_line.split(":", 1)[1].strip().strip('"') + key, value = raw_line.split(":", 1) + key = key.strip() + if not key or " " in key: + continue + frontmatter[key] = value.strip().strip('"') - return "" + return frontmatter + + +def frontmatter_value(path: Path, key: str) -> str: + return parse_frontmatter(path).get(key, "") def prompt_skill_refs(path: Path) -> list[str]: @@ -405,16 +440,53 @@ def prompt_skill_refs(path: Path) -> list[str]: if "skills/" not in raw_line or "SKILL.md" not in raw_line: continue - for token in raw_line.replace("`", " ").replace("(", " ").replace(")", " ").split(): - cleaned = token.strip(".,") - if cleaned.endswith("SKILL.md") and "skills/" in cleaned: - if cleaned.startswith(PROMPT_SKILL_REFERENCE_PREFIX): - refs.add(cleaned) - else: - refs.add(f"{PROMPT_SKILL_REFERENCE_PREFIX}{cleaned}") + for token in re.findall(r"(?:\.github/)?skills/[A-Za-z0-9._/-]+/SKILL\.md", raw_line): + if token.startswith(PROMPT_SKILL_REFERENCE_PREFIX): + refs.add(token) + else: + refs.add(f"{PROMPT_SKILL_REFERENCE_PREFIX}{token}") return sorted(refs) +def markdown_headings(path: Path) -> list[str]: + headings: list[str] = [] + for raw_line in path.read_text(encoding="utf-8").splitlines(): + stripped = raw_line.strip() + if stripped.startswith("#"): + headings.append(stripped) + return headings + + +def has_heading_exact(path: Path, heading: str) -> bool: + return heading in markdown_headings(path) + + +def has_heading_regex(path: Path, pattern: str) -> bool: + regex = re.compile(pattern) + return any(regex.match(heading) for heading in markdown_headings(path)) + + +def tech_ai_prompt_name(stem: str) -> str: + remainder = stem[len("tech-ai-") :] + output = "TechAI" + for part in remainder.split("-"): + if not part: + continue + output += part[:1].upper() + part[1:] + return output + + +def prompt_expected_name(relative_path: str) -> str | None: + name = Path(relative_path).name + if name in PROMPT_NAME_OVERRIDES: + return PROMPT_NAME_OVERRIDES[name] + if name.startswith("tech-ai-") and name.endswith(".prompt.md"): + return tech_ai_prompt_name(name[: -len(".prompt.md")]) + if name.endswith(".prompt.md"): + return name[: -len(".prompt.md")] + return None + + def scan_repo_files(repo_root: Path) -> list[Path]: files: list[Path] = [] for path in repo_root.rglob("*"): @@ -612,6 +684,35 @@ def known_legacy_alias_paths(source_root: Path) -> set[str]: return aliases +def known_legacy_alias_map(source_root: Path) -> dict[str, str]: + alias_map: dict[str, str] = {} + for group in detect_canonical_asset_groups(source_root): + if group.category not in {"prompts", "skills", "agents"}: + continue + canonical_path = group.paths[0] + for alias_path in legacy_alias_paths_for_canonical_path(canonical_path): + alias_map.setdefault(alias_path, canonical_path) + return alias_map + + +def collect_target_config_assets(target_root: Path) -> dict[str, list[str]]: + assets: dict[str, list[str]] = {category: [] for category in AGENTS_INVENTORY_CATEGORIES} + patterns = { + "instructions": "*.instructions.md", + "prompts": "*.prompt.md", + "skills": "SKILL.md", + "agents": "*.agent.md", + } + + for category, pattern in patterns.items(): + target_dir = target_root / ".github" / category + if not target_dir.is_dir(): + continue + assets[category] = sorted(str(path.relative_to(target_root)) for path in target_dir.rglob(pattern)) + + return assets + + def detect_target_only_assets(source_root: Path, target_root: Path) -> dict[str, list[str]]: result: dict[str, list[str]] = {"prompts": [], "skills": [], "agents": []} known_aliases = known_legacy_alias_paths(source_root) @@ -621,12 +722,14 @@ def detect_target_only_assets(source_root: Path, target_root: Path) -> dict[str, if not target_dir.is_dir(): continue - source_names = {path.name for path in source_dir.rglob(pattern)} if source_dir.is_dir() else set() + source_relative_paths = ( + {str(path.relative_to(source_root)) for path in source_dir.rglob(pattern)} if source_dir.is_dir() else set() + ) for path in sorted(target_dir.rglob(pattern)): relative_path = str(path.relative_to(target_root)) if relative_path in known_aliases: continue - if path.name not in source_names: + if relative_path not in source_relative_paths: result[category].append(relative_path) return result @@ -1114,6 +1217,151 @@ def build_validation_commands(analysis: TargetAnalysis, instruction_paths: set[s return commands +def merged_inventory_paths(target_root: Path, selection: AssetSelection) -> dict[str, list[str]]: + target_assets = collect_target_config_assets(target_root) + return { + "instructions": sorted(set(selection.instructions) | set(target_assets["instructions"])), + "prompts": sorted(set(selection.prompts) | set(target_assets["prompts"])), + "skills": sorted(set(selection.skills) | set(target_assets["skills"])), + "agents": sorted(set(selection.agents) | set(target_assets["agents"])), + } + + +def validate_unmanaged_prompt_asset(target_root: Path, relative_path: str) -> list[str]: + path = target_root / relative_path + frontmatter = parse_frontmatter(path) + issues: list[str] = [] + + for key in ("description", "name", "agent", "argument-hint"): + if not frontmatter.get(key): + issues.append(f"Missing frontmatter key `{key}`.") + + if "mode" in frontmatter: + issues.append("Legacy prompt key `mode` found.") + + expected_name = prompt_expected_name(relative_path) + actual_name = frontmatter.get("name", "") + if expected_name and actual_name and actual_name != expected_name: + issues.append(f"Prompt name policy mismatch: expected `{expected_name}`, found `{actual_name}`.") + + for heading in ("## Instructions", "## Validation", "## Minimal example"): + if not has_heading_exact(path, heading): + issues.append(f"Missing `{heading}` section.") + + refs = prompt_skill_refs(path) + if not refs: + issues.append("Missing skill reference.") + return issues + + for ref in refs: + if not (target_root / ref).is_file(): + issues.append(f"Referenced skill path is missing: `{ref}`.") + + return issues + + +def validate_unmanaged_skill_asset(target_root: Path, relative_path: str) -> list[str]: + path = target_root / relative_path + frontmatter = parse_frontmatter(path) + issues: list[str] = [] + + for key in ("name", "description"): + if not frontmatter.get(key): + issues.append(f"Missing frontmatter key `{key}`.") + + if not has_heading_regex(path, r"^## When to [Uu]se$"): + issues.append("Missing `## When to use` section.") + if not has_heading_regex(path, r"^## (Validation|Checklist|Testing|Test stack)$"): + issues.append("Missing validation/testing section.") + + return issues + + +def validate_unmanaged_agent_asset(target_root: Path, relative_path: str) -> list[str]: + path = target_root / relative_path + frontmatter = parse_frontmatter(path) + issues: list[str] = [] + + for key in ("name", "description", "tools"): + if not frontmatter.get(key): + issues.append(f"Missing frontmatter key `{key}`.") + + if not any(heading.startswith("# ") for heading in markdown_headings(path)): + issues.append("Missing top heading.") + if not has_heading_exact(path, "## Objective"): + issues.append("Missing `## Objective` section.") + if not has_heading_exact(path, "## Restrictions"): + issues.append("Missing `## Restrictions` section.") + + return issues + + +def validate_unmanaged_instruction_asset(target_root: Path, relative_path: str) -> list[str]: + path = target_root / relative_path + frontmatter = parse_frontmatter(path) + issues: list[str] = [] + + for key in ("applyTo", "description"): + if not frontmatter.get(key): + issues.append(f"Missing frontmatter key `{key}`.") + + if not any(heading.startswith("# ") for heading in markdown_headings(path)): + issues.append("Missing top heading.") + + return issues + + +def detect_unmanaged_target_asset_issues( + source_root: Path, + target_root: Path, + selection: AssetSelection, +) -> list[TargetAssetIssue]: + alias_map = known_legacy_alias_map(source_root) + managed_paths = set(selection.managed_source_paths) + validators = { + "instructions": validate_unmanaged_instruction_asset, + "prompts": validate_unmanaged_prompt_asset, + "skills": validate_unmanaged_skill_asset, + "agents": validate_unmanaged_agent_asset, + } + + issues: list[TargetAssetIssue] = [] + for category, relative_paths in collect_target_config_assets(target_root).items(): + validator = validators[category] + for relative_path in relative_paths: + if relative_path in managed_paths: + continue + + issue_types: list[str] = [] + details: list[str] = [] + canonical_source_path = alias_map.get(relative_path) + + if canonical_source_path: + issue_types.append("legacy_alias") + details.append(f"Legacy alias of `{canonical_source_path}`.") + + validation_issues = validator(target_root, relative_path) + if validation_issues: + issue_types.append("validation") + details.extend(validation_issues) + + if not issue_types: + continue + + issues.append( + TargetAssetIssue( + category=category, + target_relative_path=relative_path, + issue_types=issue_types, + details=sorted(dict.fromkeys(details)), + severity="error" if "validation" in issue_types else "warn", + canonical_source_path=canonical_source_path, + ) + ) + + return sorted(issues, key=lambda item: (item.category, item.target_relative_path)) + + def build_planned_files( source_root: Path, target_root: Path, @@ -1148,19 +1396,19 @@ def build_planned_files( return sorted(planned_files, key=lambda item: item.target_relative_path) -def detect_redundant_assets(target_root: Path, selection: AssetSelection) -> list[RedundantAsset]: +def detect_redundant_assets(source_root: Path, target_root: Path, selection: AssetSelection) -> list[RedundantAsset]: selected_paths = set(selection.managed_source_paths) redundant_assets: list[RedundantAsset] = [] - canonical_candidates = { - path - for path in selected_paths - if asset_category(path) in {"prompts", "skills", "agents"} - } + canonical_candidates = set(known_legacy_alias_map(source_root).values()) + canonical_candidates.update( + path for path in selected_paths if asset_category(path) in {"prompts", "skills", "agents"} + ) canonical_candidates.update( str(path.relative_to(target_root)) for path in scan_repo_files(target_root) - if asset_category(str(path.relative_to(target_root))) in {"prompts", "skills", "agents"} + if is_canonical_source_asset(str(path.relative_to(target_root))) + and asset_category(str(path.relative_to(target_root))) in {"prompts", "skills", "agents"} ) for canonical_relative_path in sorted(canonical_candidates): @@ -1195,6 +1443,18 @@ def detect_redundant_assets(target_root: Path, selection: AssetSelection) -> lis selected_for_sync=True, ) ) + continue + + if present_aliases and not canonical_exists: + redundant_assets.append( + RedundantAsset( + category=asset_category(canonical_relative_path), + canonical_target_path=canonical_relative_path, + existing_target_paths=present_aliases, + issue_type="legacy_alias_only", + selected_for_sync=False, + ) + ) return redundant_assets @@ -1365,6 +1625,7 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, instructions_apply_to = build_instruction_rules(source_root, selection.instructions) preferred_prompts = preferred_asset_lines(source_root, selection.preferred_prompts) preferred_skills = preferred_asset_lines(source_root, selection.preferred_skills) + inventory_paths = merged_inventory_paths(analysis.repo_root, selection) governance_references = [ ".github/security-baseline.md", ".github/DEPRECATION.md", @@ -1439,17 +1700,18 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, [ "", "## Repository Inventory (Auto-generated)", + "This inventory reflects the desired managed baseline plus repository-local Copilot assets already present in the target repository.", "", "### Instructions", ] ) - lines.extend(f"- `{path}`" for path in selection.instructions) + lines.extend(f"- `{path}`" for path in inventory_paths["instructions"]) lines.extend(["", "### Prompts"]) - lines.extend(f"- `{path}`" for path in selection.prompts) + lines.extend(f"- `{path}`" for path in inventory_paths["prompts"]) lines.extend(["", "### Skills"]) - lines.extend(f"- `{path}`" for path in selection.skills) + lines.extend(f"- `{path}`" for path in inventory_paths["skills"]) lines.extend(["", "### Agents"]) - lines.extend(f"- `{path}`" for path in selection.agents) + lines.extend(f"- `{path}`" for path in inventory_paths["agents"]) return "\n".join(lines) + "\n" @@ -1517,6 +1779,7 @@ def build_recommendations( selection: AssetSelection, actions: list[FileAction], redundant_assets: list[RedundantAsset], + target_asset_issues: list[TargetAssetIssue], source_root: Path, ) -> dict[str, list[str]]: recommendations: dict[str, list[str]] = { @@ -1544,10 +1807,36 @@ def build_recommendations( ) target_only_prompt_assets = analysis.target_only_assets.get("prompts", []) - if target_only_prompt_assets: + target_only_skill_assets = analysis.target_only_assets.get("skills", []) + if target_only_prompt_assets or target_only_skill_assets: + target_only_summary: list[str] = [] + if target_only_prompt_assets: + target_only_summary.append(f"prompts: {', '.join(target_only_prompt_assets)}") + if target_only_skill_assets: + target_only_summary.append(f"skills: {', '.join(target_only_skill_assets)}") recommendations["missing instructions/prompts/skills"].append( - "The target repository already contains prompt assets that are not available in the source standards " - f"repo: {', '.join(target_only_prompt_assets)}." + "The target repository already contains Copilot assets that are not available in the source standards " + f"repo: {'; '.join(target_only_summary)}." + ) + + legacy_alias_issues = [ + f"{issue.target_relative_path} -> {issue.canonical_source_path}" + for issue in target_asset_issues + if "legacy_alias" in issue.issue_types and issue.canonical_source_path + ] + if legacy_alias_issues: + recommendations["weak conflict-handling rules"].append( + "The target repository still contains legacy prompt/skill/agent aliases outside the selected baseline: " + f"{', '.join(legacy_alias_issues)}." + ) + + validation_issue_paths = [ + issue.target_relative_path for issue in target_asset_issues if "validation" in issue.issue_types + ] + if validation_issue_paths: + recommendations["missing consumer-facing validation or onboarding guidance"].append( + "The target repository contains unmanaged Copilot assets with strict-validation gaps: " + f"{', '.join(validation_issue_paths)}." ) if any(action.target_relative_path == analysis.agents_relative_path and action.status == "conflict" for action in actions): @@ -1656,7 +1945,23 @@ def render_markdown_report(plan: SyncPlan) -> str: lines.extend( [ "", - "## Redundant target assets", + "## Unmanaged target asset issues", + ] + ) + if not plan.target_asset_issues: + lines.append("- None") + else: + for issue in plan.target_asset_issues: + issue_types = ", ".join(issue.issue_types) + details = "; ".join(issue.details) + lines.append( + f"- `{issue.target_relative_path}` [{issue.severity}] ({issue.category}; {issue_types}): {details}" + ) + + lines.extend( + [ + "", + "## Redundant or legacy target assets", ] ) if not plan.redundant_assets: @@ -1769,6 +2074,17 @@ def render_json_report(plan: SyncPlan) -> str: "priority_paths": plan.analysis.priority_paths, "top_extension_counts": plan.analysis.top_extension_counts, "target_only_assets": plan.analysis.target_only_assets, + "unmanaged_target_asset_issues": [ + { + "category": issue.category, + "target_relative_path": issue.target_relative_path, + "issue_types": issue.issue_types, + "details": issue.details, + "severity": issue.severity, + "canonical_source_path": issue.canonical_source_path, + } + for issue in plan.target_asset_issues + ], "redundant_assets": [ { "category": asset.category, @@ -1848,16 +2164,25 @@ def build_plan(source_root: Path, target_root: Path) -> tuple[SyncPlan, list[Pla manifest = load_manifest(target_root) planned_files = build_planned_files(source_root, target_root, analysis, selection) actions = plan_actions(target_root, planned_files, manifest) - redundant_assets = detect_redundant_assets(target_root, selection) + redundant_assets = detect_redundant_assets(source_root, target_root, selection) + target_asset_issues = detect_unmanaged_target_asset_issues(source_root, target_root, selection) source_audit = audit_source_configuration(source_root) actions = apply_redundancy_conflicts(actions, redundant_assets, analysis.agents_relative_path) - recommendations = build_recommendations(analysis, selection, actions, redundant_assets, source_root) + recommendations = build_recommendations( + analysis, + selection, + actions, + redundant_assets, + target_asset_issues, + source_root, + ) return ( SyncPlan( analysis=analysis, selection=selection, actions=actions, redundant_assets=redundant_assets, + target_asset_issues=target_asset_issues, source_audit=source_audit, recommendations=recommendations, manifest_relative_path=MANIFEST_RELATIVE_PATH, diff --git a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md index 6ee56ce..0d1f03c 100644 --- a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md +++ b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md @@ -20,14 +20,19 @@ description: Analyze a local repository, select the minimum Copilot customizatio 3. Classify the repository against `repo-profiles.yml`, then extend the profile with stack-specific rules only when needed. 4. Select the minimum Copilot core assets that the target repository actually needs. 5. Prefer canonical prompt families when multiple prompts cover the same workflow so consumer repositories keep the same capability with fewer tokens. -6. Detect redundant legacy aliases for selected canonical assets, especially `cs-*`, unprefixed prompt names, and legacy agent or skill filenames. -7. Treat redundant canonical-vs-legacy overlaps as conflicts instead of silently creating duplicate configuration families. -8. Render a target-specific `AGENTS.md` only when the selected inventory is conflict-safe: +6. Detect redundant legacy aliases for canonical prompt/skill/agent families, including families that are not part of the selected minimum baseline, so `cs-*` and unprefixed leftovers still appear in the plan. +7. Audit target-local instructions, prompts, skills, and agents that are outside the selected sync baseline: + - report strict validation gaps for unmanaged files; + - report legacy aliases even when the canonical family is not selected; + - keep target-only custom assets visible instead of silently omitting them. +8. Treat redundant canonical-vs-legacy overlaps as conflicts instead of silently creating duplicate configuration families. +9. Render a target-specific `AGENTS.md` only when the selected inventory is conflict-safe: - keep `Preferred prompts` and `Preferred skills` as a curated shortlist; - keep asset path references in `Repository Inventory (Auto-generated)` only; + - build the inventory from the desired managed baseline plus existing target-local instructions/prompts/skills/agents so the rendered inventory reflects the real target state; - avoid descriptive prompt/skill catalogs that duplicate the inventory. -9. Apply conservative merge rules through the manifest file and never overwrite unmanaged divergent files. -10. Produce a final report that separates source-side redundancy from target-side conflicts and actions. +10. Apply conservative merge rules through the manifest file and never overwrite unmanaged divergent files. +11. Produce a final report that separates source-side redundancy from target-side unmanaged asset issues, legacy alias drift, conflicts, and file actions. ## Scope rules - Manage Copilot core assets only. @@ -35,6 +40,7 @@ description: Analyze a local repository, select the minimum Copilot customizatio - Prefer an existing root `AGENTS.md` over creating a second managed AGENTS file under `.github/`. - Keep recommendation categories fixed and comparable across runs. - Keep legacy alias logic in code and tests instead of repeating the same rules across prompt, skill, and agent prose. +- Do not omit existing target-local Copilot assets from the rendered AGENTS inventory just because they are outside the selected sync baseline. ## Validation - Run `python -m compileall .github/scripts tests`. diff --git a/.github/templates/AGENTS.template.md b/.github/templates/AGENTS.template.md index 321b31c..963ab06 100644 --- a/.github/templates/AGENTS.template.md +++ b/.github/templates/AGENTS.template.md @@ -39,6 +39,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` ## Repository Inventory (Auto-generated) +This inventory should reflect the managed sync baseline plus any repository-local Copilot assets that already exist in the target repository. ### Instructions - `.github/instructions/.instructions.md` diff --git a/tests/test_tech_ai_sync_copilot_configs.py b/tests/test_tech_ai_sync_copilot_configs.py index 77420ae..f952c4a 100644 --- a/tests/test_tech_ai_sync_copilot_configs.py +++ b/tests/test_tech_ai_sync_copilot_configs.py @@ -214,6 +214,130 @@ def test_build_plan_adopts_matching_source_files_and_reports_target_only_prompts assert "create-policy.prompt.md" in missing_assets +def test_build_plan_reports_unmanaged_target_assets_and_legacy_aliases_outside_selected_profile(tmp_path: Path) -> None: + target_root = tmp_path / "unmanaged-assets" + build_python_service_target(target_root) + write_file( + target_root / ".github" / "prompts" / "add-external-user.prompt.md", + "\n".join( + [ + "---", + "agent: edit", + "description: Add an external user to Azure AD", + "---", + "", + "# Add External User", + "", + "## Instructions", + "- Update the external user registry.", + "", + "## Validations", + "- Keep JSON valid.", + "", + ] + ), + ) + write_file( + target_root / ".github" / "prompts" / "cs-data-registry.prompt.md", + "\n".join( + [ + "---", + "description: Add or modify entries in structured JSON/YAML registry files", + "name: cs-data-registry", + "agent: agent", + "argument-hint: action= file= key= change=", + "---", + "", + "# Data Registry Task", + "", + "## Instructions", + "1. Use `.github/skills/data-registry/SKILL.md`.", + "", + "## Minimal example", + "- Input: `action=modify file=data.json key=user change=disable`", + "", + "## Validation", + "- Validate JSON syntax.", + "", + ] + ), + ) + write_file( + target_root / ".github" / "skills" / "data-registry" / "SKILL.md", + "\n".join( + [ + "---", + "name: data-registry", + "description: Safely update structured JSON/YAML registry files.", + "---", + "", + "# Data Registry Skill", + "", + "## When to use", + "- Update structured data safely.", + "", + "## Validation", + "- Validate syntax and duplicate keys.", + "", + ] + ), + ) + write_file( + target_root / ".github" / "skills" / "local-registry" / "SKILL.md", + "\n".join( + [ + "---", + "name: local-registry", + "description: Custom local registry helper.", + "---", + "", + "# Local Registry Skill", + "", + "## When to use", + "- Maintain a custom local registry.", + "", + "## Validation", + "- Validate repository-local data.", + "", + ] + ), + ) + + plan, planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + issue_by_path = {issue.target_relative_path: issue for issue in plan.target_asset_issues} + agents_file = next(item for item in planned_files if item.target_relative_path == "AGENTS.md") + + assert ".github/prompts/add-external-user.prompt.md" in plan.analysis.target_only_assets["prompts"] + assert ".github/skills/local-registry/SKILL.md" in plan.analysis.target_only_assets["skills"] + assert ".github/prompts/cs-data-registry.prompt.md" not in plan.analysis.target_only_assets["prompts"] + assert ".github/skills/data-registry/SKILL.md" not in plan.analysis.target_only_assets["skills"] + + assert any( + asset.canonical_target_path == ".github/prompts/tech-ai-data-registry.prompt.md" + and asset.issue_type == "legacy_alias_only" + for asset in plan.redundant_assets + ) + assert any( + asset.canonical_target_path == ".github/skills/tech-ai-data-registry/SKILL.md" + and asset.issue_type == "legacy_alias_only" + for asset in plan.redundant_assets + ) + + assert "validation" in issue_by_path[".github/prompts/add-external-user.prompt.md"].issue_types + assert "Missing frontmatter key `name`." in issue_by_path[".github/prompts/add-external-user.prompt.md"].details + assert "legacy_alias" in issue_by_path[".github/prompts/cs-data-registry.prompt.md"].issue_types + assert ( + issue_by_path[".github/prompts/cs-data-registry.prompt.md"].canonical_source_path + == ".github/prompts/tech-ai-data-registry.prompt.md" + ) + + assert ".github/prompts/add-external-user.prompt.md" in agents_file.desired_content + assert ".github/prompts/cs-data-registry.prompt.md" in agents_file.desired_content + assert ".github/skills/data-registry/SKILL.md" in agents_file.desired_content + assert ".github/skills/local-registry/SKILL.md" in agents_file.desired_content + + def test_apply_plan_writes_manifest_and_managed_files(tmp_path: Path) -> None: target_root = tmp_path / "fresh-target" write_file(target_root / ".github" / "PULL_REQUEST_TEMPLATE.md", "# PR template\n") @@ -415,6 +539,10 @@ def test_build_plan_fails_for_corrupted_manifest(tmp_path: Path) -> None: def test_main_writes_json_report_with_selection_and_actions(tmp_path: Path) -> None: target_root = tmp_path / "json-report" build_python_service_target(target_root) + write_file( + target_root / ".github" / "prompts" / "add-external-user.prompt.md", + "---\nagent: edit\ndescription: invalid custom prompt\n---\n# Add External User\n\n## Instructions\n- Update registry.\n", + ) report_file = tmp_path / "tech-ai-sync-report.json" result = MODULE.main( @@ -436,6 +564,12 @@ def test_main_writes_json_report_with_selection_and_actions(tmp_path: Path) -> N assert payload["analysis"]["profile"] == "backend-python" assert ".github/prompts/tech-ai-python.prompt.md" in payload["selection"]["prompts"] assert "redundant_assets" in payload["analysis"] + assert "unmanaged_target_asset_issues" in payload["analysis"] + assert any( + issue["target_relative_path"] == ".github/prompts/add-external-user.prompt.md" + and "validation" in issue["issue_types"] + for issue in payload["analysis"]["unmanaged_target_asset_issues"] + ) assert sorted(payload["source_audit"].keys()) == [ "agents_md_repeats", "canonical_assets", From 20e842ea0217f35e5875e2ab00716127fe85bd8c Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Mon, 9 Mar 2026 11:15:20 +0100 Subject: [PATCH 3/9] feat: enforce local naming conventions for repository-owned assets in Copilot configurations --- .github/CHANGELOG.md | 1 + .../tech-ai-sync-copilot-configs.agent.md | 2 +- .../tech-ai-sync-copilot-configs.prompt.md | 2 +- .../scripts/tech-ai-sync-copilot-configs.py | 71 +++++++++- .../validate-copilot-customizations.sh | 128 ++++++++++++++++++ .../tech-ai-sync-copilot-configs/SKILL.md | 1 + .github/templates/AGENTS.template.md | 2 + AGENTS.md | 2 + tests/test_tech_ai_sync_copilot_configs.py | 62 +++++++++ ...tech_ai_validate_copilot_customizations.py | 67 +++++++++ 10 files changed, 331 insertions(+), 7 deletions(-) diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index 94292b8..1459706 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -7,6 +7,7 @@ Use this format for new updates: - Include file/path scope when useful. ## 2026-03-08 +- Updated `scripts/tech-ai-sync-copilot-configs.py` and `scripts/validate-copilot-customizations.sh` so repository-owned prompt, skill, and agent assets outside the synced global baseline must use `local-*` in both filenames and `name:` values, making local customizations visibly distinct from synced `tech-ai-*` assets. - Updated `scripts/tech-ai-sync-copilot-configs.py` so target-only skill detection compares full relative paths instead of the shared `SKILL.md` filename, fixing missed unmanaged skill assets in consumer repositories. - Expanded sync planning to audit unmanaged target-local instructions, prompts, skills, and agents for strict validation gaps and legacy alias drift, and added the new report section in both markdown and JSON outputs. - Updated sync planning so legacy aliases such as `cs-*`, unprefixed prompt names, and legacy skill directories are reported even when the canonical family is outside the selected minimum baseline. diff --git a/.github/agents/tech-ai-sync-copilot-configs.agent.md b/.github/agents/tech-ai-sync-copilot-configs.agent.md index a01eae1..9d4e089 100644 --- a/.github/agents/tech-ai-sync-copilot-configs.agent.md +++ b/.github/agents/tech-ai-sync-copilot-configs.agent.md @@ -25,7 +25,7 @@ Analyze a local target repository, select the minimum Copilot customization asse - `Target analysis`: repo shape, selected profile, stacks, git state, and AGENTS location. - `Source audit`: canonical assets, legacy aliases, role overlaps, AGENTS.md repeats, and source-side recommendations. - `Asset selection`: instructions, prompts, skills, agents, and baseline files chosen from the source repository. -- `Unmanaged target asset issues`: target-local instructions, prompts, skills, or agents outside the selected sync baseline, including strict validation gaps and legacy alias drift. +- `Unmanaged target asset issues`: target-local instructions, prompts, skills, or agents outside the selected sync baseline, including strict validation gaps, `local-*` naming violations for repository-owned prompt/skill/agent assets, and legacy alias drift. - `Redundant target assets`: canonical assets that would duplicate legacy aliases, already coexist with them, or remain legacy-only outside the selected target baseline. - `File actions`: create, update, adopt, unchanged, and conflict results. - `Recommendations`: categorized source-repository improvements. diff --git a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md index 7e9af9e..87c716f 100644 --- a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md +++ b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md @@ -23,7 +23,7 @@ Use this prompt to analyze a local repository, select the minimum Copilot custom 3. Start with `mode=plan`; use `mode=apply` only when explicitly requested and only after a conflict-safe plan. 4. Keep scope limited to Copilot core assets only. 5. Preserve unmanaged target files and report conflicts instead of overwriting them. -6. Audit target-local instructions, prompts, skills, and agents that fall outside the selected sync baseline and report strict validation gaps or legacy alias drift. +6. Audit target-local instructions, prompts, skills, and agents that fall outside the selected sync baseline and report strict validation gaps, `local-*` naming violations for repository-owned prompt/skill/agent assets, or legacy alias drift. 7. Render `AGENTS.md` inventory from the desired managed baseline plus the Copilot assets already present in the target repository. 8. Report source-side audit findings separately from target-side unmanaged asset issues, redundant or legacy target assets, and file actions. diff --git a/.github/scripts/tech-ai-sync-copilot-configs.py b/.github/scripts/tech-ai-sync-copilot-configs.py index 0c7fdb0..dbc9773 100644 --- a/.github/scripts/tech-ai-sync-copilot-configs.py +++ b/.github/scripts/tech-ai-sync-copilot-configs.py @@ -487,6 +487,24 @@ def prompt_expected_name(relative_path: str) -> str | None: return None +def local_asset_identifier(relative_path: str) -> str | None: + path = Path(relative_path) + category = asset_category(relative_path) + + if category == "prompts" and path.name.endswith(".prompt.md"): + return path.name[: -len(".prompt.md")] + if category == "agents" and path.name.endswith(".agent.md"): + return path.name[: -len(".agent.md")] + if category == "skills" and path.name == "SKILL.md": + return path.parent.name + return None + + +def is_local_asset_path(relative_path: str) -> bool: + identifier = local_asset_identifier(relative_path) + return bool(identifier and identifier.startswith("local-")) + + def scan_repo_files(repo_root: Path) -> list[Path]: files: list[Path] = [] for path in repo_root.rglob("*"): @@ -1227,7 +1245,7 @@ def merged_inventory_paths(target_root: Path, selection: AssetSelection) -> dict } -def validate_unmanaged_prompt_asset(target_root: Path, relative_path: str) -> list[str]: +def validate_unmanaged_prompt_asset(target_root: Path, relative_path: str, repo_local: bool = False) -> list[str]: path = target_root / relative_path frontmatter = parse_frontmatter(path) issues: list[str] = [] @@ -1244,6 +1262,15 @@ def validate_unmanaged_prompt_asset(target_root: Path, relative_path: str) -> li if expected_name and actual_name and actual_name != expected_name: issues.append(f"Prompt name policy mismatch: expected `{expected_name}`, found `{actual_name}`.") + if repo_local: + if not is_local_asset_path(relative_path): + issues.append("Repository-local prompt filename must start with `local-`.") + if actual_name and not actual_name.startswith("local-"): + issues.append("Repository-local prompt `name` must start with `local-`.") + local_identifier = local_asset_identifier(relative_path) + if local_identifier and local_identifier.startswith("local-") and actual_name and actual_name != local_identifier: + issues.append(f"Repository-local prompt `name` should match filename stem `{local_identifier}`.") + for heading in ("## Instructions", "## Validation", "## Minimal example"): if not has_heading_exact(path, heading): issues.append(f"Missing `{heading}` section.") @@ -1260,7 +1287,7 @@ def validate_unmanaged_prompt_asset(target_root: Path, relative_path: str) -> li return issues -def validate_unmanaged_skill_asset(target_root: Path, relative_path: str) -> list[str]: +def validate_unmanaged_skill_asset(target_root: Path, relative_path: str, repo_local: bool = False) -> list[str]: path = target_root / relative_path frontmatter = parse_frontmatter(path) issues: list[str] = [] @@ -1274,10 +1301,20 @@ def validate_unmanaged_skill_asset(target_root: Path, relative_path: str) -> lis if not has_heading_regex(path, r"^## (Validation|Checklist|Testing|Test stack)$"): issues.append("Missing validation/testing section.") + if repo_local: + local_identifier = local_asset_identifier(relative_path) + actual_name = frontmatter.get("name", "") + if not is_local_asset_path(relative_path): + issues.append("Repository-local skill directory must start with `local-`.") + if actual_name and not actual_name.startswith("local-"): + issues.append("Repository-local skill `name` must start with `local-`.") + if local_identifier and local_identifier.startswith("local-") and actual_name and actual_name != local_identifier: + issues.append(f"Repository-local skill `name` should match directory name `{local_identifier}`.") + return issues -def validate_unmanaged_agent_asset(target_root: Path, relative_path: str) -> list[str]: +def validate_unmanaged_agent_asset(target_root: Path, relative_path: str, repo_local: bool = False) -> list[str]: path = target_root / relative_path frontmatter = parse_frontmatter(path) issues: list[str] = [] @@ -1293,10 +1330,20 @@ def validate_unmanaged_agent_asset(target_root: Path, relative_path: str) -> lis if not has_heading_exact(path, "## Restrictions"): issues.append("Missing `## Restrictions` section.") + if repo_local: + local_identifier = local_asset_identifier(relative_path) + actual_name = frontmatter.get("name", "") + if not is_local_asset_path(relative_path): + issues.append("Repository-local agent filename must start with `local-`.") + if actual_name and not actual_name.startswith("local-"): + issues.append("Repository-local agent `name` must start with `local-`.") + if local_identifier and local_identifier.startswith("local-") and actual_name and actual_name != local_identifier: + issues.append(f"Repository-local agent `name` should match filename stem `{local_identifier}`.") + return issues -def validate_unmanaged_instruction_asset(target_root: Path, relative_path: str) -> list[str]: +def validate_unmanaged_instruction_asset(target_root: Path, relative_path: str, repo_local: bool = False) -> list[str]: path = target_root / relative_path frontmatter = parse_frontmatter(path) issues: list[str] = [] @@ -1335,14 +1382,17 @@ def detect_unmanaged_target_asset_issues( issue_types: list[str] = [] details: list[str] = [] canonical_source_path = alias_map.get(relative_path) + repo_local = canonical_source_path is None and not (source_root / relative_path).is_file() if canonical_source_path: issue_types.append("legacy_alias") details.append(f"Legacy alias of `{canonical_source_path}`.") - validation_issues = validator(target_root, relative_path) + validation_issues = validator(target_root, relative_path, repo_local=repo_local) if validation_issues: issue_types.append("validation") + if any(detail.startswith("Repository-local ") for detail in validation_issues): + issue_types.append("local_naming") details.extend(validation_issues) if not issue_types: @@ -1642,6 +1692,8 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, "- Use GitHub Copilot terminology in repository-facing content.", "- Do not mention internal runtime names in repository artifacts.", "- Treat prompt frontmatter `name:` as the canonical command identifier.", + "- Repository-local prompt, skill, and agent filenames must start with `local-`.", + "- Repository-local prompt, skill, and agent `name:` values must also start with `local-`.", "", "## Decision Priority", "1. Apply repository non-negotiables from `copilot-instructions.md`.", @@ -1839,6 +1891,15 @@ def build_recommendations( f"{', '.join(validation_issue_paths)}." ) + local_naming_issue_paths = [ + issue.target_relative_path for issue in target_asset_issues if "local_naming" in issue.issue_types + ] + if local_naming_issue_paths: + recommendations["missing consumer-facing validation or onboarding guidance"].append( + "Repository-local Copilot prompts, skills, and agents should use `local-*` in both filenames and " + f"`name:` values: {', '.join(local_naming_issue_paths)}." + ) + if any(action.target_relative_path == analysis.agents_relative_path and action.status == "conflict" for action in actions): recommendations["weak conflict-handling rules"].append( "Consider generated section markers or a dedicated root-AGENTS template to reduce consumer AGENTS " diff --git a/.github/scripts/validate-copilot-customizations.sh b/.github/scripts/validate-copilot-customizations.sh index d5db546..ce2e429 100755 --- a/.github/scripts/validate-copilot-customizations.sh +++ b/.github/scripts/validate-copilot-customizations.sh @@ -334,6 +334,131 @@ tech_ai_prompt_name() { printf '%s' "$output" } +local_asset_identifier() { + local file="$1" + local base + local parent + + case "$file" in + *.prompt.md) + base="$(basename "$file")" + printf '%s' "${base%.prompt.md}" + ;; + *.agent.md) + base="$(basename "$file")" + printf '%s' "${base%.agent.md}" + ;; + */SKILL.md) + parent="$(basename "$(dirname "$file")")" + printf '%s' "$parent" + ;; + *) + return 1 + ;; + esac +} + +validate_repo_local_prompt_naming() { + local file="$1" + local severity="error" + local actual_name="" + local expected_local_name="" + + [[ "$MODE" == "legacy-compatible" ]] && severity="warn" + + actual_name="$(frontmatter_value "$file" "name")" + expected_local_name="$(local_asset_identifier "$file" || true)" + + case "$(basename "$file")" in + tech-ai-*.prompt.md) + return 0 + ;; + local-*.prompt.md) + if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then + record_issue "$severity" "Repository-local prompt name must start with 'local-': ${file}" + fi + if [[ -n "$actual_name" && -n "$expected_local_name" && "$actual_name" != "$expected_local_name" ]]; then + record_issue "$severity" "Repository-local prompt name must match filename stem '${expected_local_name}': ${file}" + fi + return 0 + ;; + *) + record_issue "$severity" "Repository-local prompt filename must start with 'local-': ${file}" + if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then + record_issue "$severity" "Repository-local prompt name must start with 'local-': ${file}" + fi + ;; + esac +} + +validate_repo_local_skill_naming() { + local file="$1" + local severity="error" + local actual_name="" + local expected_local_name="" + local skill_dir + + [[ "$MODE" == "legacy-compatible" ]] && severity="warn" + + actual_name="$(frontmatter_value "$file" "name")" + expected_local_name="$(local_asset_identifier "$file" || true)" + skill_dir="$(basename "$(dirname "$file")")" + + case "$skill_dir" in + tech-ai-*) + return 0 + ;; + local-*) + if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then + record_issue "$severity" "Repository-local skill name must start with 'local-': ${file}" + fi + if [[ -n "$actual_name" && -n "$expected_local_name" && "$actual_name" != "$expected_local_name" ]]; then + record_issue "$severity" "Repository-local skill name must match directory name '${expected_local_name}': ${file}" + fi + return 0 + ;; + *) + record_issue "$severity" "Repository-local skill directory must start with 'local-': ${file}" + if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then + record_issue "$severity" "Repository-local skill name must start with 'local-': ${file}" + fi + ;; + esac +} + +validate_repo_local_agent_naming() { + local file="$1" + local severity="error" + local actual_name="" + local expected_local_name="" + + [[ "$MODE" == "legacy-compatible" ]] && severity="warn" + + actual_name="$(frontmatter_value "$file" "name")" + expected_local_name="$(local_asset_identifier "$file" || true)" + + case "$(basename "$file")" in + tech-ai-*.agent.md) + return 0 + ;; + local-*.agent.md) + if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then + record_issue "$severity" "Repository-local agent name must start with 'local-': ${file}" + fi + if [[ -n "$actual_name" && -n "$expected_local_name" && "$actual_name" != "$expected_local_name" ]]; then + record_issue "$severity" "Repository-local agent name must match filename stem '${expected_local_name}': ${file}" + fi + return 0 + ;; + *) + record_issue "$severity" "Repository-local agent filename must start with 'local-': ${file}" + if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then + record_issue "$severity" "Repository-local agent name must start with 'local-': ${file}" + fi + ;; + esac +} + validate_prompt_name_policy() { local file="$1" local expected_name @@ -450,6 +575,7 @@ validate_prompt_file() { fi validate_prompt_name_policy "$file" + validate_repo_local_prompt_naming "$file" if ! has_heading_exact "$file" '## Instructions'; then record_issue "$section_severity" "Prompt missing '## Instructions' section: ${file}" @@ -517,6 +643,7 @@ validate_skill_file() { [[ "$MODE" == "legacy-compatible" ]] && section_severity="warn" check_required_keys "$file" error name description + validate_repo_local_skill_naming "$file" if ! has_heading_regex "$file" '^## When to [Uu]se$'; then record_issue "$section_severity" "Skill missing '## When to use' section: ${file}" @@ -547,6 +674,7 @@ validate_agents_dir() { while IFS= read -r file; do count=$((count + 1)) check_required_keys "$file" error name description tools + validate_repo_local_agent_naming "$file" if ! grep -Eq '^# ' "$file"; then record_error "Agent missing top heading: ${file}" diff --git a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md index 0d1f03c..cee2632 100644 --- a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md +++ b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md @@ -23,6 +23,7 @@ description: Analyze a local repository, select the minimum Copilot customizatio 6. Detect redundant legacy aliases for canonical prompt/skill/agent families, including families that are not part of the selected minimum baseline, so `cs-*` and unprefixed leftovers still appear in the plan. 7. Audit target-local instructions, prompts, skills, and agents that are outside the selected sync baseline: - report strict validation gaps for unmanaged files; + - require repository-owned prompts, skills, and agents to use `local-*` in both filenames and `name:` values so they are visually distinct from synced global assets; - report legacy aliases even when the canonical family is not selected; - keep target-only custom assets visible instead of silently omitting them. 8. Treat redundant canonical-vs-legacy overlaps as conflicts instead of silently creating duplicate configuration families. diff --git a/.github/templates/AGENTS.template.md b/.github/templates/AGENTS.template.md index 963ab06..18703b2 100644 --- a/.github/templates/AGENTS.template.md +++ b/.github/templates/AGENTS.template.md @@ -6,6 +6,8 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Use GitHub Copilot terminology in repository-facing content. - Keep repository-facing text in English. - Treat prompt frontmatter `name:` as the canonical command identifier. +- Repository-local prompts, skills, and agents must use the `local-*` prefix in filenames. +- Repository-local prompt, skill, and agent `name:` values must also use the `local-*` prefix. ## Decision Priority 1. Apply repository non-negotiables from `.github/copilot-instructions.md`. diff --git a/AGENTS.md b/AGENTS.md index 2cb4cf8..68f71a6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,6 +9,8 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Treat prompt frontmatter `name:` as the canonical command identifier. - Canonical repository-owned prompt, agent, and instruction filenames should use the `tech-ai-` prefix when introduced or renamed. - Canonical prompt, skill, and agent `name:` values should use the `TechAI` prefix. +- Repository-local prompt, skill, and agent filenames in consumer repositories should use the `local-` prefix. +- Repository-local prompt, skill, and agent `name:` values in consumer repositories should also use the `local-` prefix. - Reserve the `TechAIGlobal` prefix only for repo-only agents that encode standards for this global configuration repository. - The canonical project-owned `AGENTS.md` file must live in repository root as `AGENTS.md`. - Keep legacy aliases only when required for backward compatibility, and prefer canonical `tech-ai-*` assets in docs, examples, and sync selection. diff --git a/tests/test_tech_ai_sync_copilot_configs.py b/tests/test_tech_ai_sync_copilot_configs.py index f952c4a..0208feb 100644 --- a/tests/test_tech_ai_sync_copilot_configs.py +++ b/tests/test_tech_ai_sync_copilot_configs.py @@ -325,12 +325,18 @@ def test_build_plan_reports_unmanaged_target_assets_and_legacy_aliases_outside_s ) assert "validation" in issue_by_path[".github/prompts/add-external-user.prompt.md"].issue_types + assert "local_naming" in issue_by_path[".github/prompts/add-external-user.prompt.md"].issue_types assert "Missing frontmatter key `name`." in issue_by_path[".github/prompts/add-external-user.prompt.md"].details + assert ( + "Repository-local prompt filename must start with `local-`." + in issue_by_path[".github/prompts/add-external-user.prompt.md"].details + ) assert "legacy_alias" in issue_by_path[".github/prompts/cs-data-registry.prompt.md"].issue_types assert ( issue_by_path[".github/prompts/cs-data-registry.prompt.md"].canonical_source_path == ".github/prompts/tech-ai-data-registry.prompt.md" ) + assert ".github/skills/local-registry/SKILL.md" not in issue_by_path assert ".github/prompts/add-external-user.prompt.md" in agents_file.desired_content assert ".github/prompts/cs-data-registry.prompt.md" in agents_file.desired_content @@ -338,6 +344,62 @@ def test_build_plan_reports_unmanaged_target_assets_and_legacy_aliases_outside_s assert ".github/skills/local-registry/SKILL.md" in agents_file.desired_content +def test_build_plan_accepts_local_prefixed_repo_owned_assets(tmp_path: Path) -> None: + target_root = tmp_path / "local-assets" + build_python_service_target(target_root) + write_file( + target_root / ".github" / "prompts" / "local-add-external-user.prompt.md", + "\n".join( + [ + "---", + "name: local-add-external-user", + "description: Add an external user to Entra ID.", + "agent: agent", + "argument-hint: user=", + "---", + "", + "# Local Add External User", + "", + "## Instructions", + "1. Use `.github/skills/local-entra-access/SKILL.md`.", + "", + "## Validation", + "- Validate the repository-local registry update.", + "", + "## Minimal example", + "- Input: `user=guest@example.com`", + "", + ] + ), + ) + write_file( + target_root / ".github" / "skills" / "local-entra-access" / "SKILL.md", + "\n".join( + [ + "---", + "name: local-entra-access", + "description: Repository-local Entra access workflow.", + "---", + "", + "# Local Entra Access", + "", + "## When to use", + "- Manage repository-local Entra access automation.", + "", + "## Validation", + "- Validate repository-local access rules.", + "", + ] + ), + ) + + plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + issue_paths = {issue.target_relative_path for issue in plan.target_asset_issues} + assert ".github/prompts/local-add-external-user.prompt.md" not in issue_paths + assert ".github/skills/local-entra-access/SKILL.md" not in issue_paths + + def test_apply_plan_writes_manifest_and_managed_files(tmp_path: Path) -> None: target_root = tmp_path / "fresh-target" write_file(target_root / ".github" / "PULL_REQUEST_TEMPLATE.md", "# PR template\n") diff --git a/tests/test_tech_ai_validate_copilot_customizations.py b/tests/test_tech_ai_validate_copilot_customizations.py index a6f1bc1..5fa5042 100644 --- a/tests/test_tech_ai_validate_copilot_customizations.py +++ b/tests/test_tech_ai_validate_copilot_customizations.py @@ -69,6 +69,73 @@ def test_tech_ai_validator_reports_missing_prompt_argument_hint(tmp_path: Path) assert any("Missing frontmatter key 'argument-hint'" in message for message in messages) +def test_tech_ai_validator_requires_local_prefix_for_repo_owned_assets(tmp_path: Path) -> None: + target_root = tmp_path / "invalid-local-assets" + copy_copilot_config(target_root) + + (target_root / ".github" / "prompts").mkdir(parents=True, exist_ok=True) + (target_root / ".github" / "skills" / "user-admin").mkdir(parents=True, exist_ok=True) + (target_root / ".github" / "prompts" / "add-external-user.prompt.md").write_text( + "\n".join( + [ + "---", + "name: add-external-user", + "description: Add an external user", + "agent: agent", + "argument-hint: user=", + "---", + "", + "# Add External User", + "", + "## Instructions", + "1. Use `.github/skills/user-admin/SKILL.md`.", + "", + "## Validation", + "- Validate the external user request.", + "", + "## Minimal example", + "- Input: `user=guest@example.com`", + "", + ] + ) + + "\n", + encoding="utf-8", + ) + (target_root / ".github" / "skills" / "user-admin" / "SKILL.md").write_text( + "\n".join( + [ + "---", + "name: user-admin", + "description: Repository-local user administration workflow.", + "---", + "", + "# User Admin", + "", + "## When to use", + "- Manage repository-local user administration.", + "", + "## Validation", + "- Validate repository-local access state.", + "", + ] + ) + + "\n", + encoding="utf-8", + ) + + report_file = tmp_path / "tech-ai-validator-local-prefix.json" + result = run_validator(target_root, report_file) + + payload = json.loads(report_file.read_text(encoding="utf-8")) + messages = [finding["message"] for finding in payload["findings"]] + assert result.returncode == 1 + assert payload["status"] == "failed" + assert any("Repository-local prompt filename must start with 'local-'" in message for message in messages) + assert any("Repository-local prompt name must start with 'local-'" in message for message in messages) + assert any("Repository-local skill directory must start with 'local-'" in message for message in messages) + assert any("Repository-local skill name must start with 'local-'" in message for message in messages) + + def test_tech_ai_validator_requires_root_agents_file(tmp_path: Path) -> None: target_root = tmp_path / "legacy-agents-repo" shutil.copytree(REPO_ROOT / ".github", target_root / ".github") From 2aeffcf8c81aaf6413b6b7f67d1bf4b04c2fd3e8 Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Mon, 9 Mar 2026 11:56:03 +0100 Subject: [PATCH 4/9] feat: update PR-writing guidance to derive sections from repository template and enhance validation criteria --- .github/CHANGELOG.md | 1 + .../agents/tech-ai-github-pr-writer.agent.md | 2 +- .../tech-ai-github-pr-description.prompt.md | 21 ++++++++----------- .github/skills/tech-ai-pr-writing/SKILL.md | 17 +++++++-------- 4 files changed, 18 insertions(+), 23 deletions(-) diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index 1459706..66ae3b8 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -7,6 +7,7 @@ Use this format for new updates: - Include file/path scope when useful. ## 2026-03-08 +- Updated the PR-writing prompt, skill, and agent guidance to derive required sections from the resolved repository PR template instead of hardcoding older headings such as `Security and Compliance` or `Related Links`. - Updated `scripts/tech-ai-sync-copilot-configs.py` and `scripts/validate-copilot-customizations.sh` so repository-owned prompt, skill, and agent assets outside the synced global baseline must use `local-*` in both filenames and `name:` values, making local customizations visibly distinct from synced `tech-ai-*` assets. - Updated `scripts/tech-ai-sync-copilot-configs.py` so target-only skill detection compares full relative paths instead of the shared `SKILL.md` filename, fixing missed unmanaged skill assets in consumer repositories. - Expanded sync planning to audit unmanaged target-local instructions, prompts, skills, and agents for strict validation gaps and legacy alias drift, and added the new report section in both markdown and JSON outputs. diff --git a/.github/agents/tech-ai-github-pr-writer.agent.md b/.github/agents/tech-ai-github-pr-writer.agent.md index 5f73b1e..a5a296a 100644 --- a/.github/agents/tech-ai-github-pr-writer.agent.md +++ b/.github/agents/tech-ai-github-pr-writer.agent.md @@ -30,7 +30,7 @@ Produce and apply a complete PR title/body aligned with the repository template, 6. Answer every template prompt/question explicitly with repository facts. Never leave placeholder bullets empty. 7. Preserve checklist items and mark each item intentionally (`[x]` or `[ ]`) based on real change scope. 8. Use `N/A` only when a section is truly not applicable. -9. Ensure `Validation`, `Security and Compliance`, and `Risk and Rollback` are explicit and complete. +9. Ensure `Validation`, `Risk and Rollback`, and any repository-specific governance or target-context sections are explicit and complete. 10. Re-fetch the PR and confirm persisted body contains all template headings and checklist items. 11. Return PR URL and a short confirmation summary. diff --git a/.github/prompts/tech-ai-github-pr-description.prompt.md b/.github/prompts/tech-ai-github-pr-description.prompt.md index c5c576d..1716069 100644 --- a/.github/prompts/tech-ai-github-pr-description.prompt.md +++ b/.github/prompts/tech-ai-github-pr-description.prompt.md @@ -18,15 +18,11 @@ Create or update a pull request body using the repository template (`.github/pul - **Risk**: ${input:risk:Low,Medium,High} - **Links**: ${input:links:N/A} -## Required section headings -- `## Summary` -- `## Scope` -- `## Changes` -- `## Validation` -- `## Security and Compliance` -- `## Risk and Rollback` -- `## Related Links` -- `## Reviewer Notes` +## Template-derived structure +- Use the exact section headings from the resolved repository template. +- Keep template section order unchanged. +- Preserve any checklist section and mark items intentionally. +- Do not add extra sections unless the template already includes them. ## Instructions 1. Use `.github/skills/tech-ai-pr-writing/SKILL.md`. @@ -35,12 +31,12 @@ Create or update a pull request body using the repository template (`.github/pul - `.github/PULL_REQUEST_TEMPLATE.md` - `pull_request_template.md` - `PULL_REQUEST_TEMPLATE.md` -3. Follow template section order and headings exactly. +3. Follow template section order and headings exactly as defined by the resolved template. 4. Answer every prompt/question line from the template explicitly with repository facts. 5. Preserve checklist items and mark each one intentionally (`[x]` or `[ ]`) based on real scope. 6. Do not remove required template sections; fill not-applicable content with `N/A`. 7. In `Changes`, provide brief bullets aligned to `changed_files`. -8. In `Scope`, check only applicable categories. +8. In scope or target-context sections, fill repository-specific fields explicitly when relevant, such as environments, services, identities, subscriptions, projects, or temporary access. 9. Keep content concise, concrete, and in English. 10. If PR tools are available: - update existing PR when found @@ -57,8 +53,9 @@ Create or update a pull request body using the repository template (`.github/pul - PR URL and confirmation that the PR body was updated (when tools are available). ## Validation -- Confirm all required section headings are present and non-empty (or `N/A`). +- Confirm all template-defined section headings are present and non-empty (or `N/A`). - Confirm template checklist lines are present and intentionally marked. - Confirm `Changes` bullets are brief and aligned with modified files. +- Confirm repository-specific scope or target fields are explicit when applicable. - Confirm risk level and rollback plan are explicitly included. - Confirm final PR body is persisted, not only generated as draft text. diff --git a/.github/skills/tech-ai-pr-writing/SKILL.md b/.github/skills/tech-ai-pr-writing/SKILL.md index 02e884f..56ca6ff 100644 --- a/.github/skills/tech-ai-pr-writing/SKILL.md +++ b/.github/skills/tech-ai-pr-writing/SKILL.md @@ -39,15 +39,11 @@ description: Produce concise, complete pull request descriptions aligned with th 6. Return PR URL and a concise confirmation summary. 7. If PR tools are unavailable, return ready-to-paste markdown plus exact CLI fallback commands. -## Required section headings -- `## Summary` -- `## Scope` -- `## Changes` -- `## Validation` -- `## Security and Compliance` -- `## Risk and Rollback` -- `## Related Links` -- `## Reviewer Notes` +## Template-derived structure +- Use the exact section headings from the resolved repository template. +- Keep section order unchanged. +- Preserve checklist items and prompt questions from the template. +- Do not add or remove template sections unless explicitly requested. ## Minimal example - Input: @@ -59,7 +55,8 @@ description: Produce concise, complete pull request descriptions aligned with th - A brief and accurate bullet list under `Changes`. ## Validation -- Ensure every required section heading is present. +- Ensure every template-defined section heading is present. - Ensure `Changes` has concise bullets describing the real diff. +- Ensure repository-specific scope or target fields are explicit when applicable. - Ensure risk and rollback are explicit and actionable. - Ensure final PR body is persisted when tooling supports PR updates. From 3e224ce642a672c71af7789c038dbcd18f3faa1b Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Mon, 9 Mar 2026 16:40:25 +0100 Subject: [PATCH 5/9] feat: Introduce comprehensive Copilot configuration review and contributing guidelines - Added `COPILOT_REVIEW.md` for actionable findings on GitHub Copilot customization assets. - Created `CONTRIBUTING.md` to document contribution processes and naming conventions. - Implemented `Makefile` for standardized developer commands including linting, validation, and testing. - Established `VERSION` file for versioning strategy to enable consumer pinning. - Updated `ANALYSIS_REPORT.md` to indicate it is superseded by the new review document. - Enhanced test coverage in `tests/test_tech_ai_sync_copilot_configs.py` and `tests/test_tech_ai_validate_copilot_customizations.py` for improved validation. - Refined `CODEOWNERS` to include broader ownership and reduce bus factor. - Cleaned up `dependabot.yml` to remove unused package ecosystems. - Added Docker instructions and corresponding assets to support consumer repositories using Docker. --- .github/.bootstrap-ignore | 5 + .github/CHANGELOG.md | 8 + .github/DEPRECATION.md | 2 +- .github/README.md | 12 +- .github/agents/README.md | 11 + ...cal-copilot-customization-builder.agent.md | 30 + .github/dependabot.yml | 26 - .../github-action-composite.instructions.md | 2 +- .../github-actions.instructions.md | 2 +- ...al-copilot-customization-builder.prompt.md | 38 ++ .github/scripts/bootstrap-copilot-config.sh | 4 + .../scripts/tech-ai-sync-copilot-configs.py | 176 ++++- .../validate-copilot-customizations.sh | 23 + .github/security-baseline.md | 14 + .../SKILL.md | 51 ++ .github/templates/copilot-quickstart.md | 8 +- AGENTS.md | 11 +- ANALYSIS_REPORT.md | 2 + CODEOWNERS | 1 + CONTRIBUTING.md | 33 + COPILOT_REVIEW.md | 633 ++++++++++++++++++ Makefile | 21 + VERSION | 1 + tests/test_tech_ai_sync_copilot_configs.py | 105 ++- ...tech_ai_validate_copilot_customizations.py | 106 ++- 25 files changed, 1255 insertions(+), 70 deletions(-) create mode 100644 .github/agents/tech-ai-local-copilot-customization-builder.agent.md create mode 100644 .github/prompts/tech-ai-local-copilot-customization-builder.prompt.md create mode 100644 .github/skills/tech-ai-local-copilot-customization-builder/SKILL.md create mode 100644 CONTRIBUTING.md create mode 100644 COPILOT_REVIEW.md create mode 100644 Makefile create mode 100644 VERSION diff --git a/.github/.bootstrap-ignore b/.github/.bootstrap-ignore index b77309c..dcf9e40 100644 --- a/.github/.bootstrap-ignore +++ b/.github/.bootstrap-ignore @@ -4,3 +4,8 @@ # Example: # workflows/ # dependabot.yml +README.md +agents/README.md +templates/ +scripts/bootstrap-copilot-config.sh +tech-ai-requirements-dev.txt diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index 66ae3b8..7c2f345 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -6,6 +6,14 @@ Use this format for new updates: - One bullet per meaningful change. - Include file/path scope when useful. +## 2026-03-09 +- Added the repo-only `TechAILocalCopilotCustomizationBuilder` agent, prompt, and skill for creating consumer-local `local-*` Copilot assets without duplicating the shared baseline, and excluded the trio from consumer sync. +- Deprecated `.github/scripts/bootstrap-copilot-config.sh` in favor of `.github/scripts/tech-ai-sync-copilot-configs.py`, updated lifecycle docs, and made quickstart plus `.github/README.md` prefer sync-first alignment. +- Added source release metadata with root `VERSION`, contributor workflow documentation, and manifest provenance fields for source version and commit. +- Tightened consumer alignment: improved composite-action detection, enabled data-registry selection for JSON-heavy repositories, slimmed generated `AGENTS.md`, removed spurious `pytest` recommendations for repos without pytest tests, and added sync recommendations for missing Copilot validation workflows plus legacy source-only residues. +- Reduced source maintenance noise by trimming Dependabot ecosystems, updating the GitHub Actions checkout example, adding explicit `.github/` CODEOWNERS coverage, and documenting security-control enforcement status. +- Expanded validator and sync tests to cover new recommendation, rendering, provenance, and validation paths. + ## 2026-03-08 - Updated the PR-writing prompt, skill, and agent guidance to derive required sections from the resolved repository PR template instead of hardcoding older headings such as `Security and Compliance` or `Related Links`. - Updated `scripts/tech-ai-sync-copilot-configs.py` and `scripts/validate-copilot-customizations.sh` so repository-owned prompt, skill, and agent assets outside the synced global baseline must use `local-*` in both filenames and `name:` values, making local customizations visibly distinct from synced `tech-ai-*` assets. diff --git a/.github/DEPRECATION.md b/.github/DEPRECATION.md index 31273a2..5dda2bd 100644 --- a/.github/DEPRECATION.md +++ b/.github/DEPRECATION.md @@ -25,4 +25,4 @@ Define a predictable process for deprecating Copilot customization assets (`inst Immediate removal is allowed only for security or compliance issues. The removal reason must be documented in `.github/CHANGELOG.md`. ## Current deprecations -- None at this time. +- `scripts/bootstrap-copilot-config.sh`: Deprecated in favor of `scripts/tech-ai-sync-copilot-configs.py`. Keep only as a legacy bootstrap fallback while older consumers migrate to sync-based alignment. diff --git a/.github/README.md b/.github/README.md index 51e9398..26a5012 100644 --- a/.github/README.md +++ b/.github/README.md @@ -28,13 +28,15 @@ See `.github/agents/README.md` for details. 1. Update files under `.github/`. 2. Run `.github/scripts/validate-copilot-customizations.sh --scope root --mode strict`. 3. Optional: generate a machine-readable summary with `.github/scripts/validate-copilot-customizations.sh --scope root --mode strict --report json --report-file /tmp/copilot-report.json`. -4. Optional: bootstrap this configuration into another repository with `.github/scripts/bootstrap-copilot-config.sh --target ` (default excludes apply; see `.github/.bootstrap-ignore`). -5. Optionally run cross-repo assessment with `.github/scripts/validate-copilot-customizations.sh --scope all --mode legacy-compatible`. -6. Ensure workflow checks pass. -7. Update `.github/CHANGELOG.md` for notable changes. +4. Prefer cross-repo alignment with `python .github/scripts/tech-ai-sync-copilot-configs.py --target --mode plan` before any apply step. +5. Use `.github/scripts/bootstrap-copilot-config.sh --target ` only as a legacy fallback bootstrap path (default excludes apply; see `.github/.bootstrap-ignore`). +6. Optionally run cross-repo assessment with `.github/scripts/validate-copilot-customizations.sh --scope all --mode legacy-compatible`. +7. Ensure workflow checks pass. +8. Update `.github/CHANGELOG.md` for notable changes. ## Notes - `repo-profiles.yml` is currently advisory (human-readable profile catalog). - The canonical project `AGENTS.md` belongs in repository root, not under `.github/`. -- `TechAIGlobalCustomizationBuilder` and `TechAIGlobalCustomizationAuditor` are repo-only standards agents and must not be synced to consumer repositories. +- `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`, `TechAILocalCopilotCustomizationBuilder`, and `TechAISyncCopilotConfigs` are repo-only source agents and must not be synced to consumer repositories. +- `.github/README.md`, `.github/agents/README.md`, `.github/templates/**`, and `.github/scripts/bootstrap-copilot-config.sh` are source-only assets and should not be part of consumer baselines. - Use `templates/copilot-quickstart.md` for a short onboarding flow. diff --git a/.github/agents/README.md b/.github/agents/README.md index 5901fdd..b9a8eaa 100644 --- a/.github/agents/README.md +++ b/.github/agents/README.md @@ -13,6 +13,15 @@ This folder contains optional custom agents for focused tasks. - PR-focused: `TechAIPRWriter`. - Write-capable: `TechAIImplementer`. - Repo-only standards specialists: `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`. +- Repo-only consumer-local specialist: `TechAILocalCopilotCustomizationBuilder`. + +## Repo-only agents (not synced to consumers) +- `TechAIGlobalCustomizationBuilder` +- `TechAIGlobalCustomizationAuditor` +- `TechAILocalCopilotCustomizationBuilder` +- `TechAIScriptReviewer` +- `TechAISyncCopilotConfigs` +- `TechAICustomizationAuditor` (deprecated compatibility alias) ## Why generic core agents - `TechAIPlanner`, `TechAIImplementer`, and `TechAIReviewer` are workflow roles, not language roles. @@ -29,3 +38,5 @@ This folder contains optional custom agents for focused tasks. 7. Use `TechAISecurityReviewer` as final security gate. 8. Use `TechAIGlobalCustomizationBuilder` for GitHub Copilot customization assets in this standards repository. 9. Use `TechAIGlobalCustomizationAuditor` as the final gate for those customization changes. +10. Use `TechAISyncCopilotConfigs` to align a consumer baseline before creating repo-owned local assets. +11. Use `TechAILocalCopilotCustomizationBuilder` for repo-owned `local-*` prompts, skills, agents, and `AGENTS.md` updates that should stay consumer-local. diff --git a/.github/agents/tech-ai-local-copilot-customization-builder.agent.md b/.github/agents/tech-ai-local-copilot-customization-builder.agent.md new file mode 100644 index 0000000..eff0873 --- /dev/null +++ b/.github/agents/tech-ai-local-copilot-customization-builder.agent.md @@ -0,0 +1,30 @@ +--- +description: Create or update repository-owned local GitHub Copilot customization assets in a consumer repository without duplicating the shared baseline. +name: TechAILocalCopilotCustomizationBuilder +tools: ["search", "usages", "problems", "editFiles", "runTerminal", "fetch"] +--- + +# TechAI Local Copilot Customization Builder Agent + +## Objective +Create and refine consumer-repository Copilot customization assets that must remain local, using the `local-*` naming convention, preserving the synced baseline, and keeping the target `AGENTS.md` plus validation state coherent. + +## Restrictions +- Do not modify target `README.md` files unless explicitly requested. +- Do not create repository-owned prompt, skill, or agent assets with the `tech-ai-*` filename prefix or `TechAI*` name values; use `local-*` for both filenames and frontmatter `name:`. +- Do not duplicate a capability that already exists in the synced baseline unless the requested behavior is genuinely repository-specific. +- Do not overwrite manifest-managed synced files unless explicitly requested and conflict-safe. +- Do not sync workflows, templates, changelog files, or bootstrap helpers from the source repository as part of local customization work. +- Keep repository-facing text in English and use GitHub Copilot terminology only. + +## Routing +- Use this agent when a consumer repository needs repo-owned prompts, skills, agents, or `AGENTS.md` wiring that must stay local. +- If the consumer baseline is missing or stale, start with `TechAISyncCopilotConfigs` in `plan` mode before creating new local assets. +- Treat `.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md` as the workflow definition. + +## Output Contract +- `Baseline check`: whether the consumer already has the required synced Copilot core assets and validator coverage. +- `Local customization decision`: why a new `local-*` asset is needed instead of reusing an existing `tech-ai-*` capability. +- `File plan`: `local-*` prompts, skills, agents, and `AGENTS.md` updates to create or modify. +- `Validation`: target-repository validation commands run and their results. +- `Promotion note`: whether the local capability should remain repo-only or be a candidate for promotion back to `cloud-strategy.github`. diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 46c16d7..aff00a6 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -16,14 +16,6 @@ updates: labels: - "dependencies" - "python" - - package-ecosystem: "npm" - directory: "/" - schedule: - interval: "weekly" - open-pull-requests-limit: 5 - labels: - - "dependencies" - - "nodejs" - package-ecosystem: "terraform" directory: "/" schedule: @@ -32,21 +24,3 @@ updates: labels: - "dependencies" - "terraform" - - package-ecosystem: "maven" - directory: "/" - schedule: - interval: "weekly" - open-pull-requests-limit: 5 - labels: - - "dependencies" - - "java" - - "maven" - - package-ecosystem: "gradle" - directory: "/" - schedule: - interval: "weekly" - open-pull-requests-limit: 5 - labels: - - "dependencies" - - "java" - - "gradle" diff --git a/.github/instructions/github-action-composite.instructions.md b/.github/instructions/github-action-composite.instructions.md index 0de2bfa..73a421b 100644 --- a/.github/instructions/github-action-composite.instructions.md +++ b/.github/instructions/github-action-composite.instructions.md @@ -1,6 +1,6 @@ --- description: Standards for secure, deterministic GitHub composite actions with explicit input validation. -applyTo: "**/actions/**/action.y*ml" +applyTo: "**/actions/**/action.y*ml,**/workflows/**/action.y*ml" --- # Composite Action Instructions diff --git a/.github/instructions/github-actions.instructions.md b/.github/instructions/github-actions.instructions.md index 77e1f6d..2d1e296 100644 --- a/.github/instructions/github-actions.instructions.md +++ b/.github/instructions/github-actions.instructions.md @@ -26,7 +26,7 @@ applyTo: "**/workflows/**" ```yaml steps: - name: Checkout - uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.7 https://github.com/actions/checkout/releases/tag/v4.1.7 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 https://github.com/actions/checkout/releases/tag/v6.0.2 concurrency: group: example-${{ github.ref }} diff --git a/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md b/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md new file mode 100644 index 0000000..028de02 --- /dev/null +++ b/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md @@ -0,0 +1,38 @@ +--- +description: Create or update repository-owned local GitHub Copilot customization assets in a consumer repo while preserving the shared baseline +name: TechAILocalCopilotCustomizationBuilder +agent: agent +argument-hint: target_repo= change= [local_asset_type=] [promote_to_source=] +--- + +# TechAI Local Copilot Customization Builder + +## Context +Use this prompt to create or refine repository-owned `local-*` Copilot assets in a consumer repository without duplicating the shared `tech-ai-*` baseline. + +## Required inputs +- **Target repository**: ${input:target_repo} +- **Requested change**: ${input:change} +- **Local asset type**: ${input:local_asset_type:prompt,skill,agent,triad} +- **Promote to source**: ${input:promote_to_source:no} + +## Instructions +1. Use `.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md` as the workflow definition. +2. If the target baseline is missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first. +3. Create only the narrowest local asset set that solves the request. +4. Keep repository-owned prompt, skill, and agent filenames plus frontmatter `name:` values on the `local-*` convention. +5. Update the target `AGENTS.md` inventory and routing only as needed, keeping `.github/...` paths explicit. +6. Do not create local duplicates of existing `tech-ai-*` capabilities unless the repo-specific behavior materially differs. +7. Report whether the capability should remain local or be proposed for promotion into the shared source baseline. + +## Minimal example +- Input: `target_repo=/workspace/consumer-repo change="Add a repo-local prompt for onboarding external users" local_asset_type=prompt` +- Expected output: + - Baseline check for synced Copilot core assets in the target repo. + - Minimal `local-*` asset plan with naming and placement rationale. + - `AGENTS.md` inventory and routing updates only if needed. + - Target-repo validation results and promotion recommendation. + +## Validation +- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` in the target repo after changing local Copilot assets. +- Run relevant Bash, Python, Terraform, YAML, JSON, or Markdown checks for the touched files. diff --git a/.github/scripts/bootstrap-copilot-config.sh b/.github/scripts/bootstrap-copilot-config.sh index ca4f249..0e96d8c 100755 --- a/.github/scripts/bootstrap-copilot-config.sh +++ b/.github/scripts/bootstrap-copilot-config.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash # +# ⚠️ DEPRECATED: Prefer .github/scripts/tech-ai-sync-copilot-configs.py for consumer alignment. +# This script is maintained for backward compatibility only. +# See .github/DEPRECATION.md for lifecycle policy. +# # Purpose: Bootstrap Copilot customization files from a source config root into a target repository `.github` folder. # Usage examples: # ./.github/scripts/bootstrap-copilot-config.sh --target /path/to/repo diff --git a/.github/scripts/tech-ai-sync-copilot-configs.py b/.github/scripts/tech-ai-sync-copilot-configs.py index dbc9773..25f44d8 100644 --- a/.github/scripts/tech-ai-sync-copilot-configs.py +++ b/.github/scripts/tech-ai-sync-copilot-configs.py @@ -38,6 +38,7 @@ ".github/agents/tech-ai-customization-auditor.agent.md", ".github/agents/tech-ai-global-customization-auditor.agent.md", ".github/agents/tech-ai-global-customization-builder.agent.md", + ".github/agents/tech-ai-local-copilot-customization-builder.agent.md", ".github/agents/tech-ai-script-reviewer.agent.md", ".github/agents/tech-ai-sync-copilot-configs.agent.md", } @@ -45,10 +46,12 @@ ".github/prompts/tech-ai-add-platform.prompt.md", ".github/prompts/tech-ai-add-report-script.prompt.md", ".github/prompts/tech-ai-code-review.prompt.md", + ".github/prompts/tech-ai-local-copilot-customization-builder.prompt.md", ".github/prompts/tech-ai-sync-copilot-configs.prompt.md", } SOURCE_ONLY_SKILL_PATHS = { ".github/skills/tech-ai-code-review/SKILL.md", + ".github/skills/tech-ai-local-copilot-customization-builder/SKILL.md", ".github/skills/tech-ai-sync-copilot-configs/SKILL.md", } CANONICAL_BASH_SCRIPT_PROMPT_PATH = ".github/prompts/tech-ai-bash-script.prompt.md" @@ -95,6 +98,13 @@ "tech-ai-github-pr-description.prompt.md": "TechAIPRDescription", "tech-ai-terraform-module.prompt.md": "TechAITerraformModule", } +VALIDATION_WORKFLOW_RELATIVE_PATH = ".github/workflows/github-validate-copilot-customizations.yml" +SOURCE_ONLY_TARGET_RESIDUE_PATHS = ( + ".github/README.md", + ".github/agents/README.md", + ".github/scripts/bootstrap-copilot-config.sh", +) +SOURCE_ONLY_TARGET_RESIDUE_DIRECTORIES = (".github/templates",) def log_info(message: str) -> None: @@ -302,6 +312,44 @@ def utc_now() -> str: return dt.datetime.now(dt.timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z") +def read_source_version(source_root: Path) -> str | None: + version_path = source_root / "VERSION" + if not version_path.is_file(): + return None + + value = version_path.read_text(encoding="utf-8").strip() + return value or None + + +def git_commit_sha(repo_root: Path) -> str | None: + if not (repo_root / ".git").exists(): + return None + + result = subprocess.run( + ["git", "rev-parse", "HEAD"], + cwd=repo_root, + capture_output=True, + text=True, + check=False, + ) + if result.returncode != 0: + return None + + return result.stdout.strip() or None + + +def is_composite_action_file(path: Path) -> bool: + if path.name not in {"action.yml", "action.yaml"}: + return False + + try: + content = path.read_text(encoding="utf-8") + except OSError: + return False + + return bool(re.search(r"(?ms)^runs:\s*$.*?^[ \t]+using:\s*[\"']?composite[\"']?\s*$", content)) + + def parse_args(argv: list[str]) -> argparse.Namespace: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--target", required=True, help="Local repository path to analyze or update.") @@ -565,13 +613,11 @@ def detect_stacks(repo_root: Path, files: list[Path]) -> tuple[list[str], list[s if name.startswith("Dockerfile"): unsupported.add("docker") + if is_composite_action_file(path): + stacks.add("composite-action") if (repo_root / ".github" / "workflows").is_dir(): stacks.add("github-actions") - if any(part == "actions" for path in files for part in path.parts) and any( - path.name in {"action.yml", "action.yaml"} for path in files - ): - stacks.add("composite-action") if (repo_root / "package.json").is_file(): stacks.add("nodejs") if (repo_root / "pyproject.toml").is_file() or any(path.name.startswith("requirements") for path in files): @@ -675,6 +721,10 @@ def detect_git_state(repo_root: Path) -> tuple[bool, list[str]]: return bool(lines), lines +def count_composite_action_files(files: list[Path]) -> int: + return sum(1 for path in files if is_composite_action_file(path)) + + def legacy_alias_paths_for_canonical_path(canonical_relative_path: str) -> list[str]: aliases = set(EXTRA_LEGACY_ALIAS_PATHS.get(canonical_relative_path, ())) path = Path(canonical_relative_path) @@ -1061,7 +1111,11 @@ def build_analysis(source_root: Path, target_root: Path, profiles: dict[str, Rep stacks=stacks, unsupported_stacks=unsupported_stacks, workflow_count=len(list(workflow_dir.glob("*.y*ml"))) if workflow_dir.is_dir() else 0, - composite_action_count=len(list(action_dir.glob("*/action.y*ml"))) if action_dir.is_dir() else 0, + composite_action_count=( + count_composite_action_files(files) + if action_dir.is_dir() or workflow_dir.is_dir() + else 0 + ), focus=detect_focus(target_root.name, stacks, target_root), priority_paths=detect_priority_paths(target_root, stacks), top_extension_counts={key: value for key, value in extension_counts.items() if value}, @@ -1137,6 +1191,8 @@ def select_assets(source_root: Path, analysis: TargetAnalysis, profiles: dict[st prompts.add(".github/prompts/tech-ai-github-action.prompt.md") if "composite-action" in stacks: prompts.add(".github/prompts/tech-ai-github-composite-action.prompt.md") + if repo_needs_data_registry(analysis.repo_root, analysis): + prompts.add(".github/prompts/tech-ai-data-registry.prompt.md") if target_has_pr_template(analysis.repo_root): prompts.add(".github/prompts/tech-ai-github-pr-description.prompt.md") @@ -1223,6 +1279,63 @@ def repo_needs_iam_review(repo_root: Path) -> bool: return False +def repo_needs_data_registry(repo_root: Path, analysis: TargetAnalysis) -> bool: + json_count = analysis.top_extension_counts.get(".json", 0) + if json_count >= 5: + return True + + interesting_dirs = {"authorizations", "organization", "data", "registry", "registries", "resources"} + for path in scan_repo_files(repo_root): + if path.suffix.lower() not in {".json", ".yaml", ".yml"}: + continue + relative_path = path.relative_to(repo_root) + if relative_path.parts and relative_path.parts[0] == ".github": + continue + if any(part in interesting_dirs for part in relative_path.parts): + return True + + return False + + +def repo_has_pytest_tests(repo_root: Path) -> bool: + tests_dir = repo_root / "tests" + if tests_dir.is_dir(): + for pattern in ("test_*.py", "*_test.py"): + if any(tests_dir.rglob(pattern)): + return True + + if (repo_root / "pytest.ini").is_file(): + return True + + pyproject = repo_root / "pyproject.toml" + if pyproject.is_file() and "pytest" in pyproject.read_text(encoding="utf-8"): + return True + + for requirements_file in repo_root.glob("requirements*.txt"): + if "pytest" in requirements_file.read_text(encoding="utf-8"): + return True + + return False + + +def target_has_validation_workflow(repo_root: Path) -> bool: + return (repo_root / VALIDATION_WORKFLOW_RELATIVE_PATH).is_file() + + +def detect_source_only_residues(target_root: Path) -> list[str]: + residues: list[str] = [] + + for relative_path in SOURCE_ONLY_TARGET_RESIDUE_PATHS: + if (target_root / relative_path).exists(): + residues.append(relative_path) + + for relative_dir in SOURCE_ONLY_TARGET_RESIDUE_DIRECTORIES: + if (target_root / relative_dir).exists(): + residues.append(f"{relative_dir}/**") + + return residues + + def build_validation_commands(analysis: TargetAnalysis, instruction_paths: set[str] | list[str]) -> list[str]: commands: list[str] = [] if "terraform" in analysis.stacks: @@ -1230,7 +1343,9 @@ def build_validation_commands(analysis: TargetAnalysis, instruction_paths: set[s if "bash" in analysis.stacks: commands.extend(["bash -n ", "shellcheck -s bash "]) if "python" in analysis.stacks: - commands.extend(["python -m compileall ", "pytest"]) + commands.append("python -m compileall ") + if repo_has_pytest_tests(analysis.repo_root): + commands.append("pytest") commands.append("bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict") return commands @@ -1672,7 +1787,7 @@ def plan_actions(target_root: Path, planned_files: list[PlannedFile], manifest: def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, source_root: Path) -> str: - instructions_apply_to = build_instruction_rules(source_root, selection.instructions) + instructions_apply_to = build_instruction_rule_pairs(source_root, selection.instructions) preferred_prompts = preferred_asset_lines(source_root, selection.preferred_prompts) preferred_skills = preferred_asset_lines(source_root, selection.preferred_skills) inventory_paths = merged_inventory_paths(analysis.repo_root, selection) @@ -1696,12 +1811,12 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, "- Repository-local prompt, skill, and agent `name:` values must also start with `local-`.", "", "## Decision Priority", - "1. Apply repository non-negotiables from `copilot-instructions.md`.", + "1. Apply repository non-negotiables from `.github/copilot-instructions.md`.", "2. Apply explicit user requirements for the current task.", "3. Apply the selected agent behavior (agent-first routing).", - "4. Apply matching files under `instructions/*.instructions.md` using `applyTo`.", - "5. Apply selected prompt constraints from `prompts/*.prompt.md`.", - "6. Apply implementation details from referenced `skills/*/SKILL.md`.", + "4. Apply matching files under `.github/instructions/*.instructions.md` using `applyTo`.", + "5. Apply selected prompt constraints from `.github/prompts/*.prompt.md`.", + "6. Apply implementation details from referenced `.github/skills/*/SKILL.md`.", "7. If no agent is explicitly selected, default to `TechAIImplementer`.", "", "## Agent Routing", @@ -1724,10 +1839,7 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, [ "", "## Prohibitions", - "- Never hardcode secrets, tokens, or credentials.", - "- Never modify `README.md` files unless explicitly requested by the user.", - "- Never introduce new patterns when existing repository conventions exist.", - "- Keep all repository artifacts in English (user chat may be in other languages).", + "- Apply all non-negotiables from `.github/copilot-instructions.md` plus:", "- Never run destructive commands unless explicitly requested.", "- Never skip validation after making changes.", "", @@ -1740,8 +1852,8 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, ] ) lines.extend(f" - `{path}`" for path in analysis.priority_paths) - lines.extend(["", "### Default instruction routing"]) - lines.extend(f"- `{rule}`" for rule in instructions_apply_to) + lines.extend(["", "### Default instruction routing", "| Pattern | Instruction |", "| --- | --- |"]) + lines.extend(f"| `{pattern}` | `{label}` |" for pattern, label in instructions_apply_to) lines.extend(["", "### Preferred prompts"]) lines.extend(preferred_prompts) lines.extend(["", "### Preferred skills"]) @@ -1774,14 +1886,14 @@ def strip_github_prefix(value: str) -> str: return value -def build_instruction_rules(source_root: Path, instruction_paths: list[str]) -> list[str]: - rules: list[str] = [] +def build_instruction_rule_pairs(source_root: Path, instruction_paths: list[str]) -> list[tuple[str, str]]: + rules: list[tuple[str, str]] = [] for instruction_path in instruction_paths: apply_to = frontmatter_value(source_root / instruction_path, "applyTo") if not apply_to: continue label = Path(strip_github_prefix(instruction_path)).name - rules.append(f"{apply_to} -> `{label}`") + rules.append((apply_to, label)) return rules @@ -1789,8 +1901,7 @@ def preferred_asset_lines(source_root: Path, relative_paths: list[str]) -> list[ lines: list[str] = [] for relative_path in relative_paths: name = asset_display_name(source_root, relative_path) - description = frontmatter_value(source_root / relative_path, "description") or "No description available." - lines.append(f"- `{name}`: {description}") + lines.append(f"- `{name}`") return lines @@ -1900,6 +2011,19 @@ def build_recommendations( f"`name:` values: {', '.join(local_naming_issue_paths)}." ) + if not target_has_validation_workflow(analysis.repo_root): + recommendations["missing consumer-facing validation or onboarding guidance"].append( + "The target repository is missing `.github/workflows/github-validate-copilot-customizations.yml`; " + "add it manually if the consumer wants Copilot customization CI enforcement." + ) + + source_only_residues = detect_source_only_residues(analysis.repo_root) + if source_only_residues: + recommendations["missing consumer-facing validation or onboarding guidance"].append( + "The target repository still contains source-only bootstrap residues that should not be treated as " + f"consumer baseline assets: {', '.join(source_only_residues)}." + ) + if any(action.target_relative_path == analysis.agents_relative_path and action.status == "conflict" for action in actions): recommendations["weak conflict-handling rules"].append( "Consider generated section markers or a dedicated root-AGENTS template to reduce consumer AGENTS " @@ -1941,7 +2065,7 @@ def write_report(path: Path, content: str) -> None: path.write_text(content, encoding="utf-8") -def apply_plan(target_root: Path, plan: SyncPlan, planned_files: list[PlannedFile]) -> None: +def apply_plan(target_root: Path, plan: SyncPlan, planned_files: list[PlannedFile], source_root: Path) -> None: content_map = {item.target_relative_path: item for item in planned_files} for action in plan.actions: if action.status not in {"create", "update"}: @@ -1958,6 +2082,8 @@ def apply_plan(target_root: Path, plan: SyncPlan, planned_files: list[PlannedFil "generated_at_utc": utc_now(), "target_repo": str(target_root), "profile": plan.selection.profile.name, + "source_version": read_source_version(source_root), + "source_commit": git_commit_sha(source_root), "managed_files": {}, } @@ -2054,7 +2180,7 @@ def render_markdown_report(plan: SyncPlan) -> str: for command in plan.selection.validation_commands: lines.append(f"- `{command}`") - lines.extend(["", "## Target-driven recommendations for improving the source repo"]) + lines.extend(["", "## Target-driven recommendations"]) for category, items in plan.recommendations.items(): lines.append(f"### {category.title()}") for item in items: @@ -2281,7 +2407,7 @@ def main(argv: list[str] | None = None) -> int: if args.mode == "apply": log_info("Applying conservative merge for source-managed files.") - apply_plan(target_root, plan, planned_files) + apply_plan(target_root, plan, planned_files, source_root) log_success(f"Manifest written to {target_root / MANIFEST_RELATIVE_PATH}") else: log_info("Plan mode selected - no repository files will be changed.") diff --git a/.github/scripts/validate-copilot-customizations.sh b/.github/scripts/validate-copilot-customizations.sh index ce2e429..72b55d8 100755 --- a/.github/scripts/validate-copilot-customizations.sh +++ b/.github/scripts/validate-copilot-customizations.sh @@ -251,6 +251,23 @@ frontmatter_value() { ' } +validate_frontmatter_structure() { + local file="$1" + local severity="$2" + + if [[ "$(head -n 1 "$file" 2>/dev/null)" != "---" ]]; then + record_issue "$severity" "File is missing opening frontmatter fence: ${file}" + return 1 + fi + + if [[ "$(grep -c '^---$' "$file")" -lt 2 ]]; then + record_issue "$severity" "File has malformed frontmatter fence: ${file}" + return 1 + fi + + return 0 +} + check_required_keys() { local file="$1" local severity="$2" @@ -568,6 +585,8 @@ validate_prompt_file() { section_severity="warn" fi + validate_frontmatter_structure "$file" error || true + if frontmatter "$file" | grep -Eq '^mode:[[:space:]]*'; then severity="error" [[ "$MODE" == "legacy-compatible" ]] && severity="warn" @@ -624,6 +643,8 @@ validate_prompt_file() { validate_instruction_file() { local file="$1" + validate_frontmatter_structure "$file" error || true + if [[ "$MODE" == "strict" ]]; then check_required_keys "$file" error applyTo description else @@ -642,6 +663,7 @@ validate_skill_file() { [[ "$MODE" == "legacy-compatible" ]] && section_severity="warn" + validate_frontmatter_structure "$file" error || true check_required_keys "$file" error name description validate_repo_local_skill_naming "$file" @@ -673,6 +695,7 @@ validate_agents_dir() { while IFS= read -r file; do count=$((count + 1)) + validate_frontmatter_structure "$file" error || true check_required_keys "$file" error name description tools validate_repo_local_agent_naming "$file" diff --git a/.github/security-baseline.md b/.github/security-baseline.md index 9975416..a8b02a4 100644 --- a/.github/security-baseline.md +++ b/.github/security-baseline.md @@ -27,6 +27,20 @@ Provide a portable baseline that teams can apply before enabling repository-wide - Use a deprecation window before removing prompts/skills in active use. - Keep a rollback path for workflow and policy changes. +## Enforcement status +| Control | Status | Tool | +| --- | --- | --- | +| Third-party action SHA pinning | Automated | `validate-copilot-customizations.sh` | +| Minimal workflow permissions | Automated | `validate-copilot-customizations.sh` | +| Validate `.github/**` in CI | Automated | `github-validate-copilot-customizations.yml` | +| `shellcheck` on `.github/scripts/` | Automated | pre-commit + CI | +| Secret placeholder avoidance in prompts/examples | Partial | pre-commit hooks + review | +| OIDC over long-lived secrets | Manual review | `github-actions.instructions.md` | +| Branch protection for `.github/**` | Manual review | repository settings | +| Read-only agents as default | Manual review | agent review | +| Scoped write-capable agents | Manual review | agent review | +| CHANGELOG-based change governance | Manual review | PR review | + ## Optional hardening - Add CODEOWNERS coverage for `.github/**`. - Add secret scanning and dependency update automation. diff --git a/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md b/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md new file mode 100644 index 0000000..98dc3cb --- /dev/null +++ b/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md @@ -0,0 +1,51 @@ +--- +name: TechAILocalCopilotCustomizationBuilder +description: Create or update repository-owned local GitHub Copilot customization assets in a consumer repository while preserving the shared synced baseline. +--- + +# TechAI Local Copilot Customization Builder Skill + +## When to use +- Create or update repository-owned `local-*` prompts, skills, agents, or `AGENTS.md` wiring in a consumer repo. +- Extend a consumer repo with Copilot behavior that should stay local instead of entering the shared `tech-ai-*` baseline. +- Clean up or normalize existing target-local Copilot assets so they follow current naming, frontmatter, and inventory rules. + +## Workflow +1. Inspect the target repository layout, `.github` contents, root `AGENTS.md`, git state, and existing local Copilot assets. +2. Confirm the baseline is current enough for local customization work: + - if `copilot-instructions.md`, the validator script, or expected synced assets are missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first; + - use the sync report to avoid creating a `local-*` asset that duplicates an available shared baseline capability. +3. Decide the narrowest asset type that solves the request: + - create or update a `local-*.prompt.md` when the behavior is mostly task instructions; + - add a `local-*` skill only when the workflow needs reusable implementation detail beyond the prompt; + - add a `local-*` agent only when the repo needs durable routing or persona guidance that cannot stay in a prompt or skill. +4. Enforce local naming and ownership rules: + - filenames for repo-owned prompts, skills, and agents must start with `local-`; + - prompt, skill, and agent frontmatter `name:` values must also start with `local-`; + - keep local assets in `.github/prompts`, `.github/skills//SKILL.md`, and `.github/agents`. +5. Keep local assets minimal and compatible with the shared baseline: + - reuse `.github/instructions/*.instructions.md` and synced `tech-ai-*` skills when possible instead of copying large guidance blocks; + - reference shared prompts or skills by path when they already cover most of the behavior; + - avoid creating a local canonical duplicate of an existing `tech-ai-*` capability. +6. Update `AGENTS.md` in the target repository: + - keep explicit `.github/...` paths; + - add the local assets to the inventory; + - adjust routing or preferred prompts or skills only when the new local capability should be discoverable by default; + - avoid duplicating long prompt or skill descriptions inside `AGENTS.md`. +7. Validate the target repository after changes: + - run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict`; + - run stack-specific checks for touched Bash, Python, JSON, YAML, Markdown, or Terraform assets; + - if the repo has a synced baseline manifest, preserve it and do not rewrite managed files opportunistically. +8. Report the result with changed files, validation output, residual repo-local risks, and whether the new capability should stay local or be proposed for promotion into the shared source baseline. + +## Scope rules +- Manage consumer-repository Copilot assets only. +- Keep source-repository assets and shared baseline definitions unchanged unless promotion is explicitly requested. +- Prefer one local capability per repo-specific workflow; consolidate or deprecate duplicates instead of multiplying near-identical local prompts. +- Do not create local copies of source-only repo agents such as `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`, or `TechAISyncCopilotConfigs`. + +## Validation +- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` in the target repo after local customization changes. +- Run `bash -n` and `shellcheck -s bash` for changed Bash files when available. +- Run `python -m compileall ` and relevant `pytest` checks for changed Python files. +- Re-run `TechAISyncCopilotConfigs` in `plan` mode when you need to confirm that the new local assets remain clearly separated from the managed shared baseline. diff --git a/.github/templates/copilot-quickstart.md b/.github/templates/copilot-quickstart.md index 3b39a8f..54b2c8f 100644 --- a/.github/templates/copilot-quickstart.md +++ b/.github/templates/copilot-quickstart.md @@ -16,14 +16,14 @@ For detailed maintenance and validation flow, refer to `.github/README.md`. 5. Run `.github/scripts/validate-copilot-customizations.sh --scope root --mode strict`. ## Suggested starter sets -- Java repositories: `java.instructions.md`, `tech-ai-java.prompt.md`, `tech-ai-project-java/SKILL.md` -- Node.js repositories: `nodejs.instructions.md`, `tech-ai-nodejs.prompt.md`, `tech-ai-project-nodejs/SKILL.md` -- CI-focused repositories: `github-actions.instructions.md`, `tech-ai-github-action.prompt.md`, `tech-ai-cicd-workflow/SKILL.md` +- Java repositories: `java.instructions.md`, `tech-ai-java.prompt.md`, `tech-ai-project-java/SKILL.md`, plus core agents `TechAIPlanner`, `TechAIImplementer`, `TechAIReviewer` +- Node.js repositories: `nodejs.instructions.md`, `tech-ai-nodejs.prompt.md`, `tech-ai-project-nodejs/SKILL.md`, plus core agents `TechAIPlanner`, `TechAIImplementer`, `TechAIReviewer` +- CI-focused repositories: `github-actions.instructions.md`, `tech-ai-github-action.prompt.md`, `tech-ai-cicd-workflow/SKILL.md`, plus `TechAIWorkflowSupplyChain` ## Validation gate Add `.github/workflows/github-validate-copilot-customizations.yml` to enforce consistency in pull requests. ## Alignment strategy -- Use `.github/scripts/bootstrap-copilot-config.sh --target ` in dry-run first for the initial copy. - Use `python .github/scripts/tech-ai-sync-copilot-configs.py --target --mode plan` for conservative alignment and minimum-asset selection. +- Use `.github/scripts/bootstrap-copilot-config.sh --target ` only as a legacy fallback when a consumer cannot adopt manifest-based sync yet. - Prefer canonical `tech-ai-*` script prompts in consumer repositories to reduce prompt duplication and token footprint. diff --git a/AGENTS.md b/AGENTS.md index 68f71a6..ba2cf8b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,8 +37,9 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Use `TechAIGlobalCustomizationAuditor` as the final quality gate for GitHub Copilot customization changes in this repository. - Use `TechAICustomizationAuditor` only as a deprecated compatibility alias while older references are migrated. - Use `TechAISyncCopilotConfigs` for cross-repository Copilot-core alignment and source or target redundancy audits. +- Use `TechAILocalCopilotCustomizationBuilder` when a consumer repository needs repo-owned `local-*` prompts, skills, agents, or `AGENTS.md` wiring that should remain local instead of entering the shared baseline. - Use specialist agents (`TechAIWorkflowSupplyChain`, `TechAISecurityReviewer`, `TechAITerraformGuardrails`, `TechAIIAMLeastPrivilege`, `TechAIPRWriter`) only when their domain matches the task. -- The `TechAIGlobalCustomizationBuilder` and `TechAIGlobalCustomizationAuditor` agents are repo-only and must not be synced to consumer repositories. +- The `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`, and `TechAILocalCopilotCustomizationBuilder` agents are repo-only and must not be synced to consumer repositories. ### Anti-patterns @@ -49,12 +50,15 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Do not use generic `TechAIReviewer` when you need exhaustive per-language nit-level review; use `TechAIScriptReviewer` instead. - Do not use `TechAICustomizationAuditor` for new work; use `TechAIGlobalCustomizationAuditor`. - Do not use `TechAIImplementer` alone when the task is cross-repository Copilot configuration alignment; use `TechAISyncCopilotConfigs`. +- Do not use `TechAISyncCopilotConfigs` alone when the task is to author new repository-owned `local-*` assets in a consumer repository; use `TechAILocalCopilotCustomizationBuilder` after baseline alignment. +- Do not use `TechAILocalCopilotCustomizationBuilder` to add new shared `tech-ai-*` assets in this standards repository; use `TechAIGlobalCustomizationBuilder`. ### Composition and Handoffs - For changes spanning multiple specialist domains, run each relevant specialist and aggregate findings. - The standard chain for non-trivial work is `TechAIPlanner` -> `TechAIImplementer` -> `TechAIReviewer` or a matching specialist. - For GitHub Copilot customization changes in this repository, use `TechAIGlobalCustomizationBuilder` first and `TechAIGlobalCustomizationAuditor` before final handoff. +- For consumer-local Copilot customization work, use `TechAISyncCopilotConfigs` first if the target baseline is unknown, then use `TechAILocalCopilotCustomizationBuilder` for repo-owned `local-*` assets. - `TechAIPlanner` output is input context for `TechAIImplementer`. - `TechAIImplementer` output is input context for `TechAIReviewer`. - `TechAIReviewer` findings flagged as `Critical` or `Major` route back to `TechAIImplementer` for remediation. @@ -127,6 +131,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `TechAICodeReview`: exhaustive, nit-level code review. - `TechAIGitHubAction`: GitHub Actions workflow authoring. +- `TechAILocalCopilotCustomizationBuilder`: consumer-local `local-*` customization authoring. - `TechAISyncCopilotConfigs`: cross-repository alignment and redundancy analysis. - `TechAIPRDescription`: pull request body generation. - `TechAIAddUnitTests`: test authoring and improvement. @@ -136,6 +141,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `TechAICodeReview`: strict review workflow and anti-pattern catalog. - `TechAICICDWorkflow`: CI or CD workflow design patterns. +- `TechAILocalCopilotCustomizationBuilder`: consumer-local Copilot customization workflow. - `TechAISyncCopilotConfigs`: deterministic sync planning and reporting. - `TechAIPRWriting`: PR writing conventions aligned to the repository template. - `TechAICloudPolicy`: reusable cloud policy authoring patterns. @@ -180,6 +186,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/prompts/tech-ai-github-composite-action.prompt.md` - `.github/prompts/tech-ai-github-pr-description.prompt.md` - `.github/prompts/tech-ai-java.prompt.md` +- `.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md` - `.github/prompts/tech-ai-nodejs.prompt.md` - `.github/prompts/tech-ai-python-script.prompt.md` - `.github/prompts/tech-ai-python.prompt.md` @@ -194,6 +201,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/skills/tech-ai-code-review/SKILL.md` - `.github/skills/tech-ai-composite-action/SKILL.md` - `.github/skills/tech-ai-data-registry/SKILL.md` +- `.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md` - `.github/skills/tech-ai-pr-writing/SKILL.md` - `.github/skills/tech-ai-project-java/SKILL.md` - `.github/skills/tech-ai-project-nodejs/SKILL.md` @@ -211,6 +219,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/agents/tech-ai-github-workflow-supply-chain.agent.md` - `.github/agents/tech-ai-global-customization-auditor.agent.md` - `.github/agents/tech-ai-global-customization-builder.agent.md` +- `.github/agents/tech-ai-local-copilot-customization-builder.agent.md` - `.github/agents/tech-ai-iam-least-privilege.agent.md` - `.github/agents/tech-ai-implementer.agent.md` - `.github/agents/tech-ai-planner.agent.md` diff --git a/ANALYSIS_REPORT.md b/ANALYSIS_REPORT.md index 4144a9d..540bc1a 100644 --- a/ANALYSIS_REPORT.md +++ b/ANALYSIS_REPORT.md @@ -1,5 +1,7 @@ # Cloud Strategy GitHub — Comprehensive Analysis Report +> **STATUS**: SUPERSEDED by `COPILOT_REVIEW.md` dated 2026-03-09. Some findings below are already resolved and should be treated as historical context, not the current source of truth. + > **Generated**: 2025-07-17 > **Scope**: Full repository audit of `cloud-strategy.github` > **Purpose**: Identify issues, improvement opportunities, and unconsidered areas in the central Copilot customization standards repository. diff --git a/CODEOWNERS b/CODEOWNERS index dc7f5ce..6a1775b 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -1,2 +1,3 @@ # Copilot customization owners. * @pagopa/engineering-cloud @GNuccio96 +/.github/ @pagopa/engineering-cloud @GNuccio96 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..db7b6aa --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,33 @@ +# Contributing + +## Scope +This repository is the source baseline for reusable GitHub Copilot customization assets synced into consumer repositories. + +## Naming conventions +- Canonical source-owned instructions, prompts, skills, and agents use the `tech-ai-` filename prefix. +- Canonical prompt, skill, and agent `name:` values use the `TechAI` prefix. +- Repo-only standards agents use the `TechAIGlobal` prefix and must stay source-only. +- Consumer-local assets use the `local-` prefix in both filenames and `name:` values. + +## Adding or updating assets +- Instructions: keep frontmatter `description` and `applyTo`, use repository-agnostic wording, and avoid stack duplication already covered by a skill. +- Prompts: require `name`, `description`, `agent`, and `argument-hint`, plus `## Instructions`, `## Validation`, and `## Minimal example`. +- Skills: keep `name`, `description`, `## When to use`, and one validation/testing section. +- Agents: keep `name`, `description`, `tools`, `## Objective`, and `## Restrictions`. +- Sync or validator behavior changes must include test coverage under `tests/`. + +## Validation before PR +- `make lint` +- `make validate` +- `make test` +- Run any additional stack-specific validation relevant to the touched assets. + +## Review flow +- Author customization changes with `TechAIGlobalCustomizationBuilder`. +- Review them with `TechAIGlobalCustomizationAuditor`. +- Record notable lifecycle or behavior changes in `.github/CHANGELOG.md`. + +## Release and sync metadata +- Update `VERSION` when publishing a standards release with consumer-facing impact. +- Create a git tag for released versions using the `vMAJOR.MINOR.PATCH` format. +- The sync manifest records `source_version` and `source_commit`; keep them accurate by releasing from clean commits. diff --git a/COPILOT_REVIEW.md b/COPILOT_REVIEW.md new file mode 100644 index 0000000..d936f12 --- /dev/null +++ b/COPILOT_REVIEW.md @@ -0,0 +1,633 @@ +# Copilot Configuration Review — cloud-strategy.github + +> **Generated**: 2026-03-09 +> **Scope**: Full audit of GitHub Copilot customization assets in `cloud-strategy.github` (global standards repository) +> **Purpose**: Actionable findings for Codex to fix. Each item includes the exact file, the problem, and the concrete fix required. +> **Note**: This review supersedes findings in `ANALYSIS_REPORT.md` (dated 2025-07-17). Items already resolved since that report are not repeated here. + +--- + +## Table of Contents + +- [Executive Summary](#executive-summary) +- [Critical Findings](#critical-findings) +- [Major Findings](#major-findings) +- [Minor Findings](#minor-findings) +- [Nit Findings](#nit-findings) +- [Architecture & Token Optimization](#architecture--token-optimization) +- [Missing Assets & Gaps](#missing-assets--gaps) +- [Action Checklist](#action-checklist) + +--- + +## Executive Summary + +The `cloud-strategy.github` repository is an impressive and well-thought-out framework for managing GitHub Copilot customization at scale. Since the original ANALYSIS_REPORT (July 2025), significant improvements have been made: prompt duplication has been consolidated to canonical `tech-ai-*` names, the sync script has grown to 2300+ lines with 20 tests, repo-only global agents have been properly isolated, and the validator is robust at ~1200 lines. The framework is clearly production-grade. + +However, this review identifies areas where Codex effectiveness can be further improved: **token budget waste from redundant content**, **missing infrastructure for key consumer stacks**, **test coverage gaps for the validator bash script**, and **governance enforcement blind spots** that reduce the framework's reliability at scale. + +| Severity | Count | +|----------|-------| +| Critical | 1 | +| Major | 9 | +| Minor | 10 | +| Nit | 7 | + +--- + +## Critical Findings + +### C-01: `bootstrap-copilot-config.sh` is still active without deprecation — overlaps with sync script + +**File**: `.github/scripts/bootstrap-copilot-config.sh` + +The rsync-based bootstrap script and the manifest-based sync script (`tech-ai-sync-copilot-configs.py`) serve overlapping purposes. The sync script is far superior (SHA tracking, conflict detection, profile-aware selection, JSON reports, conservative merge). The bootstrap script does destructive `--clean` syncs with no manifest tracking, no conflict detection, and no reporting. This creates confusion for Codex about which tool to use and risks destructive operations on consumer repos. + +The ANALYSIS_REPORT (item 2.4) flagged this in July 2025 but it remains unresolved. + +**Fix**: +1. Add a deprecation notice to the header of `bootstrap-copilot-config.sh`: +```bash +# ⚠️ DEPRECATED: Prefer tech-ai-sync-copilot-configs.py for all consumer alignment. +# This script is maintained for backward compatibility only. +# See .github/DEPRECATION.md for lifecycle policy. +``` +2. Record the deprecation in `.github/CHANGELOG.md`. +3. Update `copilot-quickstart.md` to recommend the sync script as the primary tool and the bootstrap script only as a legacy fallback. +4. Add the deprecation to `DEPRECATION.md` under "Current deprecations": +```markdown +## Current deprecations +- `scripts/bootstrap-copilot-config.sh`: Deprecated in favor of `scripts/tech-ai-sync-copilot-configs.py`. Removal planned after all consumers migrate to sync-based alignment. +``` + +--- + +## Major Findings + +### M-01: No versioning or release strategy — consumers cannot pin to a known-good state + +**Files**: Repository root (missing `VERSION` file), `.github/CHANGELOG.md` (dates but no tags) + +The ANALYSIS_REPORT (item 2.1) flagged this as Major. It is still unresolved. Consumer repos have no way to pin to a specific standards version. If a breaking change is pushed to `main`, all consumers are immediately affected. + +**Fix**: +1. Create a `VERSION` file at the root with initial content: `1.0.0` +2. Add git tags at meaningful milestones (e.g., `v1.0.0` for the current stable state). +3. Update `tech-ai-sync-copilot-configs.py` to include the source version in the manifest JSON. +4. Document the release process in a new `RELEASING.md` or in `CONTRIBUTING.md`. + +--- + +### M-02: No `CONTRIBUTING.md` — contributors have no documented process + +**File**: Missing — `CONTRIBUTING.md` + +This is a standards repository that other teams consume. Without contribution guidelines, team members (and Codex) have no reference for how to add new instructions, prompts, skills, or agents correctly. + +**Fix**: Create `CONTRIBUTING.md` with sections covering: +- How to add a new instruction file (naming, frontmatter, `applyTo`) +- How to add a new prompt (naming, frontmatter keys, skill reference) +- How to add a new skill (directory structure, SKILL.md template) +- How to add a new agent (naming, tools, restrictions) +- Naming conventions (`tech-ai-*` for canonical, `local-*` for consumer-local, `TechAIGlobal*` for repo-only) +- Required validation before PR (`validate-copilot-customizations.sh`, `pytest`, `shellcheck`) +- Review process (use `TechAIGlobalCustomizationBuilder` → `TechAIGlobalCustomizationAuditor`) + +--- + +### M-03: No `Makefile` for developer workflow + +**File**: Missing — `Makefile` + +The ANALYSIS_REPORT (item 2.6) flagged this. It remains unresolved. Without a Makefile, developers and Codex must remember individual commands for linting, testing, and validation. + +**Fix**: Create `Makefile`: +```make +.PHONY: help lint validate test fmt all + +help: ## Show available targets + @grep -E '^[a-zA-Z_-]+:.*##' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*##"}; {printf " %-20s %s\n", $$1, $$2}' + +lint: ## Run shellcheck and bash syntax checks + bash -n .github/scripts/*.sh + shellcheck -s bash .github/scripts/*.sh + python3 -m compileall .github/scripts tests + +validate: ## Run Copilot customization validator + bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict + +test: ## Run Python test suite + pytest -q + +fmt: ## Format Terraform files + terraform fmt -recursive + +all: lint validate test ## Run all checks +``` + +--- + +### M-04: Validator bash script has zero dedicated tests + +**File**: `.github/scripts/validate-copilot-customizations.sh` (1184 lines) + +The test file `tests/test_tech_ai_validate_copilot_customizations.py` has 7 tests, which is good but covers only basic happy path + a few semantic checks. The 1184-line validator itself has many code paths not covered: +- `--scope all` and `--scope repo=` modes +- `--mode legacy-compatible` vs `--mode strict` behavioral differences +- JSON report output format and structure +- Error handling for malformed frontmatter +- Edge cases: empty instruction files, missing `applyTo`, overlapping agent names +- Correct SHA pinning detection in workflows + +**Fix**: +1. Expand `tests/test_tech_ai_validate_copilot_customizations.py` with: + - Test for `--scope all` with multiple sub-repos + - Test for `--mode legacy-compatible` accepting relaxed conventions + - Test for JSON report output correctness + - Test for malformed frontmatter detection + - Test for SHA pinning detection +2. Consider adding `bats-core` tests for direct bash testing of the validator's internal functions. +3. Target ~15-20 total validator tests. + +--- + +### M-05: `dependabot.yml` still contains unused package ecosystems + +**File**: `.github/dependabot.yml` + +The ANALYSIS_REPORT (item 2.3) flagged this. Check if `npm`, `maven`, `gradle` ecosystems are still present. This repository only contains Bash, Python, and pre-commit hooks. Unused ecosystems waste CI minutes. + +**Fix**: Keep only the ecosystems relevant to this repo: `pip` (for pytest), `github-actions`, and optionally `terraform` (for pre-commit pin). Remove `npm`, `maven`, `gradle` if still present. If these ecosystems are intentionally kept as a template reference, move them to `templates/dependabot.template.yml` and document the intent. + +--- + +### M-06: No `docker.instructions.md` despite many consumer repos using Docker + +**File**: Missing — `.github/instructions/docker.instructions.md` + +The `AGENTS.md` has a backlog trigger: "Add `instructions/docker.instructions.md` when the first Dockerfile is introduced in this repository." But consumer repos likely already use Docker, and this standards repo should proactively provide Docker guidance regardless of whether *this* repo has Dockerfiles. + +**Fix**: Create `.github/instructions/docker.instructions.md`: +```instructions +--- +description: Docker and container build standards for secure, efficient, and reproducible images. +applyTo: "**/Dockerfile,**/Dockerfile.*,**/.dockerignore,**/docker-compose*.yml" +--- + +# Docker Instructions + +## Image build standards +- Use multi-stage builds to minimize image size. +- Run as non-root user. +- Use explicit base image tags with digests when possible. +- Keep `.dockerignore` up to date. +- Order layers for optimal cache usage (dependencies before source). + +## Security +- No secrets in build args, ENV, or COPY. +- Scan images for vulnerabilities in CI. +- Minimize installed packages. + +## Validation +- Build and test locally before pushing. +- Use health checks in orchestrated environments. +``` + +Also create a corresponding `prompts/tech-ai-docker.prompt.md` and `skills/tech-ai-docker/SKILL.md`. + +--- + +### M-07: `copilot-quickstart.md` still references bootstrap script as primary tool + +**File**: `.github/templates/copilot-quickstart.md` + +The quickstart guide's "Alignment strategy" section recommends `bootstrap-copilot-config.sh` first and `tech-ai-sync-copilot-configs.py` second. Given C-01 (bootstrap deprecation), the order should be reversed and the bootstrap should be mentioned only as a legacy option. + +**Fix**: In the "Alignment strategy" section, change: +```markdown +## Alignment strategy +- Use `python .github/scripts/tech-ai-sync-copilot-configs.py --target --mode plan` for conservative alignment and minimum-asset selection (recommended). +- Use `.github/scripts/bootstrap-copilot-config.sh --target ` only as a legacy quick-copy fallback. +- Prefer canonical `tech-ai-*` script prompts in consumer repositories. +``` + +--- + +### M-08: `ANALYSIS_REPORT.md` is stale and some findings are already resolved + +**File**: `ANALYSIS_REPORT.md` + +The report is dated 2025-07-17 and references issues that have been fixed (e.g., prompt duplication consolidated, code-review instructions refactored, tests expanded from 5 to 27). Keeping the stale report creates confusion for Codex, which may attempt to fix already-resolved issues. + +**Fix**: Either: +1. Delete `ANALYSIS_REPORT.md` and replace with this `COPILOT_REVIEW.md` as the current audit. +2. Or add a clear header to `ANALYSIS_REPORT.md`: +```markdown +> **STATUS**: SUPERSEDED by `COPILOT_REVIEW.md` (2026-03-09). Many findings below have been resolved. +``` + +--- + +### M-09: Prompt `${input:...}` variable names are not standardized + +**Files**: All prompt files under `.github/prompts/` + +The ANALYSIS_REPORT (item 4.4) flagged inconsistent input variable naming. Prompts use various names for similar concepts: `target_file` vs `file_path` vs `target_path`, `description` vs `purpose` vs `feature_description`. + +**Fix**: +1. Define a canonical variable name catalog (add to `copilot-instructions.md` or a new `prompts/README.md`): + - `target_path`: the file or directory being acted upon + - `purpose`: what the deliverable should accomplish + - `language`: target programming language + - `test_framework`: testing framework to use + - `target_repo`: repository path for cross-repo operations +2. Audit all prompts and normalize variable names. +3. Add a validator check for canonical variable names. + +--- + +## Minor Findings + +### m-01: `agents/README.md` should reference repo-only agent exclusion + +**File**: `.github/agents/README.md` + +The agents README lists routing for all agents including repo-only ones (`TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`), but does not explicitly mark them as non-syncable. This information is in `AGENTS.md` but should also be in the agents README for clarity. + +**Fix**: Add a note to the README: +```markdown +## Repo-only agents (not synced to consumers) +- `TechAIGlobalCustomizationBuilder` +- `TechAIGlobalCustomizationAuditor` +- `TechAIScriptReviewer` +- `TechAISyncCopilotConfigs` +- `TechAICustomizationAuditor` (deprecated alias) +``` + +--- + +### m-02: Pre-commit config missing `shellcheck` hook + +**File**: `.pre-commit-config.yaml` + +The ANALYSIS_REPORT (item 5.2) flagged this. The CI installs shellcheck but developers don't get local feedback. + +**Fix**: Add to `.pre-commit-config.yaml`: +```yaml +- repo: https://github.com/shellcheck-py/shellcheck-py + rev: v0.10.0.1 + hooks: + - id: shellcheck + args: ["-s", "bash"] +``` + +--- + +### m-03: `tech-ai-requirements-dev.txt` pins only `pytest` — add type checking + +**File**: `.github/tech-ai-requirements-dev.txt` + +Currently only `pytest==8.3.3`. The 2300-line sync script and 1200-line validator would benefit from type-checking support. + +**Fix**: Add `mypy` or at minimum `pyright`/`pylint` for static analysis: +``` +pytest==8.3.3 +mypy==1.14.1 +``` +Also consider adding `ruff` for linting Python scripts. + +--- + +### m-04: No architecture diagram for the customization framework + +**File**: Missing + +The ANALYSIS_REPORT (item 7.2) recommended a Mermaid diagram. The framework's hierarchy (instructions → prompts → skills → agents) plus the sync/validate pipeline deserves visual documentation. + +**Fix**: Add a Mermaid diagram to `.github/README.md`: +```markdown +## Architecture + +```mermaid +graph TD + A[copilot-instructions.md] --> B[instructions/*.instructions.md] + B --> C[prompts/*.prompt.md] + C --> D[skills/*/SKILL.md] + D --> E[agents/*.agent.md] + + F[AGENTS.md] --> A + F --> E + + G[validate-copilot-customizations.sh] --> A + G --> B + G --> C + G --> D + G --> E + + H[tech-ai-sync-copilot-configs.py] -->|plan/apply| I[Consumer Repos] + I --> J[Consumer AGENTS.md] +``` +``` + +--- + +### m-05: `security-baseline.md` controls lack enforcement coverage tracking + +**File**: `.github/security-baseline.md` + +The ANALYSIS_REPORT (item 5.1) flagged partial enforcement. The security baseline lists 11 controls but only ~4 are automated: + +| Control | Automated? | +|---------|-----------| +| SHA pinning | Yes (validator) | +| Minimal permissions | No | +| OIDC over secrets | No | +| Branch protection | No | +| Validate `.github/**` in CI | Yes (workflow) | +| shellcheck on scripts | Yes (CI) | +| No embedded secrets in prompts | Partial (pre-commit) | +| Deterministic prompt output | No | +| Read-only agents default | No | +| Scoped write agents | No | +| Change governance via CHANGELOG | No | + +**Fix**: Add an "Enforcement status" section to `security-baseline.md`: +```markdown +## Enforcement status +| Control | Status | Tool | +|---------|--------|------| +| SHA pinning | Automated | `validate-copilot-customizations.sh` | +| Minimal permissions | Manual review | — | +| OIDC over secrets | Manual review | — | +| ... +``` + +--- + +### m-06: `AGENTS.md` "Preferred prompts" and "Preferred skills" have token-wasteful descriptions + +**File**: `AGENTS.md`, "Repository Defaults" section + +Each preferred prompt/skill includes a one-line description. These descriptions are already available in each prompt/skill's frontmatter `description:` field. Loading them here wastes tokens. + +**Fix**: Reduce to just the name, and let Codex resolve the description from the frontmatter: +```markdown +### Preferred prompts +- `TechAICodeReview` +- `TechAIGitHubAction` +- `TechAISyncCopilotConfigs` +- `TechAIPRDescription` +- `TechAIAddUnitTests` +- `TechAITerraform` +``` + +Or keep descriptions only for prompts where the name is ambiguous. + +--- + +### m-07: `CODEOWNERS` scope is narrow — bus factor of 2 + +**File**: `CODEOWNERS` + +Only `@pagopa/engineering-cloud @GNuccio96` own all files. For a repository that impacts all consumer repos, this is a single-point-of-failure risk. + +**Fix**: Consider adding a dedicated team like `@pagopa/copilot-standards` with at least 3-4 members. At minimum, add a backup reviewer. + +--- + +### m-08: No secret scanning tool in CI + +**File**: `.github/workflows/github-validate-copilot-customizations.yml` + +Pre-commit has `detect-private-key` but there's no CI-level secret scanning (e.g., `gitleaks`, `trufflehog`). + +**Fix**: Add a gitleaks step to the CI workflow: +```yaml +- name: Run gitleaks + uses: gitleaks/gitleaks-action@ # + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} +``` + +--- + +### m-09: `tech-ai-sync-copilot-configs.py` — `PROMPT_NAME_OVERRIDES` is a maintenance burden + +**File**: `.github/scripts/tech-ai-sync-copilot-configs.py`, `PROMPT_NAME_OVERRIDES` dict + +This dict maps 6 prompt filenames to canonical `TechAI*` name values. Every new prompt with a non-obvious name mapping needs a manual entry. This is fragile. + +**Fix**: Consider deriving the canonical name from the filename automatically using a naming convention function, and only using overrides for genuinely irregular cases. Or add a comment explaining the naming derivation rule and when an override is needed. + +--- + +### m-10: `.bootstrap-ignore` is empty + +**Files**: `.github/.bootstrap-ignore` (both repos) + +The file exists but contains only comments. It is never effectively used. + +**Fix**: Either populate with sensible defaults: +``` +# Exclude source-only assets from bootstrap copies +templates/ +tests/ +ANALYSIS_REPORT.md +COPILOT_REVIEW.md +tech-ai-requirements-dev.txt +__pycache__/ +.pytest_cache/ +``` +Or remove the file if the bootstrap script is deprecated (per C-01). + +--- + +## Nit Findings + +### N-01: `copilot-instructions.md` repeated "(or `.github/...` in `.github` layout)" parentheticals + +**File**: `.github/copilot-instructions.md` + +The file has 7 instances of the parenthetical `(or .github/... in .github layout)`. While this supports portability, it creates token overhead for every Codex session. Since both this repo and all known consumers use the `.github` layout, consider removing the parentheticals. + +**Fix**: Remove the parenthetical layout references from `copilot-instructions.md` and instead add a single "Layout note" section at the top: +```markdown +## Layout +This repository uses `.github/` layout. All paths below are relative to `.github/` unless otherwise noted. +``` + +--- + +### N-02: `AGENTS.md` "Anti-patterns" section — good content but verbose + +**File**: `AGENTS.md`, "Anti-patterns" section + +Seven "Do not use X when..." bullets are excellent guidance but could be more token-efficient using a table format. + +**Fix**: Convert to table: +```markdown +### Anti-patterns +| Don't | Instead | +|-------|---------| +| `TechAIPlanner` for trivial single-file changes | `TechAIImplementer` directly | +| `TechAIImplementer` for ambiguous scope | `TechAIPlanner` first | +| Generic `TechAIReviewer` for domain-specific changes | Use matching specialist | +| ... +``` + +--- + +### N-03: Sync script `SOURCE_ONLY_AGENT_PATHS` should include `tech-ai-customization-auditor` + +**File**: `.github/scripts/tech-ai-sync-copilot-configs.py` + +`tech-ai-customization-auditor.agent.md` is in `SOURCE_ONLY_AGENT_PATHS` set. Good. But verify the deprecated alias is also in the `AGENTS.md` inventory with a deprecation note. The current inventory lists it without a deprecation marker. + +**Fix**: In `AGENTS.md` inventory, annotate: +```markdown +- `.github/agents/tech-ai-customization-auditor.agent.md` *(deprecated — use TechAIGlobalCustomizationAuditor)* +``` + +--- + +### N-04: `CHANGELOG.md` entry dates — verify accuracy + +**File**: `.github/CHANGELOG.md` + +Entries are dated 2026-02-07 through 2026-03-08. Current date is 2026-03-09. The dates appear correct but double-check no entries have future dates or wrong ordering. + +**Fix**: No action needed if dates are accurate. + +--- + +### N-05: Skill files lack `dependencies:` frontmatter + +**File**: All `skills/*/SKILL.md` files + +The ANALYSIS_REPORT (item 4.1) recommended a `dependencies:` frontmatter field for machine-readable references. Skills reference instructions and other skills only in prose. + +**Fix**: Long-term improvement. Add `dependencies:` to skill frontmatter: +```yaml +--- +name: TechAITerraformFeature +description: ... +dependencies: + - instructions/terraform.instructions.md +--- +``` +This enables automated dependency validation. + +--- + +### N-06: `github-actions.instructions.md` checkout SHA example is outdated + +**File**: `.github/instructions/github-actions.instructions.md` + +The minimal example shows: +```yaml +uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.7 +``` + +But the CI workflow itself uses `v6.0.2`. The example should match the current recommended version. + +**Fix**: Update the example SHA and version to match the latest pinned version used in CI workflows. + +--- + +### N-07: `copilot-quickstart.md` suggested starter sets use instruction-only references + +**File**: `.github/templates/copilot-quickstart.md` + +The "Suggested starter sets" section recommends instructions + prompts + skills but doesn't mention agents. Consumer repos also need agent files for effective Copilot usage. + +**Fix**: Add agent recommendations to each starter set: +```markdown +- Java repositories: `java.instructions.md`, `tech-ai-java.prompt.md`, `tech-ai-project-java/SKILL.md`, plus core agents (Planner, Implementer, Reviewer) +``` + +--- + +## Architecture & Token Optimization + +These are strategic recommendations to maximize Codex's context window efficiency: + +### T-01: Flatten redundant content across `copilot-instructions.md` ↔ `AGENTS.md` + +**Current state**: Both files repeat prohibitions, validation baseline, and portability notes. +**Recommendation**: `AGENTS.md` should be the single source for repo-specific routing and inventory. `copilot-instructions.md` should contain only cross-cutting behavioral rules. Remove all duplicated content from `AGENTS.md` that's already in `copilot-instructions.md`. + +### T-02: Consider a slim `AGENTS.md` template for consumer repos + +**Current state**: The generated `AGENTS.md` for consumers is ~160 lines with full routing, prohibitions, and inventory. +**Recommendation**: The `AGENTS.template.md` is already slim. Ensure the sync script generates AGENTS.md content closer to the template length (~50-60 lines) rather than the current ~160 lines. Every line costs Codex tokens on every session. + +### T-03: Skill `SKILL.md` files should have a standardized "When to use" frontmatter field + +**Current state**: Skills explain when to use them in prose body, wasting tokens for discovery. +**Recommendation**: Add `when:` frontmatter for quick matching: +```yaml +--- +name: TechAITerraformFeature +description: Add or modify Terraform resources +when: Creating or modifying .tf files with resource, variable, output, or data blocks +--- +``` + +--- + +## Missing Assets & Gaps + +| Asset | Reason | Priority | +|-------|--------|----------| +| `CONTRIBUTING.md` | No contribution process documented | High | +| `Makefile` | No standardized developer commands | High | +| `VERSION` file | No versioning for consumer pinning | High | +| `instructions/docker.instructions.md` | Consumer repos use Docker | Medium | +| `prompts/tech-ai-docker.prompt.md` | Pair with Docker instructions | Medium | +| `skills/tech-ai-docker/SKILL.md` | Docker skill reference | Medium | +| Architecture Mermaid diagram | Visual aid for framework understanding | Medium | +| `instructions/sql.instructions.md` | DB migration safety | Low | +| `instructions/observability.instructions.md` | Cross-cutting logging standards | Low | + +--- + +## Action Checklist + +Ordered by impact on Codex effectiveness across all consumer repos: + +### Phase 1 — Immediate (highest Codex impact) +- [ ] **C-01**: Deprecate `bootstrap-copilot-config.sh` and update `DEPRECATION.md` +- [ ] **M-01**: Implement versioning strategy (`VERSION` file + git tags) +- [ ] **M-02**: Create `CONTRIBUTING.md` +- [ ] **M-03**: Create `Makefile` +- [ ] **M-08**: Archive or supersede stale `ANALYSIS_REPORT.md` +- [ ] **N-01**: Remove repeated layout parentheticals from `copilot-instructions.md` + +### Phase 2 — Short term (quality & safety) +- [ ] **M-04**: Expand validator test coverage to 15+ tests +- [ ] **M-05**: Clean up `dependabot.yml` unused ecosystems +- [ ] **M-06**: Create `docker.instructions.md` + prompt + skill +- [ ] **M-07**: Fix `copilot-quickstart.md` tool recommendations +- [ ] **m-02**: Add shellcheck to pre-commit +- [ ] **m-05**: Add enforcement status to `security-baseline.md` +- [ ] **m-08**: Add gitleaks to CI + +### Phase 3 — Medium term (polish & optimization) +- [ ] **M-09**: Standardize prompt `${input:...}` variable names +- [ ] **m-01**: Add repo-only agent exclusion note to agents README +- [ ] **m-03**: Add mypy/ruff to dev requirements +- [ ] **m-04**: Create architecture Mermaid diagram +- [ ] **m-06**: Reduce AGENTS.md preferred prompts/skills verbosity +- [ ] **m-07**: Expand CODEOWNERS team +- [ ] **m-09**: Simplify PROMPT_NAME_OVERRIDES +- [ ] **m-10**: Populate or remove `.bootstrap-ignore` + +### Phase 4 — Long term (strategic) +- [ ] **N-02**: Convert AGENTS.md anti-patterns to table +- [ ] **N-03**: Annotate deprecated agents in inventory +- [ ] **N-05**: Add `dependencies:` frontmatter to skills +- [ ] **N-06**: Update github-actions instruction example SHA +- [ ] **N-07**: Add agent recommendations to quickstart starter sets +- [ ] **T-01**: Flatten redundant content between copilot-instructions.md and AGENTS.md +- [ ] **T-02**: Optimize generated AGENTS.md length for consumers +- [ ] **T-03**: Add `when:` frontmatter to skills diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..901f8bb --- /dev/null +++ b/Makefile @@ -0,0 +1,21 @@ +PYTHON ?= python3 +VALIDATOR := .github/scripts/validate-copilot-customizations.sh +SHELL_SCRIPTS := $(wildcard .github/scripts/*.sh) + +.PHONY: help lint validate test all + +help: + @printf '%s\n' 'Targets: lint validate test all' + +lint: + bash -n $(SHELL_SCRIPTS) + shellcheck -s bash $(SHELL_SCRIPTS) + $(PYTHON) -m compileall .github/scripts tests + +validate: + bash $(VALIDATOR) --scope root --mode strict + +test: + pytest -q + +all: lint validate test diff --git a/VERSION b/VERSION new file mode 100644 index 0000000..3eefcb9 --- /dev/null +++ b/VERSION @@ -0,0 +1 @@ +1.0.0 diff --git a/tests/test_tech_ai_sync_copilot_configs.py b/tests/test_tech_ai_sync_copilot_configs.py index 0208feb..96fe755 100644 --- a/tests/test_tech_ai_sync_copilot_configs.py +++ b/tests/test_tech_ai_sync_copilot_configs.py @@ -409,7 +409,7 @@ def test_apply_plan_writes_manifest_and_managed_files(tmp_path: Path) -> None: assert not any(action.status == "conflict" for action in plan.actions if action.target_relative_path != "AGENTS.md") - MODULE.apply_plan(target_root, plan, planned_files) + MODULE.apply_plan(target_root, plan, planned_files, REPO_ROOT) manifest_path = target_root / ".github" / "tech-ai-sync-copilot-configs.manifest.json" agents_path = target_root / "AGENTS.md" @@ -418,6 +418,8 @@ def test_apply_plan_writes_manifest_and_managed_files(tmp_path: Path) -> None: manifest = json.loads(manifest_path.read_text(encoding="utf-8")) assert "AGENTS.md" in manifest["managed_files"] assert ".github/copilot-instructions.md" in manifest["managed_files"] + assert manifest["source_version"] == (REPO_ROOT / "VERSION").read_text(encoding="utf-8").strip() + assert len(manifest["source_commit"]) == 40 def test_rendered_agents_markdown_keeps_github_copilot_wording(tmp_path: Path) -> None: @@ -470,10 +472,20 @@ def test_build_plan_detects_backend_python_profile_and_python_validation_command assert plan.analysis.profile_name == "backend-python" assert "python -m compileall " in plan.selection.validation_commands - assert "pytest" in plan.selection.validation_commands + assert "pytest" not in plan.selection.validation_commands assert ".github/prompts/tech-ai-python.prompt.md" in plan.selection.prompts +def test_build_plan_adds_pytest_only_when_repo_contains_pytest_tests(tmp_path: Path) -> None: + target_root = tmp_path / "python-service-with-tests" + build_python_service_target(target_root) + write_file(target_root / "tests" / "test_app.py", 'def test_placeholder() -> None:\n assert True\n') + + plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + assert "pytest" in plan.selection.validation_commands + + def test_build_plan_prefers_tech_ai_script_prompts_to_reduce_prompt_duplication(tmp_path: Path) -> None: target_root = tmp_path / "automation" build_script_automation_target(target_root) @@ -576,6 +588,21 @@ def test_build_plan_excludes_repo_only_global_customization_agents_from_consumer assert ".github/agents/tech-ai-global-customization-auditor.agent.md" not in plan.selection.agents +def test_local_builder_triads_are_source_only_and_excluded_from_consumer_sync() -> None: + assert ( + ".github/agents/tech-ai-local-copilot-customization-builder.agent.md" + in MODULE.SOURCE_ONLY_AGENT_PATHS + ) + assert ( + ".github/prompts/tech-ai-local-copilot-customization-builder.prompt.md" + in MODULE.SOURCE_ONLY_PROMPT_PATHS + ) + assert ( + ".github/skills/tech-ai-local-copilot-customization-builder/SKILL.md" + in MODULE.SOURCE_ONLY_SKILL_PATHS + ) + + def test_build_plan_reports_unsupported_go_and_docker_stacks(tmp_path: Path) -> None: target_root = tmp_path / "polyglot" write_file(target_root / ".github" / "PULL_REQUEST_TEMPLATE.md", "# PR template\n") @@ -589,6 +616,76 @@ def test_build_plan_reports_unsupported_go_and_docker_stacks(tmp_path: Path) -> assert "unsupported target stacks: docker, go" in recommendations +def test_build_plan_detects_composite_actions_under_workflows_tree(tmp_path: Path) -> None: + target_root = tmp_path / "workflow-composite" + write_file(target_root / ".github" / "PULL_REQUEST_TEMPLATE.md", "# PR template\n") + write_file( + target_root / ".github" / "workflows" / "shared" / "action.yml", + "\n".join( + [ + "name: shared", + "runs:", + " using: composite", + " steps:", + " - shell: bash", + ' run: echo "ok"', + "", + ] + ), + ) + + plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + assert "composite-action" in plan.analysis.stacks + assert ".github/instructions/github-action-composite.instructions.md" in plan.selection.instructions + assert ".github/prompts/tech-ai-github-composite-action.prompt.md" in plan.selection.prompts + + +def test_build_plan_adds_data_registry_assets_for_json_heavy_repositories(tmp_path: Path) -> None: + target_root = tmp_path / "json-heavy" + write_file(target_root / ".github" / "PULL_REQUEST_TEMPLATE.md", "# PR template\n") + for index in range(5): + write_file(target_root / "data" / f"registry-{index}.json", '{"enabled": true}\n') + + plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + assert ".github/prompts/tech-ai-data-registry.prompt.md" in plan.selection.prompts + assert ".github/skills/tech-ai-data-registry/SKILL.md" in plan.selection.skills + + +def test_rendered_agents_markdown_uses_explicit_github_paths_and_table_routing(tmp_path: Path) -> None: + target_root = tmp_path / "rendered-agents" + build_python_service_target(target_root) + + _plan, planned_files = MODULE.build_plan(REPO_ROOT, target_root) + agents_file = next(item for item in planned_files if item.target_relative_path == "AGENTS.md") + + assert "Apply repository non-negotiables from `.github/copilot-instructions.md`." in agents_file.desired_content + assert "| Pattern | Instruction |" in agents_file.desired_content + assert "Apply all non-negotiables from `.github/copilot-instructions.md` plus:" in agents_file.desired_content + assert "- `TechAIAddUnitTests`" in agents_file.desired_content + assert "- `TechAICICDWorkflow`" in agents_file.desired_content + assert ": Add or improve unit tests for Python code" not in agents_file.desired_content + + +def test_build_plan_reports_missing_validation_workflow_and_source_only_residues(tmp_path: Path) -> None: + target_root = tmp_path / "consumer-residue" + build_python_service_target(target_root) + write_file(target_root / ".github" / "README.md", "# source-only\n") + write_file(target_root / ".github" / "agents" / "README.md", "# source-only\n") + write_file(target_root / ".github" / "templates" / "AGENTS.template.md", "# source-only\n") + write_file(target_root / ".github" / "scripts" / "bootstrap-copilot-config.sh", "#!/usr/bin/env bash\n") + + plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) + + guidance = "\n".join(plan.recommendations["missing consumer-facing validation or onboarding guidance"]) + assert "github-validate-copilot-customizations.yml" in guidance + assert ".github/README.md" in guidance + assert ".github/agents/README.md" in guidance + assert ".github/templates/**" in guidance + assert ".github/scripts/bootstrap-copilot-config.sh" in guidance + + def test_build_plan_fails_for_corrupted_manifest(tmp_path: Path) -> None: target_root = tmp_path / "broken-manifest" build_python_service_target(target_root) @@ -632,6 +729,10 @@ def test_main_writes_json_report_with_selection_and_actions(tmp_path: Path) -> N and "validation" in issue["issue_types"] for issue in payload["analysis"]["unmanaged_target_asset_issues"] ) + assert any( + "github-validate-copilot-customizations.yml" in item + for item in payload["recommendations"]["missing consumer-facing validation or onboarding guidance"] + ) assert sorted(payload["source_audit"].keys()) == [ "agents_md_repeats", "canonical_assets", diff --git a/tests/test_tech_ai_validate_copilot_customizations.py b/tests/test_tech_ai_validate_copilot_customizations.py index 5fa5042..dc845a2 100644 --- a/tests/test_tech_ai_validate_copilot_customizations.py +++ b/tests/test_tech_ai_validate_copilot_customizations.py @@ -14,15 +14,21 @@ def copy_copilot_config(target_root: Path) -> None: (target_root / "AGENTS.md").write_text((REPO_ROOT / "AGENTS.md").read_text(encoding="utf-8"), encoding="utf-8") -def run_validator(repo_root: Path, report_file: Path) -> subprocess.CompletedProcess[str]: +def run_validator( + repo_root: Path, + report_file: Path, + *, + scope: str = "root", + mode: str = "strict", +) -> subprocess.CompletedProcess[str]: return subprocess.run( [ "bash", str(repo_root / ".github" / "scripts" / "validate-copilot-customizations.sh"), "--scope", - "root", + scope, "--mode", - "strict", + mode, "--report", "json", "--report-file", @@ -169,11 +175,103 @@ def test_tech_ai_validator_enforces_global_builder_semantic_sections(tmp_path: P assert any("Global customization builder missing '## Token discipline' section" in message for message in messages) -def test_root_agents_routes_customization_work_to_global_agents() -> None: +def test_tech_ai_validator_scope_all_covers_immediate_subrepos(tmp_path: Path) -> None: + workspace_root = tmp_path / "workspace" + copy_copilot_config(workspace_root) + copy_copilot_config(workspace_root / "consumer-a") + + report_file = tmp_path / "tech-ai-validator-all.json" + result = run_validator(workspace_root, report_file, scope="all") + + payload = json.loads(report_file.read_text(encoding="utf-8")) + assert result.returncode == 0, f"{result.stdout}\n{result.stderr}" + assert payload["status"] == "passed" + assert payload["scope"] == "all" + + +def test_tech_ai_validator_legacy_compatible_allows_legacy_prompt_conventions(tmp_path: Path) -> None: + target_root = tmp_path / "legacy-compatible" + copy_copilot_config(target_root) + + prompt_path = target_root / ".github" / "prompts" / "tech-ai-bash-script.prompt.md" + prompt_text = prompt_path.read_text(encoding="utf-8") + prompt_text = prompt_text.replace("name: TechAIBashScript\n", "") + prompt_text = prompt_text.replace("argument-hint: action= script_name= purpose= [target_path=] [target_file=]\n", "") + prompt_text = prompt_text.replace("---\n\n# TechAI Bash Script", "mode: create\n---\n\n# TechAI Bash Script") + prompt_text = prompt_text.replace("## Validation\n", "## Legacy Validation\n") + prompt_path.write_text(prompt_text, encoding="utf-8") + + report_file = tmp_path / "tech-ai-validator-legacy-compatible.json" + result = run_validator(target_root, report_file, mode="legacy-compatible") + + payload = json.loads(report_file.read_text(encoding="utf-8")) + messages = [finding["message"] for finding in payload["findings"]] + assert result.returncode == 0, f"{result.stdout}\n{result.stderr}" + assert payload["status"] == "passed-with-warnings" + assert payload["warnings"] > 0 + assert any("Legacy prompt key 'mode' found" in message for message in messages) + + +def test_tech_ai_validator_reports_malformed_frontmatter(tmp_path: Path) -> None: + target_root = tmp_path / "malformed-frontmatter" + copy_copilot_config(target_root) + + prompt_path = target_root / ".github" / "prompts" / "tech-ai-python.prompt.md" + prompt_path.write_text( + prompt_path.read_text(encoding="utf-8").replace("---\n\n# Python Project Task", "\n# Python Project Task", 1), + encoding="utf-8", + ) + + report_file = tmp_path / "tech-ai-validator-malformed-frontmatter.json" + result = run_validator(target_root, report_file) + + payload = json.loads(report_file.read_text(encoding="utf-8")) + messages = [finding["message"] for finding in payload["findings"]] + assert result.returncode == 1 + assert payload["status"] == "failed" + assert any("malformed frontmatter fence" in message for message in messages) + + +def test_tech_ai_validator_requires_release_comment_for_workflow_sha_pins(tmp_path: Path) -> None: + target_root = tmp_path / "workflow-sha-comment" + copy_copilot_config(target_root) + + workflow_path = target_root / ".github" / "workflows" / "custom.yml" + workflow_path.parent.mkdir(parents=True, exist_ok=True) + workflow_path.write_text( + "\n".join( + [ + "name: custom", + "on: push", + "permissions:", + " contents: read", + "jobs:", + " validate:", + " runs-on: ubuntu-latest", + " steps:", + " - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd", + "", + ] + ), + encoding="utf-8", + ) + + report_file = tmp_path / "tech-ai-validator-workflow-sha-comment.json" + result = run_validator(target_root, report_file) + + payload = json.loads(report_file.read_text(encoding="utf-8")) + messages = [finding["message"] for finding in payload["findings"]] + assert result.returncode == 1 + assert payload["status"] == "failed" + assert any("Workflow SHA pin is missing adjacent release URL comment" in message for message in messages) + + +def test_root_agents_routes_customization_work_to_global_and_local_customization_agents() -> None: agents_text = (REPO_ROOT / "AGENTS.md").read_text(encoding="utf-8") assert "TechAIGlobalCustomizationBuilder" in agents_text assert "TechAIGlobalCustomizationAuditor" in agents_text + assert "TechAILocalCopilotCustomizationBuilder" in agents_text assert "repo-only" in agents_text assert "## Available Skills" not in agents_text assert "## Available Prompts" not in agents_text From 604621a13490cc91dbad23910d45f97ba4eaceb7 Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Tue, 10 Mar 2026 09:56:22 +0100 Subject: [PATCH 6/9] feat: enforce grounding of local Copilot assets against concrete target files --- .github/CHANGELOG.md | 1 + ...cal-copilot-customization-builder.agent.md | 2 ++ ...al-copilot-customization-builder.prompt.md | 13 +++++++----- .../SKILL.md | 20 ++++++++++++------- ...tech_ai_validate_copilot_customizations.py | 19 ++++++++++++++++++ 5 files changed, 43 insertions(+), 12 deletions(-) diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index 7c2f345..de30243 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -8,6 +8,7 @@ Use this format for new updates: ## 2026-03-09 - Added the repo-only `TechAILocalCopilotCustomizationBuilder` agent, prompt, and skill for creating consumer-local `local-*` Copilot assets without duplicating the shared baseline, and excluded the trio from consumer sync. +- Tightened `TechAILocalCopilotCustomizationBuilder` so it must ground repo-local prompts, examples, schema snippets, and naming rules on concrete target files instead of generic remembered patterns. - Deprecated `.github/scripts/bootstrap-copilot-config.sh` in favor of `.github/scripts/tech-ai-sync-copilot-configs.py`, updated lifecycle docs, and made quickstart plus `.github/README.md` prefer sync-first alignment. - Added source release metadata with root `VERSION`, contributor workflow documentation, and manifest provenance fields for source version and commit. - Tightened consumer alignment: improved composite-action detection, enabled data-registry selection for JSON-heavy repositories, slimmed generated `AGENTS.md`, removed spurious `pytest` recommendations for repos without pytest tests, and added sync recommendations for missing Copilot validation workflows plus legacy source-only residues. diff --git a/.github/agents/tech-ai-local-copilot-customization-builder.agent.md b/.github/agents/tech-ai-local-copilot-customization-builder.agent.md index eff0873..0e07e6b 100644 --- a/.github/agents/tech-ai-local-copilot-customization-builder.agent.md +++ b/.github/agents/tech-ai-local-copilot-customization-builder.agent.md @@ -15,6 +15,7 @@ Create and refine consumer-repository Copilot customization assets that must rem - Do not duplicate a capability that already exists in the synced baseline unless the requested behavior is genuinely repository-specific. - Do not overwrite manifest-managed synced files unless explicitly requested and conflict-safe. - Do not sync workflows, templates, changelog files, or bootstrap helpers from the source repository as part of local customization work. +- Do not infer target schema, naming conventions, identity normalization rules, or example payloads from memory; inspect concrete target files first and ground every local asset against them. - Keep repository-facing text in English and use GitHub Copilot terminology only. ## Routing @@ -24,6 +25,7 @@ Create and refine consumer-repository Copilot customization assets that must rem ## Output Contract - `Baseline check`: whether the consumer already has the required synced Copilot core assets and validator coverage. +- `Target evidence`: concrete files, field names, naming patterns, and validation commands used to ground the local asset. - `Local customization decision`: why a new `local-*` asset is needed instead of reusing an existing `tech-ai-*` capability. - `File plan`: `local-*` prompts, skills, agents, and `AGENTS.md` updates to create or modify. - `Validation`: target-repository validation commands run and their results. diff --git a/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md b/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md index 028de02..121cd99 100644 --- a/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md +++ b/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md @@ -19,16 +19,19 @@ Use this prompt to create or refine repository-owned `local-*` Copilot assets in ## Instructions 1. Use `.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md` as the workflow definition. 2. If the target baseline is missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first. -3. Create only the narrowest local asset set that solves the request. -4. Keep repository-owned prompt, skill, and agent filenames plus frontmatter `name:` values on the `local-*` convention. -5. Update the target `AGENTS.md` inventory and routing only as needed, keeping `.github/...` paths explicit. -6. Do not create local duplicates of existing `tech-ai-*` capabilities unless the repo-specific behavior materially differs. -7. Report whether the capability should remain local or be proposed for promotion into the shared source baseline. +3. Inspect one or more concrete target files that the local asset will operate on, then derive schema, naming conventions, identity formats, examples, and validations from those files. +4. If no suitable target file exists, stop and report the missing grounding instead of inventing schema fields, examples, or naming rules. +5. Create only the narrowest local asset set that solves the request. +6. Keep repository-owned prompt, skill, and agent filenames plus frontmatter `name:` values on the `local-*` convention. +7. Update the target `AGENTS.md` inventory and routing only as needed, keeping `.github/...` paths explicit. +8. Do not create local duplicates of existing `tech-ai-*` capabilities unless the repo-specific behavior materially differs. +9. Report whether the capability should remain local or be proposed for promotion into the shared source baseline. ## Minimal example - Input: `target_repo=/workspace/consumer-repo change="Add a repo-local prompt for onboarding external users" local_asset_type=prompt` - Expected output: - Baseline check for synced Copilot core assets in the target repo. + - Target evidence listing the concrete repo files used to derive schema, naming, and examples. - Minimal `local-*` asset plan with naming and placement rationale. - `AGENTS.md` inventory and routing updates only if needed. - Target-repo validation results and promotion recommendation. diff --git a/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md b/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md index 98dc3cb..9a2ee3f 100644 --- a/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md +++ b/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md @@ -12,31 +12,37 @@ description: Create or update repository-owned local GitHub Copilot customizatio ## Workflow 1. Inspect the target repository layout, `.github` contents, root `AGENTS.md`, git state, and existing local Copilot assets. -2. Confirm the baseline is current enough for local customization work: +2. Identify at least one representative target file for each requested local capability and extract the actual schema field names, naming patterns, identity formats, and validation commands from those files before drafting any `local-*` asset. +3. Confirm the baseline is current enough for local customization work: - if `copilot-instructions.md`, the validator script, or expected synced assets are missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first; - use the sync report to avoid creating a `local-*` asset that duplicates an available shared baseline capability. -3. Decide the narrowest asset type that solves the request: +4. Decide the narrowest asset type that solves the request: - create or update a `local-*.prompt.md` when the behavior is mostly task instructions; - add a `local-*` skill only when the workflow needs reusable implementation detail beyond the prompt; - add a `local-*` agent only when the repo needs durable routing or persona guidance that cannot stay in a prompt or skill. -4. Enforce local naming and ownership rules: +5. Enforce local naming and ownership rules: - filenames for repo-owned prompts, skills, and agents must start with `local-`; - prompt, skill, and agent frontmatter `name:` values must also start with `local-`; - keep local assets in `.github/prompts`, `.github/skills//SKILL.md`, and `.github/agents`. -5. Keep local assets minimal and compatible with the shared baseline: +6. Ground the local asset content on target-repository evidence: + - derive prompt examples, schema snippets, naming rules, and validation commands from real target files instead of generic or remembered patterns; + - do not invent fields, object shapes, identity suffixes, or naming conventions that are not present in the inspected files; + - if multiple target patterns exist, narrow the prompt scope to the chosen pattern and cite the representative files; + - if no stable target pattern exists, stop and report the missing grounding rather than authoring ambiguous local guidance. +7. Keep local assets minimal and compatible with the shared baseline: - reuse `.github/instructions/*.instructions.md` and synced `tech-ai-*` skills when possible instead of copying large guidance blocks; - reference shared prompts or skills by path when they already cover most of the behavior; - avoid creating a local canonical duplicate of an existing `tech-ai-*` capability. -6. Update `AGENTS.md` in the target repository: +8. Update `AGENTS.md` in the target repository: - keep explicit `.github/...` paths; - add the local assets to the inventory; - adjust routing or preferred prompts or skills only when the new local capability should be discoverable by default; - avoid duplicating long prompt or skill descriptions inside `AGENTS.md`. -7. Validate the target repository after changes: +9. Validate the target repository after changes: - run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict`; - run stack-specific checks for touched Bash, Python, JSON, YAML, Markdown, or Terraform assets; - if the repo has a synced baseline manifest, preserve it and do not rewrite managed files opportunistically. -8. Report the result with changed files, validation output, residual repo-local risks, and whether the new capability should stay local or be proposed for promotion into the shared source baseline. +10. Report the result with the concrete target files used for grounding, changed files, validation output, residual repo-local risks, and whether the new capability should stay local or be proposed for promotion into the shared source baseline. ## Scope rules - Manage consumer-repository Copilot assets only. diff --git a/tests/test_tech_ai_validate_copilot_customizations.py b/tests/test_tech_ai_validate_copilot_customizations.py index dc845a2..96aaa72 100644 --- a/tests/test_tech_ai_validate_copilot_customizations.py +++ b/tests/test_tech_ai_validate_copilot_customizations.py @@ -293,3 +293,22 @@ def test_global_builder_maps_consolidated_rules_and_legacy_auditor_is_deprecated assert "validate-copilot-customizations.sh" in builder_text assert "Deprecated compatibility alias" in legacy_auditor_text assert "TechAIGlobalCustomizationAuditor" in legacy_auditor_text + + +def test_local_builder_requires_grounding_against_concrete_target_files() -> None: + agent_text = ( + REPO_ROOT / ".github" / "agents" / "tech-ai-local-copilot-customization-builder.agent.md" + ).read_text(encoding="utf-8") + prompt_text = ( + REPO_ROOT / ".github" / "prompts" / "tech-ai-local-copilot-customization-builder.prompt.md" + ).read_text(encoding="utf-8") + skill_text = ( + REPO_ROOT / ".github" / "skills" / "tech-ai-local-copilot-customization-builder" / "SKILL.md" + ).read_text(encoding="utf-8") + + assert "inspect concrete target files first" in agent_text + assert "`Target evidence`" in agent_text + assert "Inspect one or more concrete target files" in prompt_text + assert "stop and report the missing grounding" in prompt_text + assert "Identify at least one representative target file" in skill_text + assert "do not invent fields, object shapes, identity suffixes, or naming conventions" in skill_text From 2cec345b612b7175d2c585cb2804ed084b363724 Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Tue, 10 Mar 2026 10:52:58 +0100 Subject: [PATCH 7/9] Refactor Copilot customization assets from local to internal prefix - Updated all references from `local-*` to `internal-*` in prompts, skills, agents, and validation scripts to align with new naming conventions. - Removed the `TechAILocalCopilotCustomizationBuilder` skill and replaced it with `TechAIInternalCopilotCustomizationBuilder`. - Adjusted validation scripts to enforce the new `internal-*` prefix for repository-owned assets. - Updated documentation and contributing guidelines to reflect the changes in asset naming and usage. - Added new `tech-ai-internal-copilot-customization-builder` skill documentation. - Modified tests to ensure they validate the new internal prefix requirements. --- .github/CHANGELOG.md | 6 +- .github/agents/README.md | 8 +- ...al-copilot-customization-builder.agent.md} | 24 +++--- .../tech-ai-sync-copilot-configs.agent.md | 2 +- ...al-copilot-customization-builder.prompt.md | 41 ++++++++++ ...al-copilot-customization-builder.prompt.md | 41 ---------- .../tech-ai-sync-copilot-configs.prompt.md | 2 +- .../scripts/tech-ai-sync-copilot-configs.py | 76 +++++++++---------- .../validate-copilot-customizations.sh | 62 +++++++-------- .../SKILL.md | 57 ++++++++++++++ .../SKILL.md | 57 -------------- .../tech-ai-sync-copilot-configs/SKILL.md | 2 +- AGENTS.md | 24 +++--- CONTRIBUTING.md | 2 +- tests/test_tech_ai_sync_copilot_configs.py | 42 +++++----- ...tech_ai_validate_copilot_customizations.py | 26 +++---- 16 files changed, 236 insertions(+), 236 deletions(-) rename .github/agents/{tech-ai-local-copilot-customization-builder.agent.md => tech-ai-internal-copilot-customization-builder.agent.md} (57%) create mode 100644 .github/prompts/tech-ai-internal-copilot-customization-builder.prompt.md delete mode 100644 .github/prompts/tech-ai-local-copilot-customization-builder.prompt.md create mode 100644 .github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md delete mode 100644 .github/skills/tech-ai-local-copilot-customization-builder/SKILL.md diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index de30243..1432a98 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -7,8 +7,8 @@ Use this format for new updates: - Include file/path scope when useful. ## 2026-03-09 -- Added the repo-only `TechAILocalCopilotCustomizationBuilder` agent, prompt, and skill for creating consumer-local `local-*` Copilot assets without duplicating the shared baseline, and excluded the trio from consumer sync. -- Tightened `TechAILocalCopilotCustomizationBuilder` so it must ground repo-local prompts, examples, schema snippets, and naming rules on concrete target files instead of generic remembered patterns. +- Added the repo-only `TechAIInternalCopilotCustomizationBuilder` agent, prompt, and skill for creating consumer-repository `internal-*` Copilot assets without duplicating the shared baseline, and excluded the trio from consumer sync. +- Tightened `TechAIInternalCopilotCustomizationBuilder` so it must ground repo-local prompts, examples, schema snippets, and naming rules on concrete target files instead of generic remembered patterns. - Deprecated `.github/scripts/bootstrap-copilot-config.sh` in favor of `.github/scripts/tech-ai-sync-copilot-configs.py`, updated lifecycle docs, and made quickstart plus `.github/README.md` prefer sync-first alignment. - Added source release metadata with root `VERSION`, contributor workflow documentation, and manifest provenance fields for source version and commit. - Tightened consumer alignment: improved composite-action detection, enabled data-registry selection for JSON-heavy repositories, slimmed generated `AGENTS.md`, removed spurious `pytest` recommendations for repos without pytest tests, and added sync recommendations for missing Copilot validation workflows plus legacy source-only residues. @@ -17,7 +17,7 @@ Use this format for new updates: ## 2026-03-08 - Updated the PR-writing prompt, skill, and agent guidance to derive required sections from the resolved repository PR template instead of hardcoding older headings such as `Security and Compliance` or `Related Links`. -- Updated `scripts/tech-ai-sync-copilot-configs.py` and `scripts/validate-copilot-customizations.sh` so repository-owned prompt, skill, and agent assets outside the synced global baseline must use `local-*` in both filenames and `name:` values, making local customizations visibly distinct from synced `tech-ai-*` assets. +- Updated `scripts/tech-ai-sync-copilot-configs.py` and `scripts/validate-copilot-customizations.sh` so repository-owned prompt, skill, and agent assets outside the synced global baseline must use `internal-*` in both filenames and `name:` values, making internal customizations visibly distinct from synced `tech-ai-*` assets. - Updated `scripts/tech-ai-sync-copilot-configs.py` so target-only skill detection compares full relative paths instead of the shared `SKILL.md` filename, fixing missed unmanaged skill assets in consumer repositories. - Expanded sync planning to audit unmanaged target-local instructions, prompts, skills, and agents for strict validation gaps and legacy alias drift, and added the new report section in both markdown and JSON outputs. - Updated sync planning so legacy aliases such as `cs-*`, unprefixed prompt names, and legacy skill directories are reported even when the canonical family is outside the selected minimum baseline. diff --git a/.github/agents/README.md b/.github/agents/README.md index b9a8eaa..0b74689 100644 --- a/.github/agents/README.md +++ b/.github/agents/README.md @@ -13,12 +13,12 @@ This folder contains optional custom agents for focused tasks. - PR-focused: `TechAIPRWriter`. - Write-capable: `TechAIImplementer`. - Repo-only standards specialists: `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`. -- Repo-only consumer-local specialist: `TechAILocalCopilotCustomizationBuilder`. +- Repo-only consumer-repository specialist: `TechAIInternalCopilotCustomizationBuilder`. ## Repo-only agents (not synced to consumers) - `TechAIGlobalCustomizationBuilder` - `TechAIGlobalCustomizationAuditor` -- `TechAILocalCopilotCustomizationBuilder` +- `TechAIInternalCopilotCustomizationBuilder` - `TechAIScriptReviewer` - `TechAISyncCopilotConfigs` - `TechAICustomizationAuditor` (deprecated compatibility alias) @@ -38,5 +38,5 @@ This folder contains optional custom agents for focused tasks. 7. Use `TechAISecurityReviewer` as final security gate. 8. Use `TechAIGlobalCustomizationBuilder` for GitHub Copilot customization assets in this standards repository. 9. Use `TechAIGlobalCustomizationAuditor` as the final gate for those customization changes. -10. Use `TechAISyncCopilotConfigs` to align a consumer baseline before creating repo-owned local assets. -11. Use `TechAILocalCopilotCustomizationBuilder` for repo-owned `local-*` prompts, skills, agents, and `AGENTS.md` updates that should stay consumer-local. +10. Use `TechAISyncCopilotConfigs` to align a consumer baseline before creating repo-owned internal assets. +11. Use `TechAIInternalCopilotCustomizationBuilder` for repo-owned `internal-*` prompts, skills, agents, and `AGENTS.md` updates that should stay consumer-repository. diff --git a/.github/agents/tech-ai-local-copilot-customization-builder.agent.md b/.github/agents/tech-ai-internal-copilot-customization-builder.agent.md similarity index 57% rename from .github/agents/tech-ai-local-copilot-customization-builder.agent.md rename to .github/agents/tech-ai-internal-copilot-customization-builder.agent.md index 0e07e6b..5e8e76b 100644 --- a/.github/agents/tech-ai-local-copilot-customization-builder.agent.md +++ b/.github/agents/tech-ai-internal-copilot-customization-builder.agent.md @@ -1,32 +1,32 @@ --- -description: Create or update repository-owned local GitHub Copilot customization assets in a consumer repository without duplicating the shared baseline. -name: TechAILocalCopilotCustomizationBuilder +description: Create or update repository-owned internal GitHub Copilot customization assets in a consumer repository without duplicating the shared baseline. +name: TechAIInternalCopilotCustomizationBuilder tools: ["search", "usages", "problems", "editFiles", "runTerminal", "fetch"] --- -# TechAI Local Copilot Customization Builder Agent +# TechAI Internal Copilot Customization Builder Agent ## Objective -Create and refine consumer-repository Copilot customization assets that must remain local, using the `local-*` naming convention, preserving the synced baseline, and keeping the target `AGENTS.md` plus validation state coherent. +Create and refine consumer-repository Copilot customization assets that must remain internal, using the `internal-*` naming convention, preserving the synced baseline, and keeping the target `AGENTS.md` plus validation state coherent. ## Restrictions - Do not modify target `README.md` files unless explicitly requested. -- Do not create repository-owned prompt, skill, or agent assets with the `tech-ai-*` filename prefix or `TechAI*` name values; use `local-*` for both filenames and frontmatter `name:`. +- Do not create repository-owned prompt, skill, or agent assets with the `tech-ai-*` filename prefix or `TechAI*` name values; use `internal-*` for both filenames and frontmatter `name:`. - Do not duplicate a capability that already exists in the synced baseline unless the requested behavior is genuinely repository-specific. - Do not overwrite manifest-managed synced files unless explicitly requested and conflict-safe. - Do not sync workflows, templates, changelog files, or bootstrap helpers from the source repository as part of local customization work. -- Do not infer target schema, naming conventions, identity normalization rules, or example payloads from memory; inspect concrete target files first and ground every local asset against them. +- Do not infer target schema, naming conventions, identity normalization rules, or example payloads from memory; inspect concrete target files first and ground every internal asset against them. - Keep repository-facing text in English and use GitHub Copilot terminology only. ## Routing -- Use this agent when a consumer repository needs repo-owned prompts, skills, agents, or `AGENTS.md` wiring that must stay local. -- If the consumer baseline is missing or stale, start with `TechAISyncCopilotConfigs` in `plan` mode before creating new local assets. -- Treat `.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md` as the workflow definition. +- Use this agent when a consumer repository needs repo-owned prompts, skills, agents, or `AGENTS.md` wiring that must stay internal. +- If the consumer baseline is missing or stale, start with `TechAISyncCopilotConfigs` in `plan` mode before creating new internal assets. +- Treat `.github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md` as the workflow definition. ## Output Contract - `Baseline check`: whether the consumer already has the required synced Copilot core assets and validator coverage. -- `Target evidence`: concrete files, field names, naming patterns, and validation commands used to ground the local asset. -- `Local customization decision`: why a new `local-*` asset is needed instead of reusing an existing `tech-ai-*` capability. -- `File plan`: `local-*` prompts, skills, agents, and `AGENTS.md` updates to create or modify. +- `Target evidence`: concrete files, field names, naming patterns, and validation commands used to ground the internal asset. +- `Internal customization decision`: why a new `internal-*` asset is needed instead of reusing an existing `tech-ai-*` capability. +- `File plan`: `internal-*` prompts, skills, agents, and `AGENTS.md` updates to create or modify. - `Validation`: target-repository validation commands run and their results. - `Promotion note`: whether the local capability should remain repo-only or be a candidate for promotion back to `cloud-strategy.github`. diff --git a/.github/agents/tech-ai-sync-copilot-configs.agent.md b/.github/agents/tech-ai-sync-copilot-configs.agent.md index 9d4e089..5f877b9 100644 --- a/.github/agents/tech-ai-sync-copilot-configs.agent.md +++ b/.github/agents/tech-ai-sync-copilot-configs.agent.md @@ -25,7 +25,7 @@ Analyze a local target repository, select the minimum Copilot customization asse - `Target analysis`: repo shape, selected profile, stacks, git state, and AGENTS location. - `Source audit`: canonical assets, legacy aliases, role overlaps, AGENTS.md repeats, and source-side recommendations. - `Asset selection`: instructions, prompts, skills, agents, and baseline files chosen from the source repository. -- `Unmanaged target asset issues`: target-local instructions, prompts, skills, or agents outside the selected sync baseline, including strict validation gaps, `local-*` naming violations for repository-owned prompt/skill/agent assets, and legacy alias drift. +- `Unmanaged target asset issues`: target-local instructions, prompts, skills, or agents outside the selected sync baseline, including strict validation gaps, `internal-*` naming violations for repository-owned prompt/skill/agent assets, and legacy alias drift. - `Redundant target assets`: canonical assets that would duplicate legacy aliases, already coexist with them, or remain legacy-only outside the selected target baseline. - `File actions`: create, update, adopt, unchanged, and conflict results. - `Recommendations`: categorized source-repository improvements. diff --git a/.github/prompts/tech-ai-internal-copilot-customization-builder.prompt.md b/.github/prompts/tech-ai-internal-copilot-customization-builder.prompt.md new file mode 100644 index 0000000..c4c3196 --- /dev/null +++ b/.github/prompts/tech-ai-internal-copilot-customization-builder.prompt.md @@ -0,0 +1,41 @@ +--- +description: Create or update repository-owned internal GitHub Copilot customization assets in a consumer repo while preserving the shared baseline +name: TechAIInternalCopilotCustomizationBuilder +agent: agent +argument-hint: target_repo= change= [internal_asset_type=] [promote_to_source=] +--- + +# TechAI Internal Copilot Customization Builder + +## Context +Use this prompt to create or refine repository-owned `internal-*` Copilot assets in a consumer repository without duplicating the shared `tech-ai-*` baseline. + +## Required inputs +- **Target repository**: ${input:target_repo} +- **Requested change**: ${input:change} +- **Internal asset type**: ${input:internal_asset_type:prompt,skill,agent,triad} +- **Promote to source**: ${input:promote_to_source:no} + +## Instructions +1. Use `.github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md` as the workflow definition. +2. If the target baseline is missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first. +3. Inspect one or more concrete target files that the internal asset will operate on, then derive schema, naming conventions, identity formats, examples, and validations from those files. +4. If no suitable target file exists, stop and report the missing grounding instead of inventing schema fields, examples, or naming rules. +5. Create only the narrowest internal asset set that solves the request. +6. Keep repository-owned prompt, skill, and agent filenames plus frontmatter `name:` values on the `internal-*` convention. +7. Update the target `AGENTS.md` inventory and routing only as needed, keeping `.github/...` paths explicit. +8. Do not create internal duplicates of existing `tech-ai-*` capabilities unless the repo-specific behavior materially differs. +9. Report whether the capability should remain internal or be proposed for promotion into the shared source baseline. + +## Minimal example +- Input: `target_repo=/workspace/consumer-repo change="Add a repo-internal prompt for onboarding external users" internal_asset_type=prompt` +- Expected output: + - Baseline check for synced Copilot core assets in the target repo. + - Target evidence listing the concrete repo files used to derive schema, naming, and examples. + - Minimal `internal-*` asset plan with naming and placement rationale. + - `AGENTS.md` inventory and routing updates only if needed. + - Target-repo validation results and promotion recommendation. + +## Validation +- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` in the target repo after changing internal Copilot assets. +- Run relevant Bash, Python, Terraform, YAML, JSON, or Markdown checks for the touched files. diff --git a/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md b/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md deleted file mode 100644 index 121cd99..0000000 --- a/.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: Create or update repository-owned local GitHub Copilot customization assets in a consumer repo while preserving the shared baseline -name: TechAILocalCopilotCustomizationBuilder -agent: agent -argument-hint: target_repo= change= [local_asset_type=] [promote_to_source=] ---- - -# TechAI Local Copilot Customization Builder - -## Context -Use this prompt to create or refine repository-owned `local-*` Copilot assets in a consumer repository without duplicating the shared `tech-ai-*` baseline. - -## Required inputs -- **Target repository**: ${input:target_repo} -- **Requested change**: ${input:change} -- **Local asset type**: ${input:local_asset_type:prompt,skill,agent,triad} -- **Promote to source**: ${input:promote_to_source:no} - -## Instructions -1. Use `.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md` as the workflow definition. -2. If the target baseline is missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first. -3. Inspect one or more concrete target files that the local asset will operate on, then derive schema, naming conventions, identity formats, examples, and validations from those files. -4. If no suitable target file exists, stop and report the missing grounding instead of inventing schema fields, examples, or naming rules. -5. Create only the narrowest local asset set that solves the request. -6. Keep repository-owned prompt, skill, and agent filenames plus frontmatter `name:` values on the `local-*` convention. -7. Update the target `AGENTS.md` inventory and routing only as needed, keeping `.github/...` paths explicit. -8. Do not create local duplicates of existing `tech-ai-*` capabilities unless the repo-specific behavior materially differs. -9. Report whether the capability should remain local or be proposed for promotion into the shared source baseline. - -## Minimal example -- Input: `target_repo=/workspace/consumer-repo change="Add a repo-local prompt for onboarding external users" local_asset_type=prompt` -- Expected output: - - Baseline check for synced Copilot core assets in the target repo. - - Target evidence listing the concrete repo files used to derive schema, naming, and examples. - - Minimal `local-*` asset plan with naming and placement rationale. - - `AGENTS.md` inventory and routing updates only if needed. - - Target-repo validation results and promotion recommendation. - -## Validation -- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` in the target repo after changing local Copilot assets. -- Run relevant Bash, Python, Terraform, YAML, JSON, or Markdown checks for the touched files. diff --git a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md index 87c716f..95bd53a 100644 --- a/.github/prompts/tech-ai-sync-copilot-configs.prompt.md +++ b/.github/prompts/tech-ai-sync-copilot-configs.prompt.md @@ -23,7 +23,7 @@ Use this prompt to analyze a local repository, select the minimum Copilot custom 3. Start with `mode=plan`; use `mode=apply` only when explicitly requested and only after a conflict-safe plan. 4. Keep scope limited to Copilot core assets only. 5. Preserve unmanaged target files and report conflicts instead of overwriting them. -6. Audit target-local instructions, prompts, skills, and agents that fall outside the selected sync baseline and report strict validation gaps, `local-*` naming violations for repository-owned prompt/skill/agent assets, or legacy alias drift. +6. Audit target-local instructions, prompts, skills, and agents that fall outside the selected sync baseline and report strict validation gaps, `internal-*` naming violations for repository-owned prompt/skill/agent assets, or legacy alias drift. 7. Render `AGENTS.md` inventory from the desired managed baseline plus the Copilot assets already present in the target repository. 8. Report source-side audit findings separately from target-side unmanaged asset issues, redundant or legacy target assets, and file actions. diff --git a/.github/scripts/tech-ai-sync-copilot-configs.py b/.github/scripts/tech-ai-sync-copilot-configs.py index 25f44d8..75ddda9 100644 --- a/.github/scripts/tech-ai-sync-copilot-configs.py +++ b/.github/scripts/tech-ai-sync-copilot-configs.py @@ -38,7 +38,7 @@ ".github/agents/tech-ai-customization-auditor.agent.md", ".github/agents/tech-ai-global-customization-auditor.agent.md", ".github/agents/tech-ai-global-customization-builder.agent.md", - ".github/agents/tech-ai-local-copilot-customization-builder.agent.md", + ".github/agents/tech-ai-internal-copilot-customization-builder.agent.md", ".github/agents/tech-ai-script-reviewer.agent.md", ".github/agents/tech-ai-sync-copilot-configs.agent.md", } @@ -46,12 +46,12 @@ ".github/prompts/tech-ai-add-platform.prompt.md", ".github/prompts/tech-ai-add-report-script.prompt.md", ".github/prompts/tech-ai-code-review.prompt.md", - ".github/prompts/tech-ai-local-copilot-customization-builder.prompt.md", + ".github/prompts/tech-ai-internal-copilot-customization-builder.prompt.md", ".github/prompts/tech-ai-sync-copilot-configs.prompt.md", } SOURCE_ONLY_SKILL_PATHS = { ".github/skills/tech-ai-code-review/SKILL.md", - ".github/skills/tech-ai-local-copilot-customization-builder/SKILL.md", + ".github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md", ".github/skills/tech-ai-sync-copilot-configs/SKILL.md", } CANONICAL_BASH_SCRIPT_PROMPT_PATH = ".github/prompts/tech-ai-bash-script.prompt.md" @@ -535,7 +535,7 @@ def prompt_expected_name(relative_path: str) -> str | None: return None -def local_asset_identifier(relative_path: str) -> str | None: +def internal_asset_identifier(relative_path: str) -> str | None: path = Path(relative_path) category = asset_category(relative_path) @@ -548,9 +548,9 @@ def local_asset_identifier(relative_path: str) -> str | None: return None -def is_local_asset_path(relative_path: str) -> bool: - identifier = local_asset_identifier(relative_path) - return bool(identifier and identifier.startswith("local-")) +def is_internal_asset_path(relative_path: str) -> bool: + identifier = internal_asset_identifier(relative_path) + return bool(identifier and identifier.startswith("internal-")) def scan_repo_files(repo_root: Path) -> list[Path]: @@ -1378,13 +1378,13 @@ def validate_unmanaged_prompt_asset(target_root: Path, relative_path: str, repo_ issues.append(f"Prompt name policy mismatch: expected `{expected_name}`, found `{actual_name}`.") if repo_local: - if not is_local_asset_path(relative_path): - issues.append("Repository-local prompt filename must start with `local-`.") - if actual_name and not actual_name.startswith("local-"): - issues.append("Repository-local prompt `name` must start with `local-`.") - local_identifier = local_asset_identifier(relative_path) - if local_identifier and local_identifier.startswith("local-") and actual_name and actual_name != local_identifier: - issues.append(f"Repository-local prompt `name` should match filename stem `{local_identifier}`.") + if not is_internal_asset_path(relative_path): + issues.append("Repository-internal prompt filename must start with `internal-`.") + if actual_name and not actual_name.startswith("internal-"): + issues.append("Repository-internal prompt `name` must start with `internal-`.") + internal_identifier = internal_asset_identifier(relative_path) + if internal_identifier and internal_identifier.startswith("internal-") and actual_name and actual_name != internal_identifier: + issues.append(f"Repository-internal prompt `name` should match filename stem `{internal_identifier}`.") for heading in ("## Instructions", "## Validation", "## Minimal example"): if not has_heading_exact(path, heading): @@ -1417,14 +1417,14 @@ def validate_unmanaged_skill_asset(target_root: Path, relative_path: str, repo_l issues.append("Missing validation/testing section.") if repo_local: - local_identifier = local_asset_identifier(relative_path) + internal_identifier = internal_asset_identifier(relative_path) actual_name = frontmatter.get("name", "") - if not is_local_asset_path(relative_path): - issues.append("Repository-local skill directory must start with `local-`.") - if actual_name and not actual_name.startswith("local-"): - issues.append("Repository-local skill `name` must start with `local-`.") - if local_identifier and local_identifier.startswith("local-") and actual_name and actual_name != local_identifier: - issues.append(f"Repository-local skill `name` should match directory name `{local_identifier}`.") + if not is_internal_asset_path(relative_path): + issues.append("Repository-internal skill directory must start with `internal-`.") + if actual_name and not actual_name.startswith("internal-"): + issues.append("Repository-internal skill `name` must start with `internal-`.") + if internal_identifier and internal_identifier.startswith("internal-") and actual_name and actual_name != internal_identifier: + issues.append(f"Repository-internal skill `name` should match directory name `{internal_identifier}`.") return issues @@ -1446,14 +1446,14 @@ def validate_unmanaged_agent_asset(target_root: Path, relative_path: str, repo_l issues.append("Missing `## Restrictions` section.") if repo_local: - local_identifier = local_asset_identifier(relative_path) + internal_identifier = internal_asset_identifier(relative_path) actual_name = frontmatter.get("name", "") - if not is_local_asset_path(relative_path): - issues.append("Repository-local agent filename must start with `local-`.") - if actual_name and not actual_name.startswith("local-"): - issues.append("Repository-local agent `name` must start with `local-`.") - if local_identifier and local_identifier.startswith("local-") and actual_name and actual_name != local_identifier: - issues.append(f"Repository-local agent `name` should match filename stem `{local_identifier}`.") + if not is_internal_asset_path(relative_path): + issues.append("Repository-internal agent filename must start with `internal-`.") + if actual_name and not actual_name.startswith("internal-"): + issues.append("Repository-internal agent `name` must start with `internal-`.") + if internal_identifier and internal_identifier.startswith("internal-") and actual_name and actual_name != internal_identifier: + issues.append(f"Repository-internal agent `name` should match filename stem `{internal_identifier}`.") return issues @@ -1506,8 +1506,8 @@ def detect_unmanaged_target_asset_issues( validation_issues = validator(target_root, relative_path, repo_local=repo_local) if validation_issues: issue_types.append("validation") - if any(detail.startswith("Repository-local ") for detail in validation_issues): - issue_types.append("local_naming") + if any(detail.startswith("Repository-internal ") for detail in validation_issues): + issue_types.append("internal_naming") details.extend(validation_issues) if not issue_types: @@ -1807,8 +1807,8 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, "- Use GitHub Copilot terminology in repository-facing content.", "- Do not mention internal runtime names in repository artifacts.", "- Treat prompt frontmatter `name:` as the canonical command identifier.", - "- Repository-local prompt, skill, and agent filenames must start with `local-`.", - "- Repository-local prompt, skill, and agent `name:` values must also start with `local-`.", + "- Repository-internal prompt, skill, and agent filenames must start with `internal-`.", + "- Repository-internal prompt, skill, and agent `name:` values must also start with `internal-`.", "", "## Decision Priority", "1. Apply repository non-negotiables from `.github/copilot-instructions.md`.", @@ -1864,7 +1864,7 @@ def render_agents_markdown(analysis: TargetAnalysis, selection: AssetSelection, [ "", "## Repository Inventory (Auto-generated)", - "This inventory reflects the desired managed baseline plus repository-local Copilot assets already present in the target repository.", + "This inventory reflects the desired managed baseline plus repository-owned internal Copilot assets already present in the target repository.", "", "### Instructions", ] @@ -2002,13 +2002,13 @@ def build_recommendations( f"{', '.join(validation_issue_paths)}." ) - local_naming_issue_paths = [ - issue.target_relative_path for issue in target_asset_issues if "local_naming" in issue.issue_types + internal_naming_issue_paths = [ + issue.target_relative_path for issue in target_asset_issues if "internal_naming" in issue.issue_types ] - if local_naming_issue_paths: + if internal_naming_issue_paths: recommendations["missing consumer-facing validation or onboarding guidance"].append( - "Repository-local Copilot prompts, skills, and agents should use `local-*` in both filenames and " - f"`name:` values: {', '.join(local_naming_issue_paths)}." + "Repository-internal Copilot prompts, skills, and agents should use `internal-*` in both filenames and " + f"`name:` values: {', '.join(internal_naming_issue_paths)}." ) if not target_has_validation_workflow(analysis.repo_root): diff --git a/.github/scripts/validate-copilot-customizations.sh b/.github/scripts/validate-copilot-customizations.sh index 72b55d8..aaf4dc2 100755 --- a/.github/scripts/validate-copilot-customizations.sh +++ b/.github/scripts/validate-copilot-customizations.sh @@ -351,7 +351,7 @@ tech_ai_prompt_name() { printf '%s' "$output" } -local_asset_identifier() { +internal_asset_identifier() { local file="$1" local base local parent @@ -379,30 +379,30 @@ validate_repo_local_prompt_naming() { local file="$1" local severity="error" local actual_name="" - local expected_local_name="" + local expected_internal_name="" [[ "$MODE" == "legacy-compatible" ]] && severity="warn" actual_name="$(frontmatter_value "$file" "name")" - expected_local_name="$(local_asset_identifier "$file" || true)" + expected_internal_name="$(internal_asset_identifier "$file" || true)" case "$(basename "$file")" in tech-ai-*.prompt.md) return 0 ;; - local-*.prompt.md) - if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then - record_issue "$severity" "Repository-local prompt name must start with 'local-': ${file}" + internal-*.prompt.md) + if [[ -n "$actual_name" && "$actual_name" != internal-* ]]; then + record_issue "$severity" "Repository-internal prompt name must start with 'internal-': ${file}" fi - if [[ -n "$actual_name" && -n "$expected_local_name" && "$actual_name" != "$expected_local_name" ]]; then - record_issue "$severity" "Repository-local prompt name must match filename stem '${expected_local_name}': ${file}" + if [[ -n "$actual_name" && -n "$expected_internal_name" && "$actual_name" != "$expected_internal_name" ]]; then + record_issue "$severity" "Repository-internal prompt name must match filename stem '${expected_internal_name}': ${file}" fi return 0 ;; *) - record_issue "$severity" "Repository-local prompt filename must start with 'local-': ${file}" - if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then - record_issue "$severity" "Repository-local prompt name must start with 'local-': ${file}" + record_issue "$severity" "Repository-internal prompt filename must start with 'internal-': ${file}" + if [[ -n "$actual_name" && "$actual_name" != internal-* ]]; then + record_issue "$severity" "Repository-internal prompt name must start with 'internal-': ${file}" fi ;; esac @@ -412,32 +412,32 @@ validate_repo_local_skill_naming() { local file="$1" local severity="error" local actual_name="" - local expected_local_name="" + local expected_internal_name="" local skill_dir [[ "$MODE" == "legacy-compatible" ]] && severity="warn" actual_name="$(frontmatter_value "$file" "name")" - expected_local_name="$(local_asset_identifier "$file" || true)" + expected_internal_name="$(internal_asset_identifier "$file" || true)" skill_dir="$(basename "$(dirname "$file")")" case "$skill_dir" in tech-ai-*) return 0 ;; - local-*) - if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then - record_issue "$severity" "Repository-local skill name must start with 'local-': ${file}" + internal-*) + if [[ -n "$actual_name" && "$actual_name" != internal-* ]]; then + record_issue "$severity" "Repository-internal skill name must start with 'internal-': ${file}" fi - if [[ -n "$actual_name" && -n "$expected_local_name" && "$actual_name" != "$expected_local_name" ]]; then - record_issue "$severity" "Repository-local skill name must match directory name '${expected_local_name}': ${file}" + if [[ -n "$actual_name" && -n "$expected_internal_name" && "$actual_name" != "$expected_internal_name" ]]; then + record_issue "$severity" "Repository-internal skill name must match directory name '${expected_internal_name}': ${file}" fi return 0 ;; *) - record_issue "$severity" "Repository-local skill directory must start with 'local-': ${file}" - if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then - record_issue "$severity" "Repository-local skill name must start with 'local-': ${file}" + record_issue "$severity" "Repository-internal skill directory must start with 'internal-': ${file}" + if [[ -n "$actual_name" && "$actual_name" != internal-* ]]; then + record_issue "$severity" "Repository-internal skill name must start with 'internal-': ${file}" fi ;; esac @@ -447,30 +447,30 @@ validate_repo_local_agent_naming() { local file="$1" local severity="error" local actual_name="" - local expected_local_name="" + local expected_internal_name="" [[ "$MODE" == "legacy-compatible" ]] && severity="warn" actual_name="$(frontmatter_value "$file" "name")" - expected_local_name="$(local_asset_identifier "$file" || true)" + expected_internal_name="$(internal_asset_identifier "$file" || true)" case "$(basename "$file")" in tech-ai-*.agent.md) return 0 ;; - local-*.agent.md) - if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then - record_issue "$severity" "Repository-local agent name must start with 'local-': ${file}" + internal-*.agent.md) + if [[ -n "$actual_name" && "$actual_name" != internal-* ]]; then + record_issue "$severity" "Repository-internal agent name must start with 'internal-': ${file}" fi - if [[ -n "$actual_name" && -n "$expected_local_name" && "$actual_name" != "$expected_local_name" ]]; then - record_issue "$severity" "Repository-local agent name must match filename stem '${expected_local_name}': ${file}" + if [[ -n "$actual_name" && -n "$expected_internal_name" && "$actual_name" != "$expected_internal_name" ]]; then + record_issue "$severity" "Repository-internal agent name must match filename stem '${expected_internal_name}': ${file}" fi return 0 ;; *) - record_issue "$severity" "Repository-local agent filename must start with 'local-': ${file}" - if [[ -n "$actual_name" && "$actual_name" != local-* ]]; then - record_issue "$severity" "Repository-local agent name must start with 'local-': ${file}" + record_issue "$severity" "Repository-internal agent filename must start with 'internal-': ${file}" + if [[ -n "$actual_name" && "$actual_name" != internal-* ]]; then + record_issue "$severity" "Repository-internal agent name must start with 'internal-': ${file}" fi ;; esac diff --git a/.github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md b/.github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md new file mode 100644 index 0000000..f336c29 --- /dev/null +++ b/.github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md @@ -0,0 +1,57 @@ +--- +name: TechAIInternalCopilotCustomizationBuilder +description: Create or update repository-owned internal GitHub Copilot customization assets in a consumer repository while preserving the shared synced baseline. +--- + +# TechAI Internal Copilot Customization Builder Skill + +## When to use +- Create or update repository-owned `internal-*` prompts, skills, agents, or `AGENTS.md` wiring in a consumer repo. +- Extend a consumer repo with Copilot behavior that should stay local instead of entering the shared `tech-ai-*` baseline. +- Clean up or normalize existing target-internal Copilot assets so they follow current naming, frontmatter, and inventory rules. + +## Workflow +1. Inspect the target repository layout, `.github` contents, root `AGENTS.md`, git state, and existing internal Copilot assets. +2. Identify at least one representative target file for each requested internal capability and extract the actual schema field names, naming patterns, identity formats, and validation commands from those files before drafting any `internal-*` asset. +3. Confirm the baseline is current enough for internal customization work: + - if `copilot-instructions.md`, the validator script, or expected synced assets are missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first; + - use the sync report to avoid creating an `internal-*` asset that duplicates an available shared baseline capability. +4. Decide the narrowest asset type that solves the request: + - create or update an `internal-*.prompt.md` when the behavior is mostly task instructions; + - add an `internal-*` skill only when the workflow needs reusable implementation detail beyond the prompt; + - add an `internal-*` agent only when the repo needs durable routing or persona guidance that cannot stay in a prompt or skill. +5. Enforce internal naming and ownership rules: + - filenames for repo-owned prompts, skills, and agents must start with `internal-`; + - prompt, skill, and agent frontmatter `name:` values must also start with `internal-`; + - keep internal assets in `.github/prompts`, `.github/skills//SKILL.md`, and `.github/agents`. +6. Ground the internal asset content on target-repository evidence: + - derive prompt examples, schema snippets, naming rules, and validation commands from real target files instead of generic or remembered patterns; + - do not invent fields, object shapes, identity suffixes, or naming conventions that are not present in the inspected files; + - if multiple target patterns exist, narrow the prompt scope to the chosen pattern and cite the representative files; + - if no stable target pattern exists, stop and report the missing grounding rather than authoring ambiguous internal guidance. +7. Keep internal assets minimal and compatible with the shared baseline: + - reuse `.github/instructions/*.instructions.md` and synced `tech-ai-*` skills when possible instead of copying large guidance blocks; + - reference shared prompts or skills by path when they already cover most of the behavior; + - avoid creating an internal canonical duplicate of an existing `tech-ai-*` capability. +8. Update `AGENTS.md` in the target repository: + - keep explicit `.github/...` paths; + - add the internal assets to the inventory; + - adjust routing or preferred prompts or skills only when the new internal capability should be discoverable by default; + - avoid duplicating long prompt or skill descriptions inside `AGENTS.md`. +9. Validate the target repository after changes: + - run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict`; + - run stack-specific checks for touched Bash, Python, JSON, YAML, Markdown, or Terraform assets; + - if the repo has a synced baseline manifest, preserve it and do not rewrite managed files opportunistically. +10. Report the result with the concrete target files used for grounding, changed files, validation output, residual target-repository risks, and whether the new capability should stay internal or be proposed for promotion into the shared source baseline. + +## Scope rules +- Manage consumer-repository Copilot assets only. +- Keep source-repository assets and shared baseline definitions unchanged unless promotion is explicitly requested. +- Prefer one internal capability per repo-specific workflow; consolidate or deprecate duplicates instead of multiplying near-identical internal prompts. +- Do not create internal copies of source-only repo agents such as `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`, or `TechAISyncCopilotConfigs`. + +## Validation +- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` in the target repo after internal customization changes. +- Run `bash -n` and `shellcheck -s bash` for changed Bash files when available. +- Run `python -m compileall ` and relevant `pytest` checks for changed Python files. +- Re-run `TechAISyncCopilotConfigs` in `plan` mode when you need to confirm that the new internal assets remain clearly separated from the managed shared baseline. diff --git a/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md b/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md deleted file mode 100644 index 9a2ee3f..0000000 --- a/.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -name: TechAILocalCopilotCustomizationBuilder -description: Create or update repository-owned local GitHub Copilot customization assets in a consumer repository while preserving the shared synced baseline. ---- - -# TechAI Local Copilot Customization Builder Skill - -## When to use -- Create or update repository-owned `local-*` prompts, skills, agents, or `AGENTS.md` wiring in a consumer repo. -- Extend a consumer repo with Copilot behavior that should stay local instead of entering the shared `tech-ai-*` baseline. -- Clean up or normalize existing target-local Copilot assets so they follow current naming, frontmatter, and inventory rules. - -## Workflow -1. Inspect the target repository layout, `.github` contents, root `AGENTS.md`, git state, and existing local Copilot assets. -2. Identify at least one representative target file for each requested local capability and extract the actual schema field names, naming patterns, identity formats, and validation commands from those files before drafting any `local-*` asset. -3. Confirm the baseline is current enough for local customization work: - - if `copilot-instructions.md`, the validator script, or expected synced assets are missing or stale, run `TechAISyncCopilotConfigs` in `plan` mode first; - - use the sync report to avoid creating a `local-*` asset that duplicates an available shared baseline capability. -4. Decide the narrowest asset type that solves the request: - - create or update a `local-*.prompt.md` when the behavior is mostly task instructions; - - add a `local-*` skill only when the workflow needs reusable implementation detail beyond the prompt; - - add a `local-*` agent only when the repo needs durable routing or persona guidance that cannot stay in a prompt or skill. -5. Enforce local naming and ownership rules: - - filenames for repo-owned prompts, skills, and agents must start with `local-`; - - prompt, skill, and agent frontmatter `name:` values must also start with `local-`; - - keep local assets in `.github/prompts`, `.github/skills//SKILL.md`, and `.github/agents`. -6. Ground the local asset content on target-repository evidence: - - derive prompt examples, schema snippets, naming rules, and validation commands from real target files instead of generic or remembered patterns; - - do not invent fields, object shapes, identity suffixes, or naming conventions that are not present in the inspected files; - - if multiple target patterns exist, narrow the prompt scope to the chosen pattern and cite the representative files; - - if no stable target pattern exists, stop and report the missing grounding rather than authoring ambiguous local guidance. -7. Keep local assets minimal and compatible with the shared baseline: - - reuse `.github/instructions/*.instructions.md` and synced `tech-ai-*` skills when possible instead of copying large guidance blocks; - - reference shared prompts or skills by path when they already cover most of the behavior; - - avoid creating a local canonical duplicate of an existing `tech-ai-*` capability. -8. Update `AGENTS.md` in the target repository: - - keep explicit `.github/...` paths; - - add the local assets to the inventory; - - adjust routing or preferred prompts or skills only when the new local capability should be discoverable by default; - - avoid duplicating long prompt or skill descriptions inside `AGENTS.md`. -9. Validate the target repository after changes: - - run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict`; - - run stack-specific checks for touched Bash, Python, JSON, YAML, Markdown, or Terraform assets; - - if the repo has a synced baseline manifest, preserve it and do not rewrite managed files opportunistically. -10. Report the result with the concrete target files used for grounding, changed files, validation output, residual repo-local risks, and whether the new capability should stay local or be proposed for promotion into the shared source baseline. - -## Scope rules -- Manage consumer-repository Copilot assets only. -- Keep source-repository assets and shared baseline definitions unchanged unless promotion is explicitly requested. -- Prefer one local capability per repo-specific workflow; consolidate or deprecate duplicates instead of multiplying near-identical local prompts. -- Do not create local copies of source-only repo agents such as `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`, or `TechAISyncCopilotConfigs`. - -## Validation -- Run `bash .github/scripts/validate-copilot-customizations.sh --scope root --mode strict` in the target repo after local customization changes. -- Run `bash -n` and `shellcheck -s bash` for changed Bash files when available. -- Run `python -m compileall ` and relevant `pytest` checks for changed Python files. -- Re-run `TechAISyncCopilotConfigs` in `plan` mode when you need to confirm that the new local assets remain clearly separated from the managed shared baseline. diff --git a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md index cee2632..c4c6ac5 100644 --- a/.github/skills/tech-ai-sync-copilot-configs/SKILL.md +++ b/.github/skills/tech-ai-sync-copilot-configs/SKILL.md @@ -23,7 +23,7 @@ description: Analyze a local repository, select the minimum Copilot customizatio 6. Detect redundant legacy aliases for canonical prompt/skill/agent families, including families that are not part of the selected minimum baseline, so `cs-*` and unprefixed leftovers still appear in the plan. 7. Audit target-local instructions, prompts, skills, and agents that are outside the selected sync baseline: - report strict validation gaps for unmanaged files; - - require repository-owned prompts, skills, and agents to use `local-*` in both filenames and `name:` values so they are visually distinct from synced global assets; + - require repository-owned prompts, skills, and agents to use `internal-*` in both filenames and `name:` values so they are visually distinct from synced global assets; - report legacy aliases even when the canonical family is not selected; - keep target-only custom assets visible instead of silently omitting them. 8. Treat redundant canonical-vs-legacy overlaps as conflicts instead of silently creating duplicate configuration families. diff --git a/AGENTS.md b/AGENTS.md index ba2cf8b..7813c14 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,8 +9,8 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Treat prompt frontmatter `name:` as the canonical command identifier. - Canonical repository-owned prompt, agent, and instruction filenames should use the `tech-ai-` prefix when introduced or renamed. - Canonical prompt, skill, and agent `name:` values should use the `TechAI` prefix. -- Repository-local prompt, skill, and agent filenames in consumer repositories should use the `local-` prefix. -- Repository-local prompt, skill, and agent `name:` values in consumer repositories should also use the `local-` prefix. +- Repository-owned prompt, skill, and agent filenames in consumer repositories should use the `internal-` prefix. +- Repository-owned prompt, skill, and agent `name:` values in consumer repositories should also use the `internal-` prefix. - Reserve the `TechAIGlobal` prefix only for repo-only agents that encode standards for this global configuration repository. - The canonical project-owned `AGENTS.md` file must live in repository root as `AGENTS.md`. - Keep legacy aliases only when required for backward compatibility, and prefer canonical `tech-ai-*` assets in docs, examples, and sync selection. @@ -37,9 +37,9 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Use `TechAIGlobalCustomizationAuditor` as the final quality gate for GitHub Copilot customization changes in this repository. - Use `TechAICustomizationAuditor` only as a deprecated compatibility alias while older references are migrated. - Use `TechAISyncCopilotConfigs` for cross-repository Copilot-core alignment and source or target redundancy audits. -- Use `TechAILocalCopilotCustomizationBuilder` when a consumer repository needs repo-owned `local-*` prompts, skills, agents, or `AGENTS.md` wiring that should remain local instead of entering the shared baseline. +- Use `TechAIInternalCopilotCustomizationBuilder` when a consumer repository needs repo-owned `internal-*` prompts, skills, agents, or `AGENTS.md` wiring that should remain internal instead of entering the shared baseline. - Use specialist agents (`TechAIWorkflowSupplyChain`, `TechAISecurityReviewer`, `TechAITerraformGuardrails`, `TechAIIAMLeastPrivilege`, `TechAIPRWriter`) only when their domain matches the task. -- The `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`, and `TechAILocalCopilotCustomizationBuilder` agents are repo-only and must not be synced to consumer repositories. +- The `TechAIGlobalCustomizationBuilder`, `TechAIGlobalCustomizationAuditor`, and `TechAIInternalCopilotCustomizationBuilder` agents are repo-only and must not be synced to consumer repositories. ### Anti-patterns @@ -50,15 +50,15 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Do not use generic `TechAIReviewer` when you need exhaustive per-language nit-level review; use `TechAIScriptReviewer` instead. - Do not use `TechAICustomizationAuditor` for new work; use `TechAIGlobalCustomizationAuditor`. - Do not use `TechAIImplementer` alone when the task is cross-repository Copilot configuration alignment; use `TechAISyncCopilotConfigs`. -- Do not use `TechAISyncCopilotConfigs` alone when the task is to author new repository-owned `local-*` assets in a consumer repository; use `TechAILocalCopilotCustomizationBuilder` after baseline alignment. -- Do not use `TechAILocalCopilotCustomizationBuilder` to add new shared `tech-ai-*` assets in this standards repository; use `TechAIGlobalCustomizationBuilder`. +- Do not use `TechAISyncCopilotConfigs` alone when the task is to author new repository-owned `internal-*` assets in a consumer repository; use `TechAIInternalCopilotCustomizationBuilder` after baseline alignment. +- Do not use `TechAIInternalCopilotCustomizationBuilder` to add new shared `tech-ai-*` assets in this standards repository; use `TechAIGlobalCustomizationBuilder`. ### Composition and Handoffs - For changes spanning multiple specialist domains, run each relevant specialist and aggregate findings. - The standard chain for non-trivial work is `TechAIPlanner` -> `TechAIImplementer` -> `TechAIReviewer` or a matching specialist. - For GitHub Copilot customization changes in this repository, use `TechAIGlobalCustomizationBuilder` first and `TechAIGlobalCustomizationAuditor` before final handoff. -- For consumer-local Copilot customization work, use `TechAISyncCopilotConfigs` first if the target baseline is unknown, then use `TechAILocalCopilotCustomizationBuilder` for repo-owned `local-*` assets. +- For consumer-repository Copilot customization work, use `TechAISyncCopilotConfigs` first if the target baseline is unknown, then use `TechAIInternalCopilotCustomizationBuilder` for repo-owned `internal-*` assets. - `TechAIPlanner` output is input context for `TechAIImplementer`. - `TechAIImplementer` output is input context for `TechAIReviewer`. - `TechAIReviewer` findings flagged as `Critical` or `Major` route back to `TechAIImplementer` for remediation. @@ -131,7 +131,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `TechAICodeReview`: exhaustive, nit-level code review. - `TechAIGitHubAction`: GitHub Actions workflow authoring. -- `TechAILocalCopilotCustomizationBuilder`: consumer-local `local-*` customization authoring. +- `TechAIInternalCopilotCustomizationBuilder`: consumer-repository `internal-*` customization authoring. - `TechAISyncCopilotConfigs`: cross-repository alignment and redundancy analysis. - `TechAIPRDescription`: pull request body generation. - `TechAIAddUnitTests`: test authoring and improvement. @@ -141,7 +141,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `TechAICodeReview`: strict review workflow and anti-pattern catalog. - `TechAICICDWorkflow`: CI or CD workflow design patterns. -- `TechAILocalCopilotCustomizationBuilder`: consumer-local Copilot customization workflow. +- `TechAIInternalCopilotCustomizationBuilder`: consumer-repository Copilot customization workflow. - `TechAISyncCopilotConfigs`: deterministic sync planning and reporting. - `TechAIPRWriting`: PR writing conventions aligned to the repository template. - `TechAICloudPolicy`: reusable cloud policy authoring patterns. @@ -186,7 +186,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/prompts/tech-ai-github-composite-action.prompt.md` - `.github/prompts/tech-ai-github-pr-description.prompt.md` - `.github/prompts/tech-ai-java.prompt.md` -- `.github/prompts/tech-ai-local-copilot-customization-builder.prompt.md` +- `.github/prompts/tech-ai-internal-copilot-customization-builder.prompt.md` - `.github/prompts/tech-ai-nodejs.prompt.md` - `.github/prompts/tech-ai-python-script.prompt.md` - `.github/prompts/tech-ai-python.prompt.md` @@ -201,7 +201,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/skills/tech-ai-code-review/SKILL.md` - `.github/skills/tech-ai-composite-action/SKILL.md` - `.github/skills/tech-ai-data-registry/SKILL.md` -- `.github/skills/tech-ai-local-copilot-customization-builder/SKILL.md` +- `.github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md` - `.github/skills/tech-ai-pr-writing/SKILL.md` - `.github/skills/tech-ai-project-java/SKILL.md` - `.github/skills/tech-ai-project-nodejs/SKILL.md` @@ -219,7 +219,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/agents/tech-ai-github-workflow-supply-chain.agent.md` - `.github/agents/tech-ai-global-customization-auditor.agent.md` - `.github/agents/tech-ai-global-customization-builder.agent.md` -- `.github/agents/tech-ai-local-copilot-customization-builder.agent.md` +- `.github/agents/tech-ai-internal-copilot-customization-builder.agent.md` - `.github/agents/tech-ai-iam-least-privilege.agent.md` - `.github/agents/tech-ai-implementer.agent.md` - `.github/agents/tech-ai-planner.agent.md` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index db7b6aa..6edab8d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,7 +7,7 @@ This repository is the source baseline for reusable GitHub Copilot customization - Canonical source-owned instructions, prompts, skills, and agents use the `tech-ai-` filename prefix. - Canonical prompt, skill, and agent `name:` values use the `TechAI` prefix. - Repo-only standards agents use the `TechAIGlobal` prefix and must stay source-only. -- Consumer-local assets use the `local-` prefix in both filenames and `name:` values. +- Consumer-repository assets use the `internal-` prefix in both filenames and `name:` values. ## Adding or updating assets - Instructions: keep frontmatter `description` and `applyTo`, use repository-agnostic wording, and avoid stack duplication already covered by a skill. diff --git a/tests/test_tech_ai_sync_copilot_configs.py b/tests/test_tech_ai_sync_copilot_configs.py index 96fe755..fe71cac 100644 --- a/tests/test_tech_ai_sync_copilot_configs.py +++ b/tests/test_tech_ai_sync_copilot_configs.py @@ -283,11 +283,11 @@ def test_build_plan_reports_unmanaged_target_assets_and_legacy_aliases_outside_s ), ) write_file( - target_root / ".github" / "skills" / "local-registry" / "SKILL.md", + target_root / ".github" / "skills" / "internal-registry" / "SKILL.md", "\n".join( [ "---", - "name: local-registry", + "name: internal-registry", "description: Custom local registry helper.", "---", "", @@ -309,7 +309,7 @@ def test_build_plan_reports_unmanaged_target_assets_and_legacy_aliases_outside_s agents_file = next(item for item in planned_files if item.target_relative_path == "AGENTS.md") assert ".github/prompts/add-external-user.prompt.md" in plan.analysis.target_only_assets["prompts"] - assert ".github/skills/local-registry/SKILL.md" in plan.analysis.target_only_assets["skills"] + assert ".github/skills/internal-registry/SKILL.md" in plan.analysis.target_only_assets["skills"] assert ".github/prompts/cs-data-registry.prompt.md" not in plan.analysis.target_only_assets["prompts"] assert ".github/skills/data-registry/SKILL.md" not in plan.analysis.target_only_assets["skills"] @@ -325,10 +325,10 @@ def test_build_plan_reports_unmanaged_target_assets_and_legacy_aliases_outside_s ) assert "validation" in issue_by_path[".github/prompts/add-external-user.prompt.md"].issue_types - assert "local_naming" in issue_by_path[".github/prompts/add-external-user.prompt.md"].issue_types + assert "internal_naming" in issue_by_path[".github/prompts/add-external-user.prompt.md"].issue_types assert "Missing frontmatter key `name`." in issue_by_path[".github/prompts/add-external-user.prompt.md"].details assert ( - "Repository-local prompt filename must start with `local-`." + "Repository-internal prompt filename must start with `internal-`." in issue_by_path[".github/prompts/add-external-user.prompt.md"].details ) assert "legacy_alias" in issue_by_path[".github/prompts/cs-data-registry.prompt.md"].issue_types @@ -336,23 +336,23 @@ def test_build_plan_reports_unmanaged_target_assets_and_legacy_aliases_outside_s issue_by_path[".github/prompts/cs-data-registry.prompt.md"].canonical_source_path == ".github/prompts/tech-ai-data-registry.prompt.md" ) - assert ".github/skills/local-registry/SKILL.md" not in issue_by_path + assert ".github/skills/internal-registry/SKILL.md" not in issue_by_path assert ".github/prompts/add-external-user.prompt.md" in agents_file.desired_content assert ".github/prompts/cs-data-registry.prompt.md" in agents_file.desired_content assert ".github/skills/data-registry/SKILL.md" in agents_file.desired_content - assert ".github/skills/local-registry/SKILL.md" in agents_file.desired_content + assert ".github/skills/internal-registry/SKILL.md" in agents_file.desired_content -def test_build_plan_accepts_local_prefixed_repo_owned_assets(tmp_path: Path) -> None: - target_root = tmp_path / "local-assets" +def test_build_plan_accepts_internal_prefixed_repo_owned_assets(tmp_path: Path) -> None: + target_root = tmp_path / "internal-assets" build_python_service_target(target_root) write_file( - target_root / ".github" / "prompts" / "local-add-external-user.prompt.md", + target_root / ".github" / "prompts" / "internal-add-external-user.prompt.md", "\n".join( [ "---", - "name: local-add-external-user", + "name: internal-add-external-user", "description: Add an external user to Entra ID.", "agent: agent", "argument-hint: user=", @@ -361,7 +361,7 @@ def test_build_plan_accepts_local_prefixed_repo_owned_assets(tmp_path: Path) -> "# Local Add External User", "", "## Instructions", - "1. Use `.github/skills/local-entra-access/SKILL.md`.", + "1. Use `.github/skills/internal-entra-access/SKILL.md`.", "", "## Validation", "- Validate the repository-local registry update.", @@ -373,12 +373,12 @@ def test_build_plan_accepts_local_prefixed_repo_owned_assets(tmp_path: Path) -> ), ) write_file( - target_root / ".github" / "skills" / "local-entra-access" / "SKILL.md", + target_root / ".github" / "skills" / "internal-entra-access" / "SKILL.md", "\n".join( [ "---", - "name: local-entra-access", - "description: Repository-local Entra access workflow.", + "name: internal-entra-access", + "description: Repository-internal Entra access workflow.", "---", "", "# Local Entra Access", @@ -396,8 +396,8 @@ def test_build_plan_accepts_local_prefixed_repo_owned_assets(tmp_path: Path) -> plan, _planned_files = MODULE.build_plan(REPO_ROOT, target_root) issue_paths = {issue.target_relative_path for issue in plan.target_asset_issues} - assert ".github/prompts/local-add-external-user.prompt.md" not in issue_paths - assert ".github/skills/local-entra-access/SKILL.md" not in issue_paths + assert ".github/prompts/internal-add-external-user.prompt.md" not in issue_paths + assert ".github/skills/internal-entra-access/SKILL.md" not in issue_paths def test_apply_plan_writes_manifest_and_managed_files(tmp_path: Path) -> None: @@ -588,17 +588,17 @@ def test_build_plan_excludes_repo_only_global_customization_agents_from_consumer assert ".github/agents/tech-ai-global-customization-auditor.agent.md" not in plan.selection.agents -def test_local_builder_triads_are_source_only_and_excluded_from_consumer_sync() -> None: +def test_internal_builder_triads_are_source_only_and_excluded_from_consumer_sync() -> None: assert ( - ".github/agents/tech-ai-local-copilot-customization-builder.agent.md" + ".github/agents/tech-ai-internal-copilot-customization-builder.agent.md" in MODULE.SOURCE_ONLY_AGENT_PATHS ) assert ( - ".github/prompts/tech-ai-local-copilot-customization-builder.prompt.md" + ".github/prompts/tech-ai-internal-copilot-customization-builder.prompt.md" in MODULE.SOURCE_ONLY_PROMPT_PATHS ) assert ( - ".github/skills/tech-ai-local-copilot-customization-builder/SKILL.md" + ".github/skills/tech-ai-internal-copilot-customization-builder/SKILL.md" in MODULE.SOURCE_ONLY_SKILL_PATHS ) diff --git a/tests/test_tech_ai_validate_copilot_customizations.py b/tests/test_tech_ai_validate_copilot_customizations.py index 96aaa72..1dc1e98 100644 --- a/tests/test_tech_ai_validate_copilot_customizations.py +++ b/tests/test_tech_ai_validate_copilot_customizations.py @@ -75,8 +75,8 @@ def test_tech_ai_validator_reports_missing_prompt_argument_hint(tmp_path: Path) assert any("Missing frontmatter key 'argument-hint'" in message for message in messages) -def test_tech_ai_validator_requires_local_prefix_for_repo_owned_assets(tmp_path: Path) -> None: - target_root = tmp_path / "invalid-local-assets" +def test_tech_ai_validator_requires_internal_prefix_for_repo_owned_assets(tmp_path: Path) -> None: + target_root = tmp_path / "invalid-internal-assets" copy_copilot_config(target_root) (target_root / ".github" / "prompts").mkdir(parents=True, exist_ok=True) @@ -112,7 +112,7 @@ def test_tech_ai_validator_requires_local_prefix_for_repo_owned_assets(tmp_path: [ "---", "name: user-admin", - "description: Repository-local user administration workflow.", + "description: Repository-internal user administration workflow.", "---", "", "# User Admin", @@ -129,17 +129,17 @@ def test_tech_ai_validator_requires_local_prefix_for_repo_owned_assets(tmp_path: encoding="utf-8", ) - report_file = tmp_path / "tech-ai-validator-local-prefix.json" + report_file = tmp_path / "tech-ai-validator-internal-prefix.json" result = run_validator(target_root, report_file) payload = json.loads(report_file.read_text(encoding="utf-8")) messages = [finding["message"] for finding in payload["findings"]] assert result.returncode == 1 assert payload["status"] == "failed" - assert any("Repository-local prompt filename must start with 'local-'" in message for message in messages) - assert any("Repository-local prompt name must start with 'local-'" in message for message in messages) - assert any("Repository-local skill directory must start with 'local-'" in message for message in messages) - assert any("Repository-local skill name must start with 'local-'" in message for message in messages) + assert any("Repository-internal prompt filename must start with 'internal-'" in message for message in messages) + assert any("Repository-internal prompt name must start with 'internal-'" in message for message in messages) + assert any("Repository-internal skill directory must start with 'internal-'" in message for message in messages) + assert any("Repository-internal skill name must start with 'internal-'" in message for message in messages) def test_tech_ai_validator_requires_root_agents_file(tmp_path: Path) -> None: @@ -271,7 +271,7 @@ def test_root_agents_routes_customization_work_to_global_and_local_customization assert "TechAIGlobalCustomizationBuilder" in agents_text assert "TechAIGlobalCustomizationAuditor" in agents_text - assert "TechAILocalCopilotCustomizationBuilder" in agents_text + assert "TechAIInternalCopilotCustomizationBuilder" in agents_text assert "repo-only" in agents_text assert "## Available Skills" not in agents_text assert "## Available Prompts" not in agents_text @@ -295,15 +295,15 @@ def test_global_builder_maps_consolidated_rules_and_legacy_auditor_is_deprecated assert "TechAIGlobalCustomizationAuditor" in legacy_auditor_text -def test_local_builder_requires_grounding_against_concrete_target_files() -> None: +def test_internal_builder_requires_grounding_against_concrete_target_files() -> None: agent_text = ( - REPO_ROOT / ".github" / "agents" / "tech-ai-local-copilot-customization-builder.agent.md" + REPO_ROOT / ".github" / "agents" / "tech-ai-internal-copilot-customization-builder.agent.md" ).read_text(encoding="utf-8") prompt_text = ( - REPO_ROOT / ".github" / "prompts" / "tech-ai-local-copilot-customization-builder.prompt.md" + REPO_ROOT / ".github" / "prompts" / "tech-ai-internal-copilot-customization-builder.prompt.md" ).read_text(encoding="utf-8") skill_text = ( - REPO_ROOT / ".github" / "skills" / "tech-ai-local-copilot-customization-builder" / "SKILL.md" + REPO_ROOT / ".github" / "skills" / "tech-ai-internal-copilot-customization-builder" / "SKILL.md" ).read_text(encoding="utf-8") assert "inspect concrete target files first" in agent_text From e761ef85fb6db8324d8301fec801ffeb0d0279a4 Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos Date: Tue, 10 Mar 2026 12:54:36 +0100 Subject: [PATCH 8/9] feat: add TechAI Pair Architect agent and analysis prompt for deep change-impact analysis --- .../agents/tech-ai-pair-architect.agent.md | 264 ++++++++++++++++++ .../tech-ai-pair-architect-analysis.prompt.md | 83 ++++++ .../skills/tech-ai-pair-architect/SKILL.md | 246 ++++++++++++++++ AGENTS.md | 11 + 4 files changed, 604 insertions(+) create mode 100644 .github/agents/tech-ai-pair-architect.agent.md create mode 100644 .github/prompts/tech-ai-pair-architect-analysis.prompt.md create mode 100644 .github/skills/tech-ai-pair-architect/SKILL.md diff --git a/.github/agents/tech-ai-pair-architect.agent.md b/.github/agents/tech-ai-pair-architect.agent.md new file mode 100644 index 0000000..38e9c29 --- /dev/null +++ b/.github/agents/tech-ai-pair-architect.agent.md @@ -0,0 +1,264 @@ +--- +description: Perform deep change-impact analysis across repository modifications, generating a structured Markdown report with errors, improvements, doubts, blind spots, and architecture recommendations. +name: TechAIPairArchitect +tools: ["search", "usages", "problems", "editFiles", "runTerminal", "fetch"] +--- + +# TechAI Pair Architect Agent + +You are a senior principal engineer specialized in Domain-Driven Design, software architecture, and pragmatic business-oriented delivery. You think rigorously but always through the lens of real-world impact. + +## Persona and voice + +Channel the combined mindset of four engineering perspectives: + +- **Eric Evans** — Domain-Driven Design. Ask "Does this change respect bounded contexts and ubiquitous language?" Flag domain leakage, anemic models, and misplaced responsibilities. Business intent must be visible in the code. +- **Martin Fowler** — Architecture and refactoring. Ask "Is this the simplest thing that could possibly work, and is it telling a clear story?" Flag unnecessary complexity, tangled dependencies, and missing abstractions. +- **Gregor Hohpe** — Integration and systems thinking. Ask "How does this change affect the rest of the system, and what are the second-order consequences?" Flag hidden coupling, missing error boundaries, and integration risks. +- **Pragmatic Engineer** — Business pragmatism. Ask "Does this change deliver value proportional to its complexity? What is the operational cost?" Never recommend an improvement that costs more than the problem it solves. + +Tone: direct, respectful, and intellectually honest. Explain the *why* behind every finding. Teach through the analysis. Be opinionated but open to alternative approaches. Never be dismissive. + +## Objective + +Analyze all modifications in a repository change set (branch diff, PR, or set of changed files) and produce a comprehensive Markdown analysis report. The report must surface everything that a thorough human architect would catch during a deep review — and things they might miss. + +## Restrictions + +- Do not modify source code files unless explicitly requested. +- Do not run destructive commands. +- Base every finding on concrete evidence in the diff or repository context. +- Apply `security-baseline.md` controls as a minimum baseline. +- Keep all output in English. +- Write the report file in Markdown format. + +## Analysis scope + +### Auto-detection +- Detect all changed files from the current branch diff against the default branch. +- Auto-detect languages, frameworks, and infrastructure tools from file extensions and content. +- Load and apply all matching `instructions/*.instructions.md` files for detected languages. +- If a `.github/skills/tech-ai-code-review/SKILL.md` exists, use it as the anti-pattern reference. + +### Depth +- Go beyond line-level defects: analyze module boundaries, data flow, domain modeling, error propagation, configuration management, observability, testability, and deployment impact. +- Examine how changes interact with unchanged code in the immediate dependency graph. +- Consider temporal effects: will this change create problems in 3 months? 6 months? At scale? + +## Modes + +### Depth: full (default) +Analyze across all dimensions — correctness, DDD, architecture, blind spots, and lateral thinking. Include all report sections with maximum detail. Read dependency files beyond the immediate diff when needed. + +### Depth: quick +Focus on Errors and Blind Spots only. Skip the Architecture Advisory section. Limit analysis to changed files without dependency tracing. Useful for fast feedback loops during development. + +### Standard mode (default) +Analyze across all dimensions and produce the full report. + +### Devil's advocate mode (`mode=devil`) +In addition to the standard analysis, apply an adversarial thinking layer: +- Find at least 3 fundamental objections to the chosen design approach. +- Propose radically different alternatives with honest tradeoffs. +- Challenge assumptions that seem obvious. +- Add a dedicated `## 6. Devil's Advocate` section in the report. + +## Git history awareness + +Before analyzing the diff, gather project trajectory context: +1. Read recent commit history (`git log --oneline -20`) to understand direction. +2. Check for churn patterns, repeated similar changes, or reverted approaches. +3. Use this context in Blind Spots to flag misalignment with project trajectory. + +## Report generation + +### Default behavior +- Generate the report as a Markdown file at the repository root: `ANALYSIS_REPORT.md`. +- If the file already exists, overwrite it with the latest analysis. +- The report must be self-contained and readable without additional context. + +### Report structure + +The report MUST contain these sections in this exact order: + +```markdown +# Change Analysis Report + +> Generated: +> Branch: → +> Files analyzed: +> Languages detected: + +--- + +## Executive Summary +<2-4 sentences: what changed, overall assessment, top risk, and health score verdict> + +--- + +## 1. Errors and Defects + +Issues that are objectively wrong and must be fixed. + +### [ERROR-] +- **File**: <path>#L<line> +- **Severity**: Critical | Major +- **What is wrong**: <clear description> +- **Why it matters**: <business/technical impact> +- **How to fix**: <concrete suggestion with code snippet if applicable> + +--- + +## 2. Improvement Opportunities + +Things that work but can be done better. + +### [IMPROVE-<NNN>] <title> +- **File**: <path>#L<line> +- **Category**: Readability | Performance | Maintainability | Testability | DDD | Security +- **Current state**: <what exists now> +- **Suggested improvement**: <specific recommendation> +- **Why it is better**: <rationale with tradeoff analysis> +- **Effort**: Low | Medium | High + +--- + +## 3. Doubts and Open Questions + +Aspects that are ambiguous or where the intent is unclear. + +### [DOUBT-<NNN>] <title> +- **File**: <path>#L<line> +- **Question**: <specific question about the change> +- **Why it matters**: <what could go wrong if the assumption is incorrect> +- **Suggested clarification**: <what the author should verify or document> + +--- + +## 4. Blind Spots and Unconsidered Aspects + +Things the change does NOT address that it probably should, or second-order effects. + +### [BLIND-<NNN>] <title> +- **Area**: <domain, infrastructure, testing, operations, security, etc.> +- **What was not considered**: <description> +- **Potential consequence**: <what could happen> +- **Recommendation**: <what to investigate or add> + +--- + +## 5. Architecture and Best Practices (Advisory) + +> ⚠️ This section contains non-binding recommendations. They represent architectural guidance +> and best practices that the team should evaluate against their specific context and constraints. + +### 5.1 Domain Design Assessment +- **DDD alignment verdict**: <Aligned | Partially Aligned | Not Aligned | Not Applicable> +- **Bounded context observations**: <analysis> +- **Ubiquitous language consistency**: <analysis> +- **Aggregate/entity/value-object placement**: <observations> + +### 5.2 Architectural Recommendations +For each recommendation: +#### [ARCH-<NNN>] <title> +- **Current state**: <how things are now> +- **Recommended approach**: <what to consider> +- **Rationale**: <why this is a good practice with references> +- **Impact**: Low | Medium | High +- **Effort**: Low | Medium | High +- **Priority**: <suggested priority based on impact/effort ratio> + +### 5.3 Lateral Thinking — Outside-the-Box Observations +Things that a conventional review might miss. Think about: +- Second-order effects on other teams or services. +- Operational burden at scale (on-call, debugging, monitoring). +- Developer experience and onboarding friction. +- Alternative approaches that could simplify the entire problem space. +- Business model or product implications hidden in technical choices. +- Patterns emerging across multiple files that suggest a systemic issue. + +--- + +## 6. Devil's Advocate (only when mode=devil) + +### [DEVIL-<NNN>] <provocative question> +- **Assumption challenged**: <what the change takes for granted> +- **What if it is wrong**: <consequence> +- **Alternative approach**: <radically different solution> +- **Tradeoffs**: <pros and cons vs current approach> + +--- + +## 7. Risk Matrix + +Top findings placed on a probability/impact grid: + +``` + HIGH IMPACT + │ + URGENT │ PLAN + (fix before merge) │ (fix in next sprint) + │ + ──────────────────────┼────────────────────── + │ + MONITOR │ ACCEPT + (watch in prod) │ (low risk, defer) + │ + LOW IMPACT + HIGH PROBABILITY LOW PROBABILITY +``` + +Placement: `[<ID>] <title> → <quadrant>` + +--- + +## Summary Statistics + +| Category | Count | +|---|---| +| Errors and Defects | <count> | +| Improvement Opportunities | <count> | +| Doubts and Open Questions | <count> | +| Blind Spots | <count> | +| Architecture Recommendations | <count> | +| Devil's Advocate Challenges | <count> | + +**Health Score: <score>/100 — <verdict>** +``` + +## Analysis protocol + +1. **Gather context**: identify changed files, detect languages, load instruction files. Read recent git history for trajectory awareness. +2. **Map the change**: understand the intent behind the change set as a whole, not just individual files. +3. **Analyze layer by layer**: + - Correctness: does it do what it claims? + - Domain integrity: are DDD boundaries respected? + - Architecture: does it fit the broader system design? + - Operations: can this be debugged, monitored, and maintained? + - Security: does it follow least privilege and security baseline? + - Testing: is the change testable and tested? +4. **Apply lateral thinking**: step outside the immediate change and ask "what else?" +5. **Apply devil's advocate** (if `mode=devil`): challenge 3+ design assumptions with radical alternatives. +6. **Compute health score**: apply point deductions per finding type, determine verdict. +7. **Populate risk matrix**: place top findings on probability/impact grid. +8. **Write the report**: populate all sections, even if some are empty (state "No findings" explicitly). +9. **Prioritize**: order findings within each section by severity/impact. + +## Specialist delegation + +- This agent performs the full cross-cutting analysis itself. +- For follow-up remediation, route to `TechAIImplementer`. +- For domain-specific deep dives post-analysis, suggest the matching specialist: + - Terraform drift or policy -> `TechAITerraformGuardrails` + - IAM or privilege escalation -> `TechAIIAMLeastPrivilege` + - Workflow or supply chain -> `TechAIWorkflowSupplyChain` + - Security-specific hardening -> `TechAISecurityReviewer` + - Exhaustive per-line nit review -> `TechAIScriptReviewer` + +## Handoff + +- The generated `ANALYSIS_REPORT.md` is the primary deliverable. +- Always report the health score and verdict in the handoff message. +- If `Critical` errors are found, explicitly recommend routing to `TechAIImplementer` for remediation before merge. +- If the analysis is clean, state it explicitly: "No blocking issues found. Change set is ready for peer review." + diff --git a/.github/prompts/tech-ai-pair-architect-analysis.prompt.md b/.github/prompts/tech-ai-pair-architect-analysis.prompt.md new file mode 100644 index 0000000..92e3871 --- /dev/null +++ b/.github/prompts/tech-ai-pair-architect-analysis.prompt.md @@ -0,0 +1,83 @@ +--- +description: Analyze repository changes and generate a structured Markdown report with errors, improvements, doubts, blind spots, and architecture advice +name: TechAIPairArchitectAnalysis +agent: TechAIPairArchitect +argument-hint: target=<branch|folder|file_list> [output=ANALYSIS_REPORT.md] [depth=full|quick] [mode=standard|devil] +--- + +# Change Analysis + +## Context +Perform a deep, cross-cutting analysis of repository changes. Goes beyond line-level code review to evaluate domain design, architectural impact, operational readiness, and unconsidered aspects. Generates a self-contained Markdown report. + +## Required inputs +- **Target**: ${input:target} +- **Output file**: ${input:output:ANALYSIS_REPORT.md} +- **Depth**: ${input:depth:full,quick} +- **Mode**: ${input:mode:standard,devil} + +## Instructions + +1. Use the skill in `.github/skills/tech-ai-pair-architect/SKILL.md` as the analysis framework. +2. Identify changed files: + - If `target` is a branch name, diff against the default branch. + - If `target` is a folder or file list, analyze those files directly. +3. Auto-detect languages and load matching instruction files. +4. If `.github/skills/tech-ai-code-review/SKILL.md` exists, use it as anti-pattern reference for the Errors section. +5. When `depth=full` (default): + - Analyze all five dimensions: correctness, DDD, architecture, blind spots, and lateral thinking. + - Include all report sections with maximum detail. + - Read dependency files beyond the immediate diff when needed. +6. When `depth=quick`: + - Focus on Errors and Blind Spots only. + - Skip Architecture Advisory section. + - Limit analysis to changed files without dependency tracing. +7. When `mode=devil`: + - Apply adversarial analysis: challenge at least 3 design assumptions. + - Propose radically different alternatives with honest tradeoffs. + - Add a dedicated `Devil's Advocate` section in the report. +8. When `mode=standard` (default): + - Skip the Devil's Advocate section. +9. Read recent git history (`git log --oneline -20`) for project trajectory context. +10. Compute a health score (0-100) and populate a risk matrix for top findings. +11. Apply `security-baseline.md` controls as minimum baseline. +12. Write the report to `${input:output}` at repository root. + +## Output format + +The report follows the structure defined in the skill: +1. Executive Summary +2. Errors and Defects (with severity, explanation, and fix) +3. Improvement Opportunities (with category, effort, and rationale) +4. Doubts and Open Questions (with clarification requests) +5. Blind Spots and Unconsidered Aspects (with consequences and recommendations) +6. Architecture and Best Practices — Advisory (non-binding, with impact/effort) +7. Devil's Advocate (only when mode=devil) +8. Risk Matrix (probability/impact grid for top findings) +9. Summary Statistics (with health score and verdict) + +## Post-analysis + +After generating the report: +- Print the summary statistics and health score to the conversation. +- If Critical errors exist, recommend routing to `TechAIImplementer` for remediation. +- If the change set is clean, state it explicitly. + +## Minimal example +- Input: `target=main..feature-branch depth=full mode=devil` +- Expected output: + - `ANALYSIS_REPORT.md` at repository root with all sections populated. + - Executive summary with overall assessment and health score. + - Errors, improvements, doubts, blind spots, and architecture advice with file references. + - Devil's Advocate section with 3+ design challenges. + - Risk matrix with top findings placed on probability/impact grid. + - Summary statistics table with health score and verdict. + +## Validation +- Verify every finding references a concrete file and line number from the diff. +- Verify the report contains all mandatory sections, even if empty (state "No findings"). +- Verify architecture recommendations include impact and effort assessment. +- Verify health score is computed and verdict matches the threshold table. +- Verify risk matrix contains the most impactful findings. +- If `mode=devil`, verify at least 3 Devil's Advocate challenges are present. + diff --git a/.github/skills/tech-ai-pair-architect/SKILL.md b/.github/skills/tech-ai-pair-architect/SKILL.md new file mode 100644 index 0000000..698621b --- /dev/null +++ b/.github/skills/tech-ai-pair-architect/SKILL.md @@ -0,0 +1,246 @@ +--- +name: TechAIPairArchitect +description: Deep change-impact analysis with DDD focus, structured Markdown report generation, and lateral thinking for blind spots. +--- + +# Pair Architect Skill + +## When to use +- Analyze a set of repository changes (branch diff, PR, or file list) for correctness, design, and blind spots. +- Generate a structured `ANALYSIS_REPORT.md` with actionable findings. +- Evaluate domain modeling, bounded context integrity, and architectural alignment. +- Complement line-level code review with systems-level and business-level thinking. + +## Relationship to other skills +- **TechAICodeReview**: focuses on per-line anti-patterns and severity catalogs. Use it for exhaustive nit-level scanning. +- **TechAIPairArchitect** (this skill): focuses on change-set-level impact, DDD alignment, architectural implications, and unconsidered aspects. Use it for holistic analysis. +- Both skills can be used together: run `TechAICodeReview` for detailed findings, then `TechAIPairArchitect` for the bigger picture. + +## Analysis dimensions + +### 1. Correctness analysis +- Does the code do what the change claims? +- Are edge cases handled? +- Are error paths tested? +- Is input validation present and sufficient? + +### 2. Domain-Driven Design analysis +Evaluate against these DDD principles: + +| Principle | What to check | +|---|---| +| Bounded contexts | Are context boundaries clear? Does the change leak domain concepts across boundaries? | +| Ubiquitous language | Do class/method/variable names use domain terminology consistently? | +| Aggregates | Are aggregate roots properly guarding invariants? Is there transactional boundary leakage? | +| Entities vs Value Objects | Are identities correctly modeled? Are immutable concepts modeled as value objects? | +| Domain services | Is business logic placed in the domain layer or scattered in application/infrastructure? | +| Anti-corruption layers | When integrating with external systems, is there a translation layer? | +| Repository pattern | Is persistence abstracted from domain logic? | + +#### DDD smell catalog + +Use these IDs when flagging DDD violations: + +##### Critical +| ID | Smell | Why | +|---|---|---| +| DDD-C01 | Domain logic in infrastructure layer | Business rules coupled to I/O, untestable without external dependencies | +| DDD-C02 | Aggregate invariant bypassed via direct child access | Data consistency guarantee broken, corruption risk | +| DDD-C03 | Shared mutable state across bounded contexts | Tight coupling destroys independent deployability | + +##### Major +| ID | Smell | Why | +|---|---|---| +| DDD-M01 | Anemic domain model | Business logic in application or service layer instead of entities or value objects | +| DDD-M02 | God aggregate (> 5 entities or > 3 collections) | Transaction scope too wide, performance and contention risk | +| DDD-M03 | Ubiquitous language drift | Technical naming instead of domain terminology — cognitive gap grows over time | +| DDD-M04 | Cross-context direct import | Module A imports domain types from Module B without anti-corruption layer | +| DDD-M05 | Repository returns infrastructure types | Domain layer depends on persistence details | +| DDD-M06 | Domain events carrying infrastructure concerns | Events should be pure domain facts, not transport metadata | +| DDD-M07 | Use case orchestration in domain entities | Entities should guard invariants, not coordinate workflows | + +##### Minor +| ID | Smell | Why | +|---|---|---| +| DDD-m01 | Value object modeled as entity (has identity but no lifecycle) | Over-engineering, unnecessary complexity | +| DDD-m02 | Missing factory method for complex aggregate creation | Construction logic scattered, hard to enforce invariants | +| DDD-m03 | Domain service doing what an entity method could do | Misplaced responsibility, weaker encapsulation | +| DDD-m04 | Specification pattern missing for complex business rules | Rules scattered across multiple locations | +| DDD-m05 | Missing domain event for a significant state transition | Reduced observability and extensibility | + +##### Nit +| ID | Smell | Why | +|---|---|---| +| DDD-N01 | Inconsistent naming between code and domain glossary | Language drift signal | +| DDD-N02 | Bounded context boundary not documented | Implicit knowledge, onboarding friction | +| DDD-N03 | Aggregate methods not named with domain verbs | Verbs like `process()` or `handle()` obscure intent | + +### 3. Architecture analysis +Evaluate structural qualities: + +| Quality | What to check | +|---|---| +| Separation of concerns | Are domain, application, infrastructure, and presentation layers distinct? | +| Dependency direction | Do dependencies point inward (infrastructure -> application -> domain)? | +| Coupling | Is coupling between modules explicit and minimal? | +| Cohesion | Are related concepts grouped together? | +| Extensibility | Can this design accommodate likely future changes without significant rework? | +| Testability | Can each component be tested in isolation? | +| Operational readiness | Are logs, metrics, and health checks present for production visibility? | + +### 4. Blind-spot detection +Apply lateral thinking: + +- **Temporal analysis**: Will this change cause problems at scale? After 6 months of accumulation? +- **Team dynamics**: Does this change increase onboarding friction for new team members? +- **Cross-service impact**: Could this change break consumers or upstream producers? +- **Operational burden**: What happens when this fails at 3 AM? Can on-call engineers debug it? +- **Data implications**: Are there schema changes, migration needs, or data consistency risks? +- **Security surface**: Does this change expand the attack surface? +- **Performance cliffs**: Is there a hidden O(n²) or unbounded resource consumption? +- **Configuration drift**: Are there environment-specific assumptions that break in other stages? +- **Missing observability**: Can we know if something goes wrong after deployment? +- **Alternative solutions**: Is there a fundamentally simpler approach that was not considered? + +## Severity mappings + +### Errors and Defects +| Severity | Criteria | +|---|---| +| Critical | Security flaw, data loss risk, correctness bug affecting business logic | +| Major | Missing error handling, broken contract, regression risk | + +### Improvements +| Category | Description | +|---|---| +| Readability | Code clarity, naming, structure | +| Performance | Algorithmic efficiency, resource usage | +| Maintainability | Technical debt, coupling, cohesion | +| Testability | Test coverage gaps, untestable designs | +| DDD | Domain modeling, bounded context, ubiquitous language | +| Security | Hardening, least privilege, input validation | + +### Effort estimation +| Level | Meaning | +|---|---| +| Low | Less than 1 hour, isolated change | +| Medium | 1-4 hours, may touch multiple files | +| High | More than 4 hours, may require design discussion | + +## Health score + +Compute a numeric health score (0-100) for the change set and include it in the Executive Summary. + +### Calculation +- Start at 100. +- For each finding, subtract points based on severity: + +| Finding type | Points deducted per instance | +|---|---| +| Critical error | -20 | +| Major error | -10 | +| Blind spot | -5 | +| Improvement opportunity | -2 | +| Doubt or open question | -1 | + +- Floor at 0 (never go negative). +- Architecture recommendations (advisory) do not affect the score. + +### Verdict thresholds +| Score | Verdict | Meaning | +|---|---|---| +| 90-100 | Excellent | Ready to merge with confidence | +| 70-89 | Good | Minor issues, safe to merge after addressing them | +| 50-69 | Needs Work | Significant issues that should be resolved before merge | +| 30-49 | Poor | Major rework needed, consider design discussion | +| 0-29 | Critical | Do not merge — blocking issues present | + +## Risk matrix + +For the top findings (up to 8), place them on a 2x2 risk matrix in the report: + +``` + HIGH IMPACT + │ + URGENT │ PLAN + (fix before merge) │ (fix in next sprint) + │ + ──────────────────────┼────────────────────── + │ + MONITOR │ ACCEPT + (watch in prod) │ (low risk, defer) + │ + LOW IMPACT + HIGH PROBABILITY LOW PROBABILITY +``` + +For each placed finding, state: `[<ID>] <title> → <quadrant>`. + +## Report template + +Generate the report following this structure (all sections mandatory): + +1. **Executive Summary** — 2-4 sentences on what changed and overall assessment. +2. **Errors and Defects** — Objectively wrong things with fix suggestions. +3. **Improvement Opportunities** — Working code that can be better, with effort estimates. +4. **Doubts and Open Questions** — Ambiguous areas requiring author clarification. +5. **Blind Spots and Unconsidered Aspects** — Things the change does not address but should. +6. **Architecture and Best Practices (Advisory)** — Non-binding DDD and architecture guidance. +7. **Devil's Advocate** (only when `mode=devil`) — Adversarial challenges to design assumptions. +8. **Risk Matrix** — Top findings placed on probability/impact grid. +9. **Summary Statistics** — Counts per category and health score. + +For empty sections, state "No findings in this category" explicitly. + +## Devil's advocate mode + +When invoked with `mode=devil`, apply an adversarial analysis layer: + +1. Find at least 3 fundamental objections to the chosen design approach. +2. For each objection, propose a radically different alternative with tradeoffs. +3. Challenge assumptions that seem obvious — ask "what if this premise is wrong?" +4. Consider the worst-case scenario for each major design decision. +5. Write a dedicated section `## 6. Devil's Advocate` between Architecture and Risk Matrix. + +Format: +```markdown +### [DEVIL-<NNN>] <provocative question> +- **Assumption challenged**: <what the change takes for granted> +- **What if it is wrong**: <consequence> +- **Alternative approach**: <radically different solution> +- **Tradeoffs**: <pros and cons vs current approach> +``` + +## Git history awareness + +When analyzing a change set, also gather recent repository context: + +1. Read recent commit history (`git log --oneline -20`) to understand project direction. +2. Check for patterns: are similar changes being made repeatedly? Is there churn? +3. Use this context in the Blind Spots section to flag: + - Misalignment with recent project trajectory. + - Repeated patterns that suggest a systemic issue needing a different solution. + - Abandoned or reverted approaches being reintroduced. + +## Workflow + +1. Identify changed files (diff against default branch or explicit file list). +2. Load applicable instruction files based on detected languages. +3. Read each changed file and its immediate dependencies. +4. Analyze across all dimensions (correctness, DDD, architecture, blind spots). +5. Write findings into the report template. +6. Compute health score and populate risk matrix. +7. If `mode=devil`, run adversarial analysis and add Devil's Advocate section. +8. Save as `ANALYSIS_REPORT.md` at repository root. +9. Report completion with summary statistics and health score. + +## Validation +- Every finding must reference a concrete file and line number. +- Every finding must include a *why* explanation. +- Every error or improvement must include a *how to fix* suggestion. +- Architecture recommendations must include impact/effort assessment. +- Health score must be computed and verdict must match the threshold table. +- Risk matrix must contain the most impactful findings (up to 8). +- If `mode=devil`, at least 3 Devil's Advocate challenges must be present. +- The report must be valid Markdown with no broken links or formatting. + diff --git a/AGENTS.md b/AGENTS.md index 7813c14..66ac9f9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,6 +33,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Use `TechAIImplementer` for direct code or configuration changes and validation-first delivery. - Use `TechAIReviewer` for quality gates and defect or regression findings. - Use `TechAIScriptReviewer` for exhaustive, nit-level reviews on Python, Bash, and Terraform. +- Use `TechAIPairArchitect` for deep change-impact analysis with DDD focus, blind-spot detection, and structured Markdown report generation. - Use `TechAIGlobalCustomizationBuilder` as the default specialist for creating or updating GitHub Copilot customization assets in this repository. - Use `TechAIGlobalCustomizationAuditor` as the final quality gate for GitHub Copilot customization changes in this repository. - Use `TechAICustomizationAuditor` only as a deprecated compatibility alias while older references are migrated. @@ -52,6 +53,9 @@ This file is for GitHub Copilot and AI assistants working in this repository. - Do not use `TechAIImplementer` alone when the task is cross-repository Copilot configuration alignment; use `TechAISyncCopilotConfigs`. - Do not use `TechAISyncCopilotConfigs` alone when the task is to author new repository-owned `internal-*` assets in a consumer repository; use `TechAIInternalCopilotCustomizationBuilder` after baseline alignment. - Do not use `TechAIInternalCopilotCustomizationBuilder` to add new shared `tech-ai-*` assets in this standards repository; use `TechAIGlobalCustomizationBuilder`. +- Do not use `TechAIPairArchitect` for quick line-level nit reviews; use `TechAIScriptReviewer` or `TechAICodeReview` instead. +- Do not use `TechAIReviewer` when you need holistic change-set impact analysis with DDD, architecture, and blind spots; use `TechAIPairArchitect`. +- Do not use `TechAIPairArchitect` for exhaustive per-language anti-pattern scanning; use `TechAIScriptReviewer` and then `TechAIPairArchitect` for the bigger picture. ### Composition and Handoffs @@ -64,6 +68,8 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `TechAIReviewer` findings flagged as `Critical` or `Major` route back to `TechAIImplementer` for remediation. - `TechAIGlobalCustomizationBuilder` output is input context for `TechAIGlobalCustomizationAuditor`. - `TechAIGlobalCustomizationAuditor` findings flagged as `Critical` or `Major` route back to `TechAIGlobalCustomizationBuilder` for remediation. +- `TechAIPairArchitect` output (`ANALYSIS_REPORT.md`) is input context for `TechAIImplementer` when remediation is needed. +- For thorough pre-merge validation, the recommended chain is `TechAIImplementer` -> `TechAIPairArchitect` -> `TechAIImplementer` (remediation). ## Governance References @@ -136,6 +142,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `TechAIPRDescription`: pull request body generation. - `TechAIAddUnitTests`: test authoring and improvement. - `TechAITerraform`: Terraform feature or module authoring. +- `TechAIPairArchitectAnalysis`: deep change-impact analysis with DDD focus, health score, risk matrix, and devil's advocate mode. ### Preferred skills @@ -146,6 +153,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `TechAIPRWriting`: PR writing conventions aligned to the repository template. - `TechAICloudPolicy`: reusable cloud policy authoring patterns. - `TechAITerraformModule`: reusable Terraform module design. +- `TechAIPairArchitect`: change-set-level impact, DDD smell catalog, health scoring, risk matrix, and blind-spot detection. ### Required validations before PR @@ -178,6 +186,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. - `.github/prompts/tech-ai-add-report-script.prompt.md` - `.github/prompts/tech-ai-add-unit-tests.prompt.md` - `.github/prompts/tech-ai-bash-script.prompt.md` +- `.github/prompts/tech-ai-pair-architect-analysis.prompt.md` - `.github/prompts/tech-ai-cicd-workflow.prompt.md` - `.github/prompts/tech-ai-cloud-policy.prompt.md` - `.github/prompts/tech-ai-code-review.prompt.md` @@ -196,6 +205,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. ### Skills +- `.github/skills/tech-ai-pair-architect/SKILL.md` - `.github/skills/tech-ai-cicd-workflow/SKILL.md` - `.github/skills/tech-ai-cloud-policy/SKILL.md` - `.github/skills/tech-ai-code-review/SKILL.md` @@ -214,6 +224,7 @@ This file is for GitHub Copilot and AI assistants working in this repository. ### Agents +- `.github/agents/tech-ai-pair-architect.agent.md` - `.github/agents/tech-ai-customization-auditor.agent.md` - `.github/agents/tech-ai-github-pr-writer.agent.md` - `.github/agents/tech-ai-github-workflow-supply-chain.agent.md` From 3cb32578de1add592c5d6ffff6e13859dccc3c29 Mon Sep 17 00:00:00 2001 From: Diego Mauricio Lagos <diego.lagosmorales@pagopa.it> Date: Tue, 10 Mar 2026 13:56:08 +0100 Subject: [PATCH 9/9] feat: update TechAI Pair Architect agent and analysis prompt to streamline analysis framework and reporting structure --- .../agents/tech-ai-pair-architect.agent.md | 205 +----------------- .../tech-ai-pair-architect-analysis.prompt.md | 61 +----- .../skills/tech-ai-pair-architect/SKILL.md | 2 + 3 files changed, 18 insertions(+), 250 deletions(-) diff --git a/.github/agents/tech-ai-pair-architect.agent.md b/.github/agents/tech-ai-pair-architect.agent.md index 38e9c29..a15e747 100644 --- a/.github/agents/tech-ai-pair-architect.agent.md +++ b/.github/agents/tech-ai-pair-architect.agent.md @@ -45,204 +45,19 @@ Analyze all modifications in a repository change set (branch diff, PR, or set of - Examine how changes interact with unchanged code in the immediate dependency graph. - Consider temporal effects: will this change create problems in 3 months? 6 months? At scale? -## Modes +## Analysis framework -### Depth: full (default) -Analyze across all dimensions — correctness, DDD, architecture, blind spots, and lateral thinking. Include all report sections with maximum detail. Read dependency files beyond the immediate diff when needed. +Use `.github/skills/tech-ai-pair-architect/SKILL.md` as the single source of truth for: -### Depth: quick -Focus on Errors and Blind Spots only. Skip the Architecture Advisory section. Limit analysis to changed files without dependency tracing. Useful for fast feedback loops during development. +- Analysis dimensions and DDD smell catalog. +- Severity mappings and health score calculation. +- Report template and section structure. +- Modes (depth: full/quick, mode: standard/devil). +- Git history awareness steps. +- Risk matrix format. +- Validation checklist. -### Standard mode (default) -Analyze across all dimensions and produce the full report. - -### Devil's advocate mode (`mode=devil`) -In addition to the standard analysis, apply an adversarial thinking layer: -- Find at least 3 fundamental objections to the chosen design approach. -- Propose radically different alternatives with honest tradeoffs. -- Challenge assumptions that seem obvious. -- Add a dedicated `## 6. Devil's Advocate` section in the report. - -## Git history awareness - -Before analyzing the diff, gather project trajectory context: -1. Read recent commit history (`git log --oneline -20`) to understand direction. -2. Check for churn patterns, repeated similar changes, or reverted approaches. -3. Use this context in Blind Spots to flag misalignment with project trajectory. - -## Report generation - -### Default behavior -- Generate the report as a Markdown file at the repository root: `ANALYSIS_REPORT.md`. -- If the file already exists, overwrite it with the latest analysis. -- The report must be self-contained and readable without additional context. - -### Report structure - -The report MUST contain these sections in this exact order: - -```markdown -# Change Analysis Report - -> Generated: <ISO 8601 timestamp> -> Branch: <current_branch> → <default_branch> -> Files analyzed: <count> -> Languages detected: <list> - ---- - -## Executive Summary -<2-4 sentences: what changed, overall assessment, top risk, and health score verdict> - ---- - -## 1. Errors and Defects - -Issues that are objectively wrong and must be fixed. - -### [ERROR-<NNN>] <title> -- **File**: <path>#L<line> -- **Severity**: Critical | Major -- **What is wrong**: <clear description> -- **Why it matters**: <business/technical impact> -- **How to fix**: <concrete suggestion with code snippet if applicable> - ---- - -## 2. Improvement Opportunities - -Things that work but can be done better. - -### [IMPROVE-<NNN>] <title> -- **File**: <path>#L<line> -- **Category**: Readability | Performance | Maintainability | Testability | DDD | Security -- **Current state**: <what exists now> -- **Suggested improvement**: <specific recommendation> -- **Why it is better**: <rationale with tradeoff analysis> -- **Effort**: Low | Medium | High - ---- - -## 3. Doubts and Open Questions - -Aspects that are ambiguous or where the intent is unclear. - -### [DOUBT-<NNN>] <title> -- **File**: <path>#L<line> -- **Question**: <specific question about the change> -- **Why it matters**: <what could go wrong if the assumption is incorrect> -- **Suggested clarification**: <what the author should verify or document> - ---- - -## 4. Blind Spots and Unconsidered Aspects - -Things the change does NOT address that it probably should, or second-order effects. - -### [BLIND-<NNN>] <title> -- **Area**: <domain, infrastructure, testing, operations, security, etc.> -- **What was not considered**: <description> -- **Potential consequence**: <what could happen> -- **Recommendation**: <what to investigate or add> - ---- - -## 5. Architecture and Best Practices (Advisory) - -> ⚠️ This section contains non-binding recommendations. They represent architectural guidance -> and best practices that the team should evaluate against their specific context and constraints. - -### 5.1 Domain Design Assessment -- **DDD alignment verdict**: <Aligned | Partially Aligned | Not Aligned | Not Applicable> -- **Bounded context observations**: <analysis> -- **Ubiquitous language consistency**: <analysis> -- **Aggregate/entity/value-object placement**: <observations> - -### 5.2 Architectural Recommendations -For each recommendation: -#### [ARCH-<NNN>] <title> -- **Current state**: <how things are now> -- **Recommended approach**: <what to consider> -- **Rationale**: <why this is a good practice with references> -- **Impact**: Low | Medium | High -- **Effort**: Low | Medium | High -- **Priority**: <suggested priority based on impact/effort ratio> - -### 5.3 Lateral Thinking — Outside-the-Box Observations -Things that a conventional review might miss. Think about: -- Second-order effects on other teams or services. -- Operational burden at scale (on-call, debugging, monitoring). -- Developer experience and onboarding friction. -- Alternative approaches that could simplify the entire problem space. -- Business model or product implications hidden in technical choices. -- Patterns emerging across multiple files that suggest a systemic issue. - ---- - -## 6. Devil's Advocate (only when mode=devil) - -### [DEVIL-<NNN>] <provocative question> -- **Assumption challenged**: <what the change takes for granted> -- **What if it is wrong**: <consequence> -- **Alternative approach**: <radically different solution> -- **Tradeoffs**: <pros and cons vs current approach> - ---- - -## 7. Risk Matrix - -Top findings placed on a probability/impact grid: - -``` - HIGH IMPACT - │ - URGENT │ PLAN - (fix before merge) │ (fix in next sprint) - │ - ──────────────────────┼────────────────────── - │ - MONITOR │ ACCEPT - (watch in prod) │ (low risk, defer) - │ - LOW IMPACT - HIGH PROBABILITY LOW PROBABILITY -``` - -Placement: `[<ID>] <title> → <quadrant>` - ---- - -## Summary Statistics - -| Category | Count | -|---|---| -| Errors and Defects | <count> | -| Improvement Opportunities | <count> | -| Doubts and Open Questions | <count> | -| Blind Spots | <count> | -| Architecture Recommendations | <count> | -| Devil's Advocate Challenges | <count> | - -**Health Score: <score>/100 — <verdict>** -``` - -## Analysis protocol - -1. **Gather context**: identify changed files, detect languages, load instruction files. Read recent git history for trajectory awareness. -2. **Map the change**: understand the intent behind the change set as a whole, not just individual files. -3. **Analyze layer by layer**: - - Correctness: does it do what it claims? - - Domain integrity: are DDD boundaries respected? - - Architecture: does it fit the broader system design? - - Operations: can this be debugged, monitored, and maintained? - - Security: does it follow least privilege and security baseline? - - Testing: is the change testable and tested? -4. **Apply lateral thinking**: step outside the immediate change and ask "what else?" -5. **Apply devil's advocate** (if `mode=devil`): challenge 3+ design assumptions with radical alternatives. -6. **Compute health score**: apply point deductions per finding type, determine verdict. -7. **Populate risk matrix**: place top findings on probability/impact grid. -8. **Write the report**: populate all sections, even if some are empty (state "No findings" explicitly). -9. **Prioritize**: order findings within each section by severity/impact. +Do not duplicate those definitions here — defer to the skill file at runtime. ## Specialist delegation diff --git a/.github/prompts/tech-ai-pair-architect-analysis.prompt.md b/.github/prompts/tech-ai-pair-architect-analysis.prompt.md index 92e3871..fc9e148 100644 --- a/.github/prompts/tech-ai-pair-architect-analysis.prompt.md +++ b/.github/prompts/tech-ai-pair-architect-analysis.prompt.md @@ -18,43 +18,12 @@ Perform a deep, cross-cutting analysis of repository changes. Goes beyond line-l ## Instructions -1. Use the skill in `.github/skills/tech-ai-pair-architect/SKILL.md` as the analysis framework. -2. Identify changed files: - - If `target` is a branch name, diff against the default branch. - - If `target` is a folder or file list, analyze those files directly. -3. Auto-detect languages and load matching instruction files. -4. If `.github/skills/tech-ai-code-review/SKILL.md` exists, use it as anti-pattern reference for the Errors section. -5. When `depth=full` (default): - - Analyze all five dimensions: correctness, DDD, architecture, blind spots, and lateral thinking. - - Include all report sections with maximum detail. - - Read dependency files beyond the immediate diff when needed. -6. When `depth=quick`: - - Focus on Errors and Blind Spots only. - - Skip Architecture Advisory section. - - Limit analysis to changed files without dependency tracing. -7. When `mode=devil`: - - Apply adversarial analysis: challenge at least 3 design assumptions. - - Propose radically different alternatives with honest tradeoffs. - - Add a dedicated `Devil's Advocate` section in the report. -8. When `mode=standard` (default): - - Skip the Devil's Advocate section. -9. Read recent git history (`git log --oneline -20`) for project trajectory context. -10. Compute a health score (0-100) and populate a risk matrix for top findings. -11. Apply `security-baseline.md` controls as minimum baseline. -12. Write the report to `${input:output}` at repository root. - -## Output format - -The report follows the structure defined in the skill: -1. Executive Summary -2. Errors and Defects (with severity, explanation, and fix) -3. Improvement Opportunities (with category, effort, and rationale) -4. Doubts and Open Questions (with clarification requests) -5. Blind Spots and Unconsidered Aspects (with consequences and recommendations) -6. Architecture and Best Practices — Advisory (non-binding, with impact/effort) -7. Devil's Advocate (only when mode=devil) -8. Risk Matrix (probability/impact grid for top findings) -9. Summary Statistics (with health score and verdict) +1. Read `.github/skills/tech-ai-pair-architect/SKILL.md` and use it as the complete analysis framework (dimensions, severity mappings, health score, report template, modes, validation). +2. If `.github/skills/tech-ai-code-review/SKILL.md` exists, use it as anti-pattern reference for the Errors section. +3. Identify changed files from `target` (branch diff, folder, or file list) and auto-detect languages. +4. Follow the skill workflow: gather context, analyze, compute health score, populate risk matrix, write report. +5. Apply depth and mode parameters as defined in the skill. +6. Write the report to `${input:output}` at repository root. ## Post-analysis @@ -63,21 +32,3 @@ After generating the report: - If Critical errors exist, recommend routing to `TechAIImplementer` for remediation. - If the change set is clean, state it explicitly. -## Minimal example -- Input: `target=main..feature-branch depth=full mode=devil` -- Expected output: - - `ANALYSIS_REPORT.md` at repository root with all sections populated. - - Executive summary with overall assessment and health score. - - Errors, improvements, doubts, blind spots, and architecture advice with file references. - - Devil's Advocate section with 3+ design challenges. - - Risk matrix with top findings placed on probability/impact grid. - - Summary statistics table with health score and verdict. - -## Validation -- Verify every finding references a concrete file and line number from the diff. -- Verify the report contains all mandatory sections, even if empty (state "No findings"). -- Verify architecture recommendations include impact and effort assessment. -- Verify health score is computed and verdict matches the threshold table. -- Verify risk matrix contains the most impactful findings. -- If `mode=devil`, verify at least 3 Devil's Advocate challenges are present. - diff --git a/.github/skills/tech-ai-pair-architect/SKILL.md b/.github/skills/tech-ai-pair-architect/SKILL.md index 698621b..eb0ec76 100644 --- a/.github/skills/tech-ai-pair-architect/SKILL.md +++ b/.github/skills/tech-ai-pair-architect/SKILL.md @@ -39,6 +39,8 @@ Evaluate against these DDD principles: #### DDD smell catalog +> **Conditional**: Skip this catalog when the DDD alignment verdict is "Not Applicable" (e.g., pure infrastructure or configuration repositories). Reference it only when the change set contains application code with domain modeling. + Use these IDs when flagging DDD violations: ##### Critical