Skip to content

Update output#29

Draft
raphaelpor wants to merge 4 commits intomainfrom
feature/update-output
Draft

Update output#29
raphaelpor wants to merge 4 commits intomainfrom
feature/update-output

Conversation

@raphaelpor
Copy link
Owner

This pull request updates the test execution logging format and summary output for the CLI and test framework. The changes standardize log lines to use "Finished in" instead of "Passed in" or "Failed in", remove explicit pass/fail indicators from per-test logs, and update summary formatting to highlight only the counts in cyan and bold. The implementation and test files are updated for consistency, and related documentation/specs are revised to match the new output.

Logging and Output Format Changes

  • Updated per-test log lines to use the format - Finished in <cyan bold N ms> instead of - ✅ Passed in <cyan bold Nms> or - ❌ Failed in <cyan bold Nms>, removing explicit pass/fail indicators. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]
  • Changed summary output to highlight only the counts (e.g., 1 passed) in cyan and bold, not the labels (e.g., Files, Evals), and updated formatting for alignment and clarity. [1] [2] [3] [4]

Implementation Refactoring

  • Removed pass/fail tracking from test logging logic (didPass parameter and outcome calculation), simplifying logTestExecution and related APIs. [1] [2] [3] [4] [5] [6] [7]

Documentation and Spec Updates

  • Updated documentation and spec files to reflect new log and summary formats, including examples and descriptions of the output changes. [1] [2] [3] [4]

Test Updates

  • Revised tests and assertions to match new output formats and summary alignment, ensuring consistency across all test cases. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Code Cleanup

  • Removed unused imports and code related to pass/fail tracking in it.ts. [1] [2]

These changes ensure a more consistent and streamlined test logging experience, with clearer output and easier maintenance.

Document and implement per-test execution output and ensure suite headings are deduplicated. Updates include: docs/specs to describe per-test lines (passed/failed, model, tokens), tests in src/lib/it/it.test.ts that cover failed async test logging and suite-heading deduplication, change in src/lib/it/it.ts to log based on actual test result (didPass()), and refactor src/lib/output/testLogging.ts to track logged suites with a Set and reset it. These changes ensure accurate pass/fail messages and that each suite header is printed only once.
Replace per-test pass/fail lines with a unified "Finished in <N ms>" message and adjust duration formatting (adds space before "ms"). Remove didPass parameter from logging APIs and simplify logCurrentContextExecution/logTestExecution to always emit timing (and include model/token lines when present). Update CLI summary layout for aligned labels and cyan-bolded values. Bump spec example model to gpt-5.2 and update docs/specs/tests to match the new output format; remove some getFailedTestCount usage in it.ts.
@raphaelpor raphaelpor self-assigned this Feb 24, 2026
@raphaelpor raphaelpor added the enhancement New feature or request label Feb 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant