Run pytest in CI using the latest pandas v3 pre-releases

[OmBiradar]

Change Description

Closes #56

Solution Description

Runs a CI job that installs pandas latest nightly build (see installing_dev_branch) and runs tests to note errors for informational purposes.

Failures during this job won't block merges as it's intended for informational purposes only.

The nightly build is only available for python 3.11, 3.12, 3.13 & 3.14 and is tested against these only (Dropped 3.10). See PyPI

Code Quality

• I have read the Contribution Guide and agree to the Code of Conduct
• My code follows the code style of this project
• My code builds (or compiles) cleanly without any errors or warnings (tested with local act)
• My code contains relevant comments and necessary documentation

[hombit]

@OmBiradar thank you for your submissions. Please take into account that we are reviewing and considering only the first GSoC pull request you've submitted. Feel free to close the others or change the title/resubmit as regular pull requests, but please be aware that it will not influence our decision about your proposal.

[OmBiradar]

@hombit thank you for the information. I will change the titles on my other pull requests.

[OmBiradar]

@hombit I believe that this PR is mature as I have conducted local tests of this workflow using act.

If there are any changes that need to be done to make this PR better. Please let me know.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.30%. Comparing base (32a90df) to head (4873895).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #465   +/-   ##
=======================================
  Coverage   97.30%   97.30%           
=======================================
  Files          19       19           
  Lines        2156     2156           
=======================================
  Hits         2098     2098           
  Misses         58       58           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Before [32a90df]	After [1728cab]	Ratio	Benchmark (Parameter)
1.04±0.08s	1.09±0.05s	1.05	benchmarks.ReadFewColumnsHTTPS.time_run
1.24±0.02ms	1.29±0.01ms	1.04	benchmarks.NestedFrameReduce.time_run
10.6±0.2ms	10.8±0.2ms	1.02	benchmarks.NestedFrameAddNested.time_run
10.6±0.08ms	10.8±0.07ms	1.02	benchmarks.NestedFrameQuery.time_run
176M	179M	1.02	benchmarks.ReadFewColumnsHTTPS.peakmem_run
137M	138M	1.01	benchmarks.CountNestedBy.peakmem_run
258M	258M	1	benchmarks.AssignSingleDfToNestedSeries.peakmem_run
105M	105M	1	benchmarks.NestedFrameAddNested.peakmem_run
110M	110M	1	benchmarks.NestedFrameQuery.peakmem_run
109M	109M	1	benchmarks.NestedFrameReduce.peakmem_run

Click here to view all benchmarks.

[hombit]

Thank you! I converted it back from "draft" as you said that it is ready for review.

I guess these CI workflow will fail for a while because we haven't migrated to pandas v3 yet. Do you think it would make sense to make it fall "gently," keep it green, but just output warnings to the action summary?

[OmBiradar]

Do you think it would make sense to make it fall "gently," keep it green, but just output warnings to the action summary?

I had assumed that the continue-on-error: true would make the tests pass in any condition, but that is not the case.

I will find a way to make it green by setting the condition that if the tests run (faillure or success) it will pass the check.

Also I will work on your suggestions.

Thank you for your insignts!
@hombit

[OmBiradar]

Hey @hombit

This time I have used python -m pytest || true as the last fail safe to pass the test irrespective of the failing tests.

Also I have scheduled a cron job to run at 6:45 UTC just as asv-dev.

I hope all tests pass this time.

[hombit]

@OmBiradar Sorry, I didn't think about how we are going to test it when asked to change to nightly 🤦. Could you please put PR runs back and keep both nightly runs and PR runs (main branch runs are not needed, I think)?

Do we need both continue-on-error: true and || true? I'm not sure how it outputs now, but it would be nice to set "workflow annotations" with error messages converted to warnings. I believe these docs are relevant:
https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-commands

[OmBiradar]

@hombit

put PR runs back and keep both nightly runs and PR runs (main branch runs are not needed, I think)?

Your right, the checks make better sense running on PR's as the PR's can be modified to maybe accomodate for pandas 3.0 better.

Do we need both continue-on-error: true and || true?

I found that adding continue-on-error: true under the test-pandas-dev job - it doesn't work and the test still fails.
I have removed the || true which was a shell override for error code returns.

The resource I found helpful was - https://github.com/marketplace/actions/continue-on-error-comment

Rather adding continue-on-error: true on the job step (sub job) name: Run unit tests with pytest made the test pass and the CI pipeline work.

[hombit]

That is good, but I believe it lacks some usability. We should go and dig into the logs to see what is going on there. Could you please look into the annotations?

[OmBiradar]

I believe github-actions bot gives some details as shown in https://github.com/marketplace/actions/continue-on-error-comment

But this just shows the error code I guess, I will look into the ways to make the exact errors more visible.

[OmBiradar]

@hombit

I just added the annotations to this! Pretty cool features! Also I had tested them on my fork using a testing branch - OmBiradar#1

I hope this works perfectly

[OmBiradar]

I find that github doens't give write access to new github actions added through forked repo PR's, that is why the annotations are not visible here in this PR particularly.

I tested it on my own fork here - OmBiradar#1

I think this works perfectly!

[hombit]

@OmBiradar thanks! I've actually meant workflow annotations, like warning this

Let's ask for the third opinion, @dougbrn, what do you think about PR comments from the pandas nightly CI introduced here? (See OmBiradar#1)

@OmBiradar if we go with the PR comments implementation, I think that four comments is too much, let's maybe run with a single Python version?

[OmBiradar]

@hombit

I don't have any idea right now which python version might be the best to run against....

But persoanlly I feel that a tabular form might be more consise and still represent the same data.

one thing i want to add is that the four comments are a one time thing, thus if there are any updates to the PR, the four comments are updated (edited i mean), so it's always gonna be just 4 in number.

[hombit]

@OmBiradar four is still a lot! Every PR is already getting 2-3 automatic comments, and I'm afraid that we'll just stop reading them and will stop seeing human comments. Let's keep Python 3.11 only? And also, let's see what @dougbrn thinks on PR comments vs Action annotations.
It is not about tabular format, it is about where we see this information: as a PR comment or as an Action run annotation.

[OmBiradar]

@hombit

by tabular format i ment representing all the 4 versions of python condensed into a single table in a github-actions comment rather than 4 seperate ones.

I think keeping only 3.11 works as I see the same number of tests failing in all 4 versions of python.

[dougbrn]

I think with the annotations there's a real case that they'll go unnoticed in a given PR since you need to click into a green checked action log, which you would only really do if you're being thorough or specifically looking for this. Given that I think I like a single (python 3.11 sounds good) PR comment. Nice work @OmBiradar !

[OmBiradar]

@hombit @dougbrn

I have upated the CI workflow to only run for Python 3.11

Nice work @OmBiradar !

Thanks! 🙂