chore: Refactor CometScanRule to improve scan selection and fallback logic#2978

mbutrovich merged 6 commits intoapache:mainapache/datafusion-comet:mainfrom

andygrove:scan-rule-refineandygrove/datafusion-comet:scan-rule-refineCopy head branch name to clipboard