Skip to content

Defer pandas/omegaconf imports#207

Merged
timkpaine merged 1 commit into
mainfrom
pit/simplify_deps
May 11, 2026
Merged

Defer pandas/omegaconf imports#207
timkpaine merged 1 commit into
mainfrom
pit/simplify_deps

Conversation

@ptomecek
Copy link
Copy Markdown
Collaborator

@ptomecek ptomecek commented May 11, 2026

Summary

Defer eager imports of pandas and omegaconf so they no longer load on a bare import ccflow. Pure import-site relocation; runtime behaviour unchanged.

Changes

  • ccflow/exttypes/frequency.py: drop module-level import pandas as pd and from pandas.tseries.frequencies import to_offset. Re-import locally in Frequency.offset, Frequency.timedelta, and Frequency._validate. Replace the module-level _PD_GE_22 constant (which read pd.__version__ at import) with a @cached _pd_ge_22() function called from _normalize_frequency_alias.
  • ccflow/utils/chunker.py: move import pandas as pd into dates_to_chunks.
  • ccflow/validators.py: move import pandas as pd into normalize_date and normalize_datetime.
  • ccflow/base.py: drop module-level import omegaconf and from omegaconf import DictConfig, OmegaConf. Use if TYPE_CHECKING: from omegaconf import DictConfig so static type-checkers still resolve the annotations, and quote the runtime DictConfig annotations on ModelRegistry.load_config, ModelRegistry.create_config_from_path, and the _ModelRegistryLoader methods. Add local imports of DictConfig/OmegaConf/omegaconf inside _is_config_subregistry, _make_subregistries, and _ModelRegistryLoader.load_config where they are actually used at runtime.

Measurements

Profiled in a clean Python 3.11 venv with only ccflow's base deps installed.

Before After
import ccflow total ~525 ms ~320 ms (~38% faster)
pandas in sys.modules after import yes no
omegaconf in sys.modules after import yes no
antlr4 in sys.modules after import yes (via omegaconf) no

Compatibility

  • All public re-exports from ccflow (e.g. from ccflow import Frequency) still work.
  • No type-checking break: the quoted "DictConfig" annotations plus the TYPE_CHECKING import keep static type information intact.
  • Smoke-tested: Frequency validation (including legacy alias path that exercises _pd_ge_22()), normalize_date, normalize_datetime, and dates_to_chunks all still work.

@ptomecek ptomecek force-pushed the pit/simplify_deps branch from 9f8bad1 to 1dc6546 Compare May 11, 2026 17:23
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 11, 2026

Test Results

729 tests  ±0   727 ✅ ±0   1m 43s ⏱️ -7s
  1 suites ±0     2 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 22e4ca4. ± Comparison against base commit 59bc0e3.

♻️ This comment has been updated with latest results.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

❌ Patch coverage is 90.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.33%. Comparing base (60ee2e7) to head (22e4ca4).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
ccflow/base.py 83.33% 1 Missing ⚠️
ccflow/exttypes/frequency.py 90.90% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #207      +/-   ##
==========================================
+ Coverage   95.22%   95.33%   +0.10%     
==========================================
  Files         140      142       +2     
  Lines       10354    10714     +360     
  Branches      599      613      +14     
==========================================
+ Hits         9860    10214     +354     
- Misses        369      374       +5     
- Partials      125      126       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Move heavy module-level imports out of the import chain triggered by
`import ccflow`. Only the import sites are deferred; runtime behaviour
is unchanged.

* exttypes/frequency.py: drop module-level `import pandas as pd` and
  `from pandas.tseries.frequencies import to_offset`. Import them
  locally inside `Frequency.offset`, `Frequency.timedelta`, and
  `Frequency._validate`. Replace the module-level `_PD_GE_22`
  constant (which read `pd.__version__` at import time) with a
  `@cache`d `_pd_ge_22()` function called from
  `_normalize_frequency_alias`.
* utils/chunker.py: move `import pandas as pd` into
  `dates_to_chunks`.
* validators.py: move `import pandas as pd` into `normalize_date`
  and `normalize_datetime`.
* base.py: drop module-level `import omegaconf` and
  `from omegaconf import DictConfig, OmegaConf`. Add a
  `TYPE_CHECKING` import for `DictConfig` (so static type-checkers
  still resolve the annotations) and quote the runtime `DictConfig`
  annotations on `ModelRegistry.load_config`,
  `ModelRegistry.create_config_from_path`, and the
  `_ModelRegistryLoader` methods. Add local imports of
  `DictConfig`/`OmegaConf`/`omegaconf` inside
  `_is_config_subregistry`, `_make_subregistries`, and
  `_ModelRegistryLoader.load_config` where they are actually used at
  runtime.

Effect on `import ccflow`: pandas, omegaconf, and antlr4 are no longer
in `sys.modules` after a bare import. `-X importtime` total drops
from ~525 ms to ~320 ms (~38% faster) on Python 3.11.

Signed-off-by: Pascal Tomecek <pascal.tomecek@cubistsystematic.com>
@ptomecek ptomecek force-pushed the pit/simplify_deps branch from 1dc6546 to 22e4ca4 Compare May 11, 2026 18:57
@ptomecek ptomecek marked this pull request as ready for review May 11, 2026 18:59
@ptomecek ptomecek changed the title Trim ccflow deps and defer pandas/omegaconf imports Defer pandas/omegaconf imports May 11, 2026
@timkpaine timkpaine merged commit 43eb0c5 into main May 11, 2026
11 of 12 checks passed
@timkpaine timkpaine deleted the pit/simplify_deps branch May 11, 2026 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants