Architecture

Abacus is structured so that the public MMM API stays small while the implementation can evolve behind stable seams. The most important rule is that PanelMMM is a facade, not the place where new core behaviour should accumulate.

For the complete module map, read ARCHITECTURE.md in the repository root. This page summarises the parts that matter most when you are deciding where to put new code.

Design Principles

1. PanelMMM stays thin. Constructor normalisation, data prep, graph construction, prediction, calibration, runtime helpers, and serialisation live under abacus/mmm/models/.
2. Compute comes before presentation. Diagnostics, summaries, and plotting should consume structured outputs from the model layer rather than embedding analytical logic in presentation code.
3. Dependencies flow downward. Shared root infrastructure can be imported by MMM modules, but MMM-specific modules should not leak back into the shared layer.
4. Compatibility is deliberate. If you move imports or rename internals, keep facades or compatibility shims where public usage would otherwise break.

High-Level Package Layers

Where New Code Goes

Dependency Rules

Allowed

• Shared root modules can be imported by MMM modules.
• abacus/mmm/models/ can depend on MMM primitives and shared root modules.
• Facades such as panel.py can depend on the extracted panel modules.
• Plotting, summaries, diagnostics, and optimisation can depend on model outputs and extracted helpers.

Avoid

• Importing panel.py from abacus/mmm/models/*.
• Adding plotting or summary logic to core model-building modules.
• Adding MMM-specific behaviour to the shared abacus/modeling/ layer unless it is genuinely reusable.
• Defaulting to panel.py for new features just because it is visible.

Practical Guidance

When you touch a feature area, check whether there is already an extracted seam for it before adding a new helper. Examples:

• Plot behaviour should usually land in abacus/mmm/plotting/, not in abacus/mmm/plot.py.
• Serialisation changes should usually land in abacus/mmm/models/panel_serialize.py, not directly in PanelMMM.
• Time-varying parameter behaviour should use the HSGP and TVP support modules rather than embedding new logic in plotting or builders.

Before You Merge

• Confirm the change landed in the correct layer.
• Keep public facades thin.
• Preserve public API compatibility unless the change is explicitly breaking.
• Add or update tests in the matching test area.
• Run the local verification commands described in Testing.

Development Setup

This page describes the supported local setup for working on Abacus. The project is maintained with local verification scripts rather than a CI-first workflow, so your development environment needs to be able to run linting, pytest, and the packaging smoke checks directly.

Prerequisites

• Python 3.12
• A local environment manager such as Conda
• A writable temporary directory such as /tmp for PyTensor caches and package verification artefacts

Create the Development Environment

The simplest supported path is the repository environment file:

conda env create -f environment.yml
conda activate abacus-dev
python3 -m pip install -e .

If you know you will be running linting and tests frequently, install the optional extras as well:

python3 -m pip install .[lint,test]

Local Runtime Defaults

Some parts of the stack need writable cache directories. In restricted or sandboxed environments, set the same defaults used by the repo’s local verification scripts:

export PYTENSOR_FLAGS="base_compiledir=/tmp/pytensor,linker=py"
export JAX_PLATFORMS=cpu
export XDG_CACHE_HOME=/tmp

The Makefile already applies these defaults for make test and make smoke_mmm.

Common Commands

Lint and format

make check_lint
make lint
make check_format
make format

These targets cover the package, tests, scripts, and the local verification entry points in sandbox/.

Tests

make test
pytest tests/<path>/test_*.py -v

Use targeted pytest first when you are working on a narrow area. Run the wider verification commands before closing substantial changes.

Local verification

make smoke_mmm
make verify_local
make verify_package
make verify_local_all

What these commands do:

• make smoke_mmm runs a short end-to-end MMM smoke path against the demo config and demo data.
• make verify_local runs the retained local verification matrix from sandbox/run_local_verification.py.
• make verify_package builds package artefacts and validates an installed package smoke path.
• make verify_local_all runs the local verification matrix and includes the packaging smoke step.

Important Working Files

Local-Only Areas

The repo contains some directories that are useful locally but are not part of the shipped library surface:

• .archive/ for archived planning and reference material
• assets/engineering/standards/ for local documentation and writing standards
• sandbox/ for local scripts and verification entry points

Keep temporary scripts in sandbox/ rather than mixing them into the package.

Troubleshooting

PyTensor cache permission errors

If you see errors related to .pytensor lock files or compiledir creation, export the runtime defaults shown above and rerun the command.

Package verification fails because build is missing

Run:

python3 -m pip install build

The make verify_package target does this automatically.

You are not sure which command to run

As a rule:

• run targeted pytest and ruff while iterating
• run make verify_local before finishing non-trivial code changes
• run make verify_package when packaging, imports, or bundled assets changed

Testing

Abacus uses pytest for automated tests, plus local verification scripts for the broader confidence checks that glue linting, smoke paths, and packaging together. The expected workflow is to run targeted tests while you iterate and then run the wider local verification commands before you finish substantial work.

Test Layout

When you change a specific module seam, add or update tests in the matching test area instead of only asserting through a broad end-to-end test.

Core Commands

Fast targeted runs

pytest tests/<path>/test_*.py -v
pytest tests/mmm/plotting/test_theme.py --no-cov -q
pytest tests/mmm/models/test_panel_serialize.py --no-cov -q

Use targeted runs first. They are faster to interpret and make regressions easier to localise.

Whole-suite pytest

make test

This installs the test extras and runs pytest with the local runtime defaults from the Makefile.

Local verification

make verify_local
make verify_local_all

make verify_local is the preferred baseline for substantial changes because it runs:

• Ruff on retained source, tests, and local runner scripts
• byte-compilation checks
• targeted pytest batches across the shared and MMM surfaces
• the local MMM smoke pipeline

make verify_local_all adds the package build/install smoke path.

Packaging smoke

make verify_package

Run this when any of the following changed:

• packaging metadata in pyproject.toml
• import surfaces or compatibility facades
• bundled assets under abacus/
• install-time behaviour or README/package artefacts

Runtime Environment

Some test paths need writable cache directories. The recommended defaults are:

export PYTENSOR_FLAGS="base_compiledir=/tmp/pytensor,linker=py"
export JAX_PLATFORMS=cpu
export XDG_CACHE_HOME=/tmp

The local verification scripts apply these defaults automatically.

Special Cases

Plotting tests

Prefer tests that inspect stable properties such as axes, labels, colours, sizes, rcParams, and return types. Avoid brittle pixel-perfect assertions.

When needed, you can run the top-level plot suite the same way the local verification script does:

NUMBA_DISABLE_JIT=1 pytest tests/mmm/test_plot.py --no-cov -q

Save/load and compatibility work

If you change model serialisation, identity strings, or import compatibility, add tests that prove older saved data or old import paths still work where that compatibility is expected.

Packaging and bundled assets

If you add or move package data, use make verify_package so the change is checked against an installed wheel rather than only the editable repo checkout.

What to Run Before You Finish

Small, localised change

• Targeted pytest
• Targeted ruff check

Moderate code change

• Targeted pytest
• make check_lint
• make smoke_mmm

Broad or risky change

• make verify_local
• make verify_package if packaging or bundled assets changed

Writing Good Tests

• Test observable behaviour, not implementation noise.
• Keep fixtures close to the layer you are testing.
• Prefer additive compatibility tests when preserving old behaviour.
• Use small synthetic data where possible.
• For plotting and serialisation, assert the stable contract rather than fragile internals.