Skip to content

Canonical MRC stack and volume output suffix refactoring

Date: 2026-09-21

Status: proposed; implementation not started.

Validation level: static source inspection only. No source code was changed, compiled, or executed while preparing this plan.

This is the single living design record for this refactor. Update it as the implementation and validation land rather than creating companion plans.

1. Motivation

SIMPLE currently uses .mrc for many outputs regardless of whether the file is a stack of 2-D images or a 3-D volume. This makes generated products harder to recognize and is inconsistent with the useful convention already present in parts of the codebase:

  • a stack generated by SIMPLE has the .mrcs suffix;
  • a volume generated by SIMPLE has the .mrc suffix.

This convention cannot and should not be used to infer the contents of an arbitrary existing MRC file. Dimensions are insufficient: a stack may have equal X, Y, and Z dimensions, and a volume need not be cubic. Existing and third-party MRC headers also do not provide a dependable stack/volume contract for every file that SIMPLE must read.

The refactor therefore changes only names selected by SIMPLE for new outputs. It does not redesign the MRC format, inspect geometry to guess intent, or make input handling stricter.

2. Refactoring contract

For MRC-family output whose filename is selected or constructed by SIMPLE:

Semantic product Canonical suffix
Multi-image stack, including particles, class averages, references, reprojections, and per-class image stacks .mrcs
3-D volume, including maps, half maps, masks, and filtered or reconstructed volumes .mrc
Standalone 2-D image or micrograph .mrc

The following compatibility rules are part of the contract:

  1. Input behavior remains unchanged. A legacy stack named *.mrc must remain readable anywhere it is readable today.
  2. Both .mrc and .mrcs continue to identify the MRC storage format. The existing fname2format mapping remains format recognition, not semantic type detection.
  3. The caller and the I/O operation remain authoritative about meaning. Indexed image I/O and stack_io are stack operations; unindexed 3-D image I/O is a volume operation. The library must not guess from suffix, dimensions, or MRC header fields.
  4. Explicit user-supplied output filenames are preserved. The canonical rule applies to defaults, derived filenames, intermediate products, and other names chosen by SIMPLE. A later warning for an explicit noncanonical suffix is outside this refactor.
  5. Non-MRC output formats retain their existing behavior. In particular, when SPIDER output is selected, both semantic extensions remain .spi.
  6. No existing file is renamed or converted in place. Continuation and import paths must remain able to consume historical names.

This is a naming contract, not a claim that .mrcs makes an MRC file self-describing.

3. Current state

The filename definitions already contain both constants:

The typed parameters object, however, currently has only one ext field. Its initialization and set_img_format select .mrc for the MRC family and .spi for SPIDER. The filename derivation phase then uses that same field for both outstk and outvol in mkfnames. Consequently, the default stack and volume outputs are both given the same suffix.

The repository is already partly migrated. Some producers use STK_EXT or a literal .mrcs, while other stack producers use params%ext, MRC_EXT, or a literal .mrc. params%ext is referenced across commanders, strategies, motion, streaming, extraction, image processing, and tests. Every use must be classified by the product it names; a mechanical replacement would incorrectly rename volumes and standalone images.

The underlying I/O path already treats .mrc and .mrcs as the same storage format. image%read and image%write use the presence of an image index to address a stack member, while stack_io provides an explicitly stack-oriented API. Those semantics should remain unchanged.

4. Target parameter model

Add two typed fields to parameters:

type(string) :: stk_ext  !< extension for generated image stacks
type(string) :: vol_ext  !< extension for generated volumes

Initialize and update them together with the selected image format:

Selected format stk_ext vol_ext
MRC family .mrcs .mrc
SPIDER .spi .spi

Retain ext during this refactor as a compatibility field for code whose product kind has not yet been made explicit and for non-semantic format use. For the MRC family it remains .mrc; for SPIDER it remains .spi. New or migrated output naming code must use stk_ext or vol_ext, not ext.

The fields are derived internal state, not new command-line parameters. They do not require UI registration or parser keys.

Do not add a general is_stack(filename) or is_volume(filename) helper. Such an API would imply certainty the format cannot provide. If a diagnostic helper is later useful, its name and result must make clear that it reports only a suffix convention, not file contents.

5. Ownership and migration rules

The layer that knows what it is producing selects the semantic extension:

  • parameter defaulting uses stk_ext for outstk and vol_ext for outvol;
  • stack-producing commanders and strategies use stk_ext or STK_EXT;
  • volume-producing commanders and strategies use vol_ext or MRC_EXT;
  • project records store the resulting path without reinterpreting it;
  • file and image libraries continue to resolve the physical format as they do today.

Each output site must be classified from its domain meaning and write pattern, not from its dimensions. The audit should cover:

  1. params%ext, self%ext, p%ext, and p_ptr%ext used in output names;
  2. MRC_EXT and literal .mrc used by stack producers;
  3. conventional stack names such as class averages, references, reprojections, particles, and denoised particle products;
  4. producer/consumer pairs for continuation, distributed assembly, cleanup, copying, JPEG generation, and project registration;
  5. UI examples and tests that assert SIMPLE-generated filenames.

Input examples, import filters, stream watchers, and compatibility messages that intentionally accept both suffixes must not be narrowed.

When a conventional intermediate name changes, update its producer and every internal consumer in the same patch. Do not add duplicate .mrc and .mrcs outputs merely to bridge the change. Where continuation needs to discover an older product, prefer the new canonical name and fall back to the historical name explicitly.

6. Staged implementation

Phase Change Exit gate
0 Record an inventory of SIMPLE-owned MRC outputs and classify each as stack, volume, standalone 2-D image, or input-only path. Identify producer/consumer and continuation coupling. Every changed conventional filename has a known producer and consumer set. Ambiguous cases have an explicit domain decision before editing.
1 Add and initialize stk_ext and vol_ext; update set_img_format; change the central outstk and outvol defaults. Retain ext. Default MRC outstk is outstk.mrcs, default MRC outvol is outvol.mrc, and SPIDER defaults remain .spi.
2 Migrate direct parameters%ext output construction according to semantic product type. No known stack output still receives .mrc through the generic parameter extension; volume and standalone-image names are unchanged.
3 Audit hard-coded MRC_EXT and .mrc stack outputs. Update coupled continuation, distributed, project, cleanup, reporting, and preview paths atomically. Every SIMPLE-selected MRC stack output in the inventory uses .mrcs; every renamed output remains discoverable by its internal consumers.
4 Update focused tests, UI examples for generated outputs, and documentation; perform static and user-run workflow validation. The acceptance criteria below have evidence and outstanding runtime checks are recorded here.

Phases 1 and 2 are the central change. Phase 3 provides repository-wide completeness and is where most compatibility review belongs.

7. Risks and mitigations

Risk Level Mitigation
A producer is renamed but a continuation or downstream consumer still looks for the old name. Medium Inventory producer/consumer pairs and update them atomically; add explicit legacy-name fallback only at continuation boundaries.
User scripts depend on historical SIMPLE-generated names such as cavgs_iterNNN.mrc. Medium Document the output naming change prominently. Preserve explicit user filenames and all legacy input support.
A broad replacement changes a volume, mask, micrograph, or standalone image to .mrcs. Medium Classify each output from domain intent and I/O usage; prohibit mechanical repository-wide replacement.
Input filters or imports accidentally reject legacy .mrc stacks. Low but important Leave format recognition and existing dual-suffix input validators unchanged; add regression coverage.
SPIDER output changes unexpectedly. Low Set both semantic extensions to .spi when SPIDER is selected.
Numerical or scientific behavior changes. Low Do not modify image data, headers, algorithms, dimensions, or I/O operations. Compare representative output inventories and hashes where filenames alone changed.

The expected implementation cost is approximately four to seven engineering days for a repository-wide migration and focused validation. The work is mostly an ownership and compatibility audit, not an algorithmic change.

8. Validation plan

Compilation and runtime checks remain user-run unless separately authorized.

Static checks

  1. Search every use of ext, stk_ext, vol_ext, MRC_EXT, STK_EXT, .mrc, and .mrcs in output construction and confirm its classification.
  2. Confirm that fname2format still maps both .mrc and .mrcs to MRC.
  3. Confirm import, watcher, and UI validation paths still accept their existing legacy suffixes.
  4. Trace each renamed conventional product through creation, consumption, project registration, copying, cleanup, and continuation.
  5. Run formatting/syntax diagnostics and git diff --check without compiling.

User-run build and workflow checks

  1. Rebuild SIMPLE and run the nearest parameter, file-I/O, stack-I/O, project, extraction, class-average, and reconstruction tests.
  2. Verify a command with implicit outstk writes outstk.mrcs and reads it correctly as a stack.
  3. Verify a command with implicit outvol writes outvol.mrc and reads it correctly as a volume.
  4. Verify an explicit outstk=legacy_name.mrc is preserved and remains usable.
  5. Continue representative 2-D and 3-D workflows from historical projects whose stack paths end in .mrc.
  6. Run representative extraction, class-average/reprojection, refinement, and distributed producer/assembler workflows and compare the product inventory with the baseline. Apart from intentional stack suffix changes, contents and project associations must be unchanged.
  7. Exercise SPIDER-format output and confirm both stack and volume names retain .spi.

9. Acceptance criteria

The refactor is complete when:

  1. Every MRC stack filename selected by SIMPLE uses .mrcs.
  2. Every MRC volume filename selected by SIMPLE uses .mrc.
  3. Standalone MRC images and micrographs retain .mrc unless their owning workflow explicitly defines them as multi-image stacks.
  4. Explicit user-provided output filenames are not silently rewritten.
  5. Existing .mrc stacks and .mrcs stacks remain accepted on all previously supported input paths.
  6. No stack/volume decision depends on dimensions, filename inference, or MRC header interpretation.
  7. No image data, header-writing behavior, numerical algorithm, or project schema changes as part of this work.
  8. Continuation, distributed handoff, project registration, cleanup, and UI reporting follow every renamed canonical product.
  9. Static validation is recorded here, and outstanding compilation/runtime checks are stated without claiming they passed.

10. Non-goals

  • Making arbitrary MRC files self-describing.
  • Rejecting a stack because it ends in .mrc, or a volume because it ends in .mrcs.
  • Inferring semantic type from dimensions, ISPG, MZ, or other header fields.
  • Renaming or converting existing user data or project files.
  • Changing MRC header-writing semantics or image/stack I/O APIs.
  • Introducing new CLI parameters or a strict suffix-validation mode.
  • Combining this naming migration with numerical, performance, or workflow refactoring.

As implementation phases land, update the status, inventory decisions, validation evidence, and outstanding user-run checks in this document.