Refine3D Policy¶

This document records the current policy for the base refine3D command. It does not describe the staged abinitio3D controller, the class-average abinitio3D_cavgs initializer, or the automated wrapper refine3D_auto.

Related workflow policies:

1. Scope¶

refine3D is an iterative projection-matching workflow over an existing project and one or more starting 3D references. Its durable contract is:

initialize project state and execution mode
materialize the iteration reprojection model from current Cartesian volumes
optionally run probabilistic pre-alignment
run particle-domain search and hard assignment
write partition-local Cartesian reconstruction inputs
run volume assembly when volume reconstruction is enabled
persist project state for the next iteration or caller

The supported reconstruction path is Cartesian. Cartesian matching still uses polar Fourier central sections internally, but those sections are generated from the current Cartesian volumes. refine3D does not own a separate non-Cartesian reconstruction branch.

Particle-domain work stays in the probabilistic pre-step, matcher preparation, search strategies, pose/state/shift updates, sigma updates during Euclidean matching, and partial reconstruction writing. Volume-domain work stays in volassemble and the volume postprocessing helpers it calls.

Shared-memory and distributed refine3D must preserve the same scientific workflow and artifact contracts. Only toolbox lifetime, process launch, and scheduling differ.

2. Entry Points and Execution Mode¶

The public command is refine3D, registered in simple_ui_refine3D.f90 and routed by simple_exec_refine3D.f90.

simple_commanders_refine3D.f90 performs the thin command-level setup:

validate multi-volume input against nstates
set local defaults such as mkdir, cenlp, oritype, and prg
select a strategy through create_refine3D_strategy
run the iteration loop
delegate finalization and cleanup to the selected strategy

Strategy selection is command-line shaped:

nparts without part selects the distributed master strategy
otherwise the shared-memory strategy is used
distributed workers run the partition command path with part and outfile
the distributed master splits the rows into contiguous partitions that balance the particles with state > 0 (qsys_env%new(..., l_active), split_nobjs_active); with every row active this is the even split. The master merges the partition documents by their own ranges (merge_algndocs), so no consumer assumes a particular split
a worker whose partition samples nothing (fewer active particles than partitions, or none of a class-balanced or probabilistic sample drawn over the whole project) completes an empty transaction: its samplers accept an empty sample only when part is set (allow_empty), and it writes its unchanged sigma2 slice, its range's orientations, zero PCG accumulators and JOB_FINISHED

maxits is the number of iterations to run in the current invocation. which_iter starts at startit, and extr_iter follows the same per-call counter unless supplied by a caller.

Shared-memory refine3D currently rejects continue=yes. Distributed continue=yes resumes from project-carried output volumes, FSC files, and run-local artifacts that match the requested partitioning and sigma mode.

3. Ownership¶

simple_commanders_refine3D.f90 owns the base command entry point and the shared iteration loop. It should stay thin.

simple_refine3D_strategy.f90 owns:

shared-memory versus distributed orchestration
scheduler interaction
iteration counters and convergence checks
probabilistic pre-step dispatch
matcher dispatch
volume assembly dispatch
per-iteration reprojection-model materialization
run finalization

It must not absorb numerical volume postprocessing or assembled-reference construction.

simple_strategy3D_matcher.f90 owns refine3D_exec: consuming the driver-generated reprojection model, selecting and running the search strategy, updating orientations/states/shifts, updating Euclidean sigma estimates during search, and writing partition-local Cartesian partial reconstructions.

simple_commanders_prob.f90 plus simple_eul_prob_tab*.f90 own probabilistic pre-alignment and assignment artifacts consumed by the matcher.

simple_commanders_rec_distr.f90 owns commander_volassemble: reducing partials, restoring even/odd volumes, calculating FSCs, running volume postprocessing, writing state volumes, and updating resolution metadata.

simple_vol_pproc_policy.f90 owns NU-envelope cadence, artifact compatibility, and legacy-density-mask compatibility. The NU setup API owns its spherical mskdiam support invariant.

4. Project and Builder State¶

The project is durable workflow state. The builder owns derived execution state for one parsed command line and must not be treated as durable policy state.

Shared-memory refine3D rebuilds its strategy toolbox each iteration after the iteration command line has settled. Persist any initial project mutations, such as random orientations, sampling-counter cleanup, or state initialization, before the next rebuild reads the project.

Distributed refine3D keeps prototype command lines for worker, probability, sigma, assembly, and postprocess steps. These command lines are execution templates, not independent workflow policies.

Fresh-start checks that decide whether to consume project-carried matching metadata must use the stage interval: which_iter <= startit with continue != yes is fresh, even when startit is greater than one.

Before a matcher iteration writes partition-local Cartesian partials, stale partial reconstruction files for the active states and partitions must be removed.

5. Startup and State Initialization¶

If multiple starting volumes are supplied, nstates must be defined and greater than one. Multi-state startup requires either all vol1..volN inputs or none.

When orientations are absent, refine3D randomizes 3D orientations. When projection indices are absent, distributed initialization sets them from the Euler sampling.

For multi-state distributed startup:

existing state assignments must match the requested nstates
fresh state labels are randomized uniformly, whether complete starting volumes are supplied or the project references are used
probabilistic multi-state startup requires each state to exceed the minimum population guard

Even/odd partitioning is required for consistent half-map reconstruction. If it is absent, initialization partitions the active orientation segment and writes that state back to the project.

5.1 Sigma2 bootstrap contract¶

Work that has alignments, or nothing at all, but no noise-power estimate is one problem: a fresh abinitio3D start, the ini3D routes, external starting volumes, the refine3D_auto startup, a refine3D started at a later iteration in an empty directory, and every final reconstruction at a new sampling. Since 2026-09-06 all of them follow one rule, owned by simple_sigma2_bootstrap:

Seed canonical state from the particle image power spectra (calc_pspec), and let the first euclid pass replace the seed with residual sigmas. Image power needs neither alignment nor volume, and it is the basis every refinement already starts from. Half-map power (the former bootstrap_rec3D estimator) sat on a different basis than the residual sigmas a refinement then computes and conditioned the euclid system markedly worse; it is retired.
ensure_sigma2_for_iteration is a no-op when the project already owns a compatible committed state and seeds otherwise. The consuming refine3D reads that state directly; the external-reference pose initialization uses the same transaction path after its CC residual pass.
Refine3D is self-healing: a missing, invalid, wrong-grid, wrong-layout, or wrong-grouping state is rebuilt from particle power before workers launch.
Where no refinement iteration follows (final reconstructions at original sampling in abinitio3D and refine3D_auto), the seed is upgraded by one residual pass: refine=sigma against the seeded map at the final sampling (no search, no volume assembly, no orientation output, alignment docs are not merged, and the particle field left exactly as it was: shared-memory initialisation does not reset updatecnt/sampled for it, the run finalizers do not write the field, and the distributed master refreshes it from disk before its whole-project write), committed as the next canonical generation, then the shipped euclid ML reconstruction runs on the residual sigmas. Since 2026-09-07 bootstrap_rec3D (module simple_commanders_refine3D) owns this whole sequence: seed, bootstrap map, residual pass, commit, final map; the shared ending calc_final_rec (module simple_final_rec, 2026-09-12), which closes abinitio3D, refine3D_auto, refine3D_states and classify3D_refs, calls it and carries no copy of the sequence. It runs standalone on any project with 3D orientations and is the test entry point for the final-reconstruction stage. The bootstrap map only serves as the residual reference, so it is always a gridding assembly with ML regularization (one particle pass, no postprocessing) that keeps the workflow's filt_mode and automsk: the residual sigmas depend on the regularization of the reference they are scored against, so that reference is regularized exactly as the refinement's matching references were; the shipped map keeps the caller's backend and automsk (so on PCG it is estimated on the same density-envelope support as the refinement, 2026-09-09; filt_mode=none keeps it classical) and, on PCG, starts from nothing at the native box and therefore gets the cold-solve budget of at least FINAL_PCG_MAXITS_FLOOR (5) iterations whoever the caller is (2026-09-07). The base plus ML solve pair of that final PCG reconstruction is inherent to ML regularization: the prior is built from the base pair's independent-half FSC, which is also the reported FSC and the unfiltered pair postprocessing uses.
The residual pass is not an update (2026-09-27). Before this, the shared-memory run finalizer wrote the particle field with the pass's sampling bookkeeping: every active particle came back with updatecnt + 1, a new sampled round and cleared search statistics (poses untouched). So particles never updated by the refinement entered the shipped map, unlike the bootstrap map (sample4rec takes updatecnt > 0 rows), and counted as updated for update_missing and for the frozen membership of abinitio3D_addon; a standalone bootstrap_rec3D at which_iter=1 (startit=1) even reset updatecnt/sampled for every particle first. Nothing reads the bump: the pass only closes final reconstructions, and a later refinement starts its own sampling round.
The unfiltered pair belongs to the assembly that wrote the map it accompanies (2026-09-12): every gridding volassemble writes the _unfil halves, the unregularized pair under ML regularization and copies of the halves otherwise, so postprocess_nu always finds a current pair; and postprocessing ignores a pair whose box differs from the map. A final reconstruction at native sampling therefore never reads the cropped pair left behind by the last refinement block.
The registration-box rule is the same for both stores (2026-09-07): a final reconstruction whose registration (crop) box differs from the native box refreshes the sigmas at native sampling (image-power seed, bootstrap map, residual pass) before the shipped map. Before this the canonical store reused its committed state whenever it was structurally consumable, which is always true at the native box, and so skipped the refresh legacy performs; the two stores could not produce the same final map.

6. Reference Preparation¶

The strategy materializes the iteration reprojection model before probabilistic pre-alignment or matcher work starts. Reference preparation:

reads the current Cartesian reference volumes
applies reference masking/filtering/centering policy
determines the active high-pass/low-pass shell range
writes even/odd PFTC reference sections for the matcher

When automsk=yes, matching references are NOT multiplied by any envelope (2026-09-02): the NU-evidence envelope acts only through the NU filter field, whose background (the envelope complement) takes the coarsest bank candidate, so the excluded density reaches the reference heavily low-pass filtered rather than removed. The matcher applies the spherical soft reference mask only; there is no separate envref control. automsk=yes also implies envfsc=yes (policy 2026-09-09), so FSC evaluation follows the density envelope (post hoc with solvent correction on gridding; inside the estimator on PCG, reported as >>> FSC MODE). Particles and matching-bandwidth selection are unchanged.

This path is valid only with filt_mode=nonuniform|nonuniform_lpset. automsk=yes with none, uniform, or fsc is rejected during parameter validation. automsk=tight is also rejected in NU refinement because the NU-evidence envelope has no Otsu tightness mode.

The lifecycle is lag-by-one: volume assembly from iteration N writes the NU envelope used when iteration N+1 materializes its reprojection model. A missing mask at the initial bootstrap is therefore expected.

volassemble must not generate or promote matcher reprojections. The next iteration's reprojection model is generated by the strategy from the current Cartesian outputs.

The model file header is the authority for the matching shell range consumed by matcher workers and probabilistic table workers.

Reference preparation must project only the active matching shell range. It must not silently project to the crop interpolation limit unless matching policy explicitly asks for that range.

The 3D low-pass range comes from one of these sources:

explicit LP-set lp
project-carried NU-selected handoff in a continuing nonuniform refinement
current FSC files
an existing project lp field

lpstop caps the selected matching bandwidth. The crop Nyquist limit caps the final Fourier index after the matching policy has selected a candidate range.

7. Matching Topology¶

Gold-standard matching keeps even and odd references independent.

If low-resolution even/odd docking is needed for registration, it is applied only after reference read/mask/filter and immediately before PFTC generation. Those blended references are not written as half-map outputs and do not feed FSC, automasking, NU filtering, or ordinary half-map handoff.

LP-set matching uses a merged registration reference. State count alone must not force merged-reference matching; topology is controlled by LP-set policy. filt_mode=uniform is disabled for multi-state search until it has a state-specific low-pass contract.

In nonuniform mode, reference loading follows the NU policy:

plain nonuniform prefers independent _nu_filt even/odd references and falls back to regular even/odd references before using a merged map
nonuniform_lpset with active LP-set matching uses the merged reference and prefers the merged _nu_filt product when present. In both modes the matching band is the finest member of the NU bank, the finest rung of the ladder cut at fsc/1.5 or the regularized pair once it is at or beyond the ladder's finest rung (nonuniform_filtering_policy.md sections 8 and 12, 2026-09-19)

The ordinary reference low-pass filter is not applied on top of a NU reference path. NU and ML-regularized references are treated as already filtered during volume assembly.

8. Probabilistic and Matcher Work¶

The current workflow is probabilistic pre-alignment followed by hard particle assignment. It is not a monolithic soft-assignment volume-integrated EM implementation.

Use the terms "probabilistic pre-alignment" and "hard-assignment particle update" unless the ownership, artifacts, and update model actually change.

refine=prob* modes run the probability pre-step before the main matcher. prob_neigh dispatches to the neighborhood probability command. The matcher then consumes the generated assignment artifact.

prob_neigh_mode controls how prob_neigh chooses candidate neighborhoods before evaluating candidates:

state: score coarse subspace representatives independently per state, pool the selected neighborhoods across states, and evaluate the same pooled projection search space for every active state.
geom: select the subspace containing the particle's current projection for every active state, with no coarse scoring or pooled peaks.
shc: use the direct stochastic candidate path. When shifts are enabled, it first estimates a shift seed before candidate scoring.
snhc: use the same direct stochastic candidate path without the initial shift seed. It is consequently also the zero-shift default for this mode.

Probabilistic-table assignments use the calibrated likelihood path. The stored distances are noise-normalized negative log-likelihoods for the Euclidean objective, and evaluated candidates are sampled with weights proportional to exp(-dist) over an explicit top-K support. The implementation uses a per-particle minimum shift before exponentiation for numerical stability; this does not change normalized weights.

The top-K truncation is deliberate. It defines the local discrete support actually evaluated by the pre-alignment step; it is not meant to represent a full posterior over all SO(3) grid points. If the CC objective is enabled, its existing likelihood-compatible transformation remains the source of the probability weights, but should not be described as a calibrated Gaussian likelihood.

Likelihood-weighted probability-table modes may still profile or MAP-refine shifts, and sometimes in-plane rotation, after stochastic candidate selection. The stored assignment distance is then the refined/profiled objective value. This is intended current behavior, not a full soft-assignment EM update.

Continuous in-plane policy¶

inpl_cont has exactly two values. no preserves the historical alternating search: continuous shift optimization with the discrete in-plane angle callback. yes is the default and follows the polish-only principle: continuous refinement changes pose precision, never search behavior. Selection sees exactly what the legacy route sees -- candidate scoring, probability-table profiling, shift-seed estimation, and multi-peak shift refinement all use the legacy discrete machinery unchanged -- and the joint raw-Euclidean (sx,sy,rotind_frac) optimizer runs exactly once per particle per iteration, after selection, to polish the committed assignment. This makes the search trajectory identical to the validated legacy trajectory by construction; the continuous route is a strict refinement of it and cannot alter exploration dynamics on any sample. Objective type is a capability check and must never activate continuous-angle behavior by itself.

Raw objfun=euclid, objfun=cc, and the objfun_den=yes hybrid provide the analytic joint (sx,sy,rotind_frac) gradient. The cc route minimizes -cc with a quotient-rule angular derivative and maps scores as the clamped correlation rather than exp(-loss). The hybrid route minimizes the negative of the established weighted score: raw exp(-loss) plus clamped denoised correlation. Parameter validation does not couple inpl_cont to objfun, objfun_den, ptcl_src, projrec, or a specific program. Objective capability is owned by the PFTC/search implementation. The opt-in is not restricted to refine=shc: deterministic, neighborhood, evaluation, and probabilistic matcher routes use the same policy wherever they commit a pose. A mode with no pose search, such as sigma-only setup, naturally invokes neither optimizer. Unsupported capability combinations fail validation rather than silently reverting to the callback. The joint cc route is validated on nanoparticle data as well, so the nano 2D/3D workflows follow the inpl_cont=yes default.

The continuous in-plane sub-suite of the fast gate (simple_test_exec test=unit_pftc_align2D3D, simple_pftc_inplane_tester; it replaced simple_test_continuous_inplane_hybrid_grad in September 2026) guards the hybrid route with an integer-grid score identity, finite-difference checks of the derivative components, and construction of the production joint optimizer.

Probability tables are pure legacy under both inpl_cont values: candidate scoring, shift-seed estimation, and per-candidate shift refinement use the legacy discrete optimizer, and the table schema remains discrete (rounded inpl, score, shift in the rounded-index frame). The joint optimizer never touches table construction; polishing candidates would sharpen the score contrast the stochastic sampler sees and collapse exploration on difficult samples (statistics parity below addresses the same failure through the convergence side).

After hard state/projection/in-plane assignment, the matcher runs the joint three-parameter optimizer locally for the selected pose -- the single joint solve of the iteration. The assignment index is authoritative: it converts the stored shift back to the native particle frame and seeds a solve bounded to plus or minus two in-plane cells, without another global all-angle selection. An accepted result persists one coupled pose: fractional Euler e3, nearest integer inpl, shift, and score. A finite non-improving evaluation retains the incoming discrete pose with its re-scored objective value. A numerically invalid evaluation leaves the incoming assignment untouched. No inpl_cont=yes path falls back to the legacy callback. State and projection identity remain fixed during the final solve. A persisted fractional e3 is rounded to the canonical integer cell before the next search.

Statistics parity. Every statistic that steers search control -- dist_inpl, angular distances, class/projection overlap, and anything the convergence checker reads -- is computed from the DISCRETE cells on both sides of the comparison, never from fractional poses. Sub-grid pose precision would otherwise read as premature convergence and truncate the annealing schedule that difficult samples require. The fractional e3 and polished shift are carried for reconstruction and persistence only.

Joint acceptance is guarded twice, in the shared optimizer. An improvement must be material -- it must beat the seed cost by a relative tolerance far above solver roundoff -- otherwise the solve counts as non-improving and the seed pose stands; this keeps the improved fraction an honest diagnostic and prevents noise-level "improvements" from committing fractional poses. A solution pinned to any joint search bound (the shift box or the plus-or-minus two-cell angular window) is likewise demoted to non-improving: an angular-bound-pinned solution contradicts the exhaustive seed selection, and a shift-corner solution is the signature of descent into truncated-series artifacts rather than signal.

Convergence dist_inpl reporting is symmetry-aware. When multiple symmetry operations have the same Euler distance within numerical tolerance, including projection-preserving operations in dihedral groups, the reported in-plane distance uses the equivalent operation with the smallest in-plane change. A symmetry-equivalent branch switch must not appear as a spurious 180-degree in-plane move.

Shared-memory and distributed probabilistic child commands must retain inpl_cont; reconstruction, sigma, assembly, and postprocessing children must not receive matcher-only search policy.

For multi-state alignment (nstates > 1), shift-first candidate scoring is disabled. The matcher and probability-table paths may still refine shifts after candidate/state selection, but they must not use a shift seed estimated from a particle's previous state to score candidates in other states.

The matcher must preserve a single particle-stack read per batch within each phase: batch construction reads each particle once for search, and the reconstruction phase reads each selected particle once after the search-phase teardown (the PFTC memory phase boundary makes retaining the raw images across phases prohibitive). Do not add further reads to either phase unless the performance contract is explicitly changed.

The downscaled particle cache is a 2D-only feature: refine3D rejects cache=yes and both phases always read the original full-size stacks (see doc/policies/particle_cache_policy.md).

9. Volume Assembly¶

When volrec=yes, matcher workers write partition-local Cartesian partials and the strategy dispatches volassemble.

volassemble:

reduces partition-local partial reconstructions
restores dense even/odd half-volumes, deapodizes them and applies the soft spherical support at msk_crop (identical to the PCG solve support) before writing; the merged volume gets the same support; the support is recorded in the <vol>_pcg_support.txt sidecar so downstream consumers never mask again (2026-09-09)
calculates FSC curves and state resolutions on the halves as shipped
applies conical FSC curves for directional ML regularization when ml_reg=yes and conical_fsc=yes; this is opt-in
calculates conical FSC and cFAR from copies of the shipped halves; with envfsc=yes the copies are additionally masked by the on-the-fly density envelope low-pass filtered at envmsklp, and the phase-randomized radial FSC uses the shipped half-volumes to determine its 0.8 randomization onset. No second spherical mask is applied anywhere in the FSC evaluation. envmsklp defaults to ENVMSKLP_DEFAULT (20 A) and is separate from the amsklp NU-evidence smoothing scale
writes automask3D_stateNN.mrc when envfsc=yes; the same density envelope is available to compatible non-PCG final postprocessing, but is never multiplied into a matching reference
restores merged state volumes
derives the NU-evidence envelope from the completed accepted NU bank before unary storage is released
writes derived NU reference products (_nu_filt, _nu_locres) on both backends through the shared simple_nu_state_filter competition
records resolution and NU matching metadata in the project: per-particle res (state FSC=0.143 resolution) and res05 (state FSC=0.5 resolution), the raw NU matching handoff in lp (clipped against lpstop by the matcher before use) and, when NU filtering is active, the same NU-estimated limit in lp_est. The convergence readout reports RESOLUTION @ FSC=0.143, RESOLUTION @ FSC=0.5, the band this iteration actually matched at (MATCHING LOW-PASS LIMIT (THIS ITERATION), from params%lp/kfromto(2) as fixed by set_bp_range3D before the search -- an explicit lp or the previous handoff) and the handoff for the next iteration (NU HANDOFF LOW-PASS (NEXT), the project lp field assembly writes after the search; 2026-09-14, after an explicit lp=3.6 run reported the handoff as the matching band), per state when nstates > 1, and persists them as RESOLUTION, RESOLUTION_FSC05, LP_MATCHING (the matched band), LP_NU_HANDOFF, LP_ESTIMATED (plus _STATEnn variants) in the iteration stats. Both backends write the same fields (2026-09-06). res05 occupies fixed particle-record slot 50 (I_RES05, the former spare I_EMPTY10): the binary project stores particles as fixed 50-float records, so a key without a slot never reaches disk

Volume assembly does not refresh matcher PFTC references. It only produces Cartesian volumes and metadata for the next iteration.

The shared helper restore_state_from_parts owns the restoration sequence. Changes to ML ordering, sampling-density correction, FSC handling, trailing reconstruction, low-resolution even/odd insertion, or LP-set behavior must go through that shared helper rather than duplicated assembly branches.

FSCs and FSC-derived diagnostics are calculated from dense Cartesian half-volumes, not from sparse, intermediate, or low-resolution-blended registration-reference representations.

9.1 PCG reconstruction backend¶

rec_backend=gridding|pcg selects the reconstruction backend; gridding is the unchanged default and protects all existing workflows. The PCG solver contract, output convention, ML two-map behavior, warm starts, and diagnostics are owned by reconstruct3D_pcg_policy.md; this section records only the refine3D-side integration contract:

Solver controls are distinct from refinement controls. maxits counts refinement iterations; the PCG solve budget is maxits_pcg (default 2, capped at 8 in production) and rtol is PCG-specific (rtol <= 0 demands exactly maxits_pcg iterations).
The backend seam is dense even/odd half maps. volassemble dispatches to a backend-specific state restorer: gridding reduces (cmat,rho) and sampling-density-corrects; PCG reduces raw (B,D), finalizes the kernel and solves. FSC/cFAR diagnostics, filenames, and project updates share policy helpers, and both backends synthesize _nu_filt references through the same assembly-owned NU competition (base pair = the unregularized _unfil halves, auxiliary member = the P_tau-regularized halves). PCG maps never receive gridding correction, a second sampling-density correction, or a post-hoc mask.
Raw statistics boundary. Workers accumulate and atomically publish raw, unregularized (B,D) per (state,half,part); only the master folds, finalizes, regularizes, and solves, reducing parts in ascending order.
Sigma ordering. For objfun=euclid, iteration n scores and reconstructs with the sigma model committed before the iteration began. It writes new residual estimates into a pending transaction, completes assembly, and only then publishes them for iteration n+1. A stage-owned reconstruction such as abinitio3D symmetry consumes the same lagged model before the final transaction is published. objfun=cc is unweighted.
Weights versus priors. Particle/data weights (including 1/sigma2) multiply both B and D. The zero-mean ML prior adds precision to the normal operator and preconditioner only; it never weights B, and nothing prior-related is persisted in raw statistics.
FSC ownership. The FSC comes from the unregularized _unfil base pair and remains the resolution authority. The regularized pair is the closed-form P_tau optimum of the base pair on the replayed operator (voxelwise, shrink_by_ml_prior, 2026-09-14; the replay solve is retired); NU filtering is applied afterwards by the shared competition, never inside the solve (policy 2026-09-06).
Solve support. automsk=yes uses the conservative density envelope; automsk=nu prefers the lagged NU-evidence envelope and falls back to density; an explicit pcg_mskfile remains an override. The selected mask constrains both base and replay and envfsc=yes is implied. PCG reports FSC on the constrained pair without post-hoc masking or phase randomization, logged as >>> FSC MODE. Before either mask source exists, the base bootstraps on the sphere and its current pair supplies replay density.
Current exclusions (hard-errored, not approximated): projrec=yes, conical_fsc=yes, and matrix-free workflow execution. Fractional/trailing reconstruction is implemented in the distributed master path.
New regularization is research, tracked in doc/implementation_notes/pcg_priors_history.md; it cannot be used to close integration gates.

10. Trailing and Combined Even/Odd¶

Trailing reconstruction blends even with previous even and odd with previous odd. It runs before NU-evidence envelope generation and derived reference filtering.

In LP-set mode, if trailing is active, assembly trails the even and odd half-volumes first and then merges the trailed halves. Assembly must not use low-resolution even/odd docking insertion for LP-set outputs.

When combine_eo=yes, distributed refine3D schedules one additional final iteration after convergence or run-length termination. That final iteration:

disables fractional update
forces full update
sets combine_eo=yes
tightens the low-pass criterion to at most 0.143

The combined even/odd iteration is part of base refine3D, not a terminal refine3D_auto or ab initio reconstruction step.

11. Finalization and Artifacts¶

On each iteration, strategy benchmark files should stay simple: context plus one TIMINGS (s) section. Labels should include the coarse operation buckets setup, probabilistic pre-step, matcher/scheduler, assembly/postprocess, and total time, together with separately timed reprojection-model materialization and group-sigma consolidation. These two measurements keep a per-iteration sigma update distinguishable from reference preparation.

Each refine3D stage also writes one stage-entry benchmark at its first iteration. It records the total stage initialization wall time and the nested calc_pspec wall time. The latter is the per-particle sigma estimation that may be reused across compatible stage changes; it must not be inferred from a per-iteration setup bucket.

Base objfun=cc refinement still does not bootstrap or consume sigma2. The external-reference wrappers are the explicit exception: their shared service runs calc_pspec before the fixed-reference CC pass, preserves that native-grid partition state outside the initialized cohort, and replaces active cohort shells with residual sigma2 after committed assignments.

Distributed matching writes partition alignment documents and merges them into the project after worker completion. Shared-memory matching writes the project directly after each iteration.

After the current iteration's committed orientations are available, both strategies attempt to write orientations_distribution_stateNN.jpg for each nonempty state. This projection-direction distribution is a secondary, latest-view visualization, generated and overwritten at every iteration. It is not authoritative scientific state, is not registered in os_out, and is not a restart or workflow-handoff artifact. An empty state may produce no new image. Failure to write this JPEG warns but does not invalidate the refinement iteration.

On finalization:

endit is written to the command line
startit is removed from the command line
active state volumes and FSCs are registered in os_out
empty states are removed from os_out
cls3D distributed runs map class-orientation output back to particles
JOB_FINISHED is touched by the shared-memory path

The original-sampling final reconstruction is distinct from an ordinary refinement iteration. On the PCG backend, both abinitio3D and refine3D_auto apply the shared minimum five-iteration budget to this cold solve. An explicit positive residual tolerance may still stop convergence earlier. Final automatic sharpening follows the isotropic postprocess protocol (2026-09-21, the postprocess_nu v2 recipe with one cutoff): the cutoff is the FSC=0.143 of the reconstruction's FSC file (the base pair's curve, envfsc-corrected where that applies), or of the _even_unfil/_odd_unfil pair beside the map computed by postprocess when no file is given, one Guinier B-factor of the unfiltered pair average between HPLIM_GUINIER (10 A since 2026-09-26, RELION's --autob_lowres default and Rosenthal & Henderson's lower limit; 20 A before, which put the steep envelope/micelle region into the fit) and that cutoff (never of the shipped regularized map: its prior's amplitude suppression steepens the slope, -150 on streptavidin against -77 to -83 from unregularized maps), sharpen, the FSC weighting exactly once, then the Butterworth low-pass at the cutoff. Exactly once (2026-09-26): an ML-regularized map (support-provenance solve_kind=regularized, or gridding_regularized from a gridding reconstruction with ml_reg=yes) already carries it -- its prior shrank every Fourier component by rho/(rho + <rho>/(tau SSNR_half)), about FSC per shell -- and is only B-sharpened and Butterworth-filtered; every other map (the imgkind=unfil|solvent pair averages, base or mixed PCG solves, unregularized gridding maps, foreign maps) gets RELION's weighting sqrt(2FSC/(1+FSC)) (Rosenthal & Henderson's C_ref, postprocessing.cpp applyFscWeighting, zero from the first shell with FSC < 1e-4). The 2026-09-22 recipe applied the Wiener 2FSC/(1+FSC) on top of the regularized map's own shrinkage, leaving 0.036 of the amplitude at FSC=0.143 (0.33 at 0.5): every map over-smoothed. Its trigger -- exp_gate a cloud of structured noise under the Butterworth alone at an automatic B of -108 while -50 looked right -- was a defect in the closing filter (2026-09-27): the Butterworth array was sized to the box, so apply_filter reached the corners of the Fourier cube (out to sqrt(3) x Nyquist), where apply_bfac's exp(-B s^2/4) is largest, with only the Butterworth's k^-8 tail against it. At exp_gate's 0.822 A/pixel the corner gain is ~1e8 at B -108, ~1e5 at -80, ~4e2 at -60 (msp1 at 1.073 A: ~4e4 at -115); the noise sits outside the molecule because there is no density there to hide it. Any filter sized to the FSC zeroes the corners, which is why the FSC weighting and the NU sharpening never showed it. The Butterworth now covers the Fourier shells up to Nyquist only, and the sharpening is capped at the cutoff shell (2026-09-28): every shell beyond the cutoff keeps the cutoff's gain exp(-B s_c^2/4) and the Butterworth alone rolls it off (bfac_cap_filter), since there is no signal beyond the cutoff to restore and at fine pixels exp(-B s^2/4) otherwise outgrows the order-8 Butterworth from ~1.5 x the cutoff on (exp_gate's Nyquist shell: x45 at B -108 uncapped). RELION differs in two places kept on purpose: it fits B on the FSC-weighted map out to where the FSC reaches zero (steeper) and applies no low-pass by default; SIMPLE fits the unweighted unfiltered pair up to the FSC=0.143 cutoff (a conservative B) and closes with the Butterworth there. No density-windowed pair estimate. With pcg_solvent=yes the _unfil pair is the prior-free base pair and the sharpened map is the shipped (prior'd, replayed) map; imgkind=unfil and imgkind=solvent postprocess the respective pair averages with the same cutoff.

Grouped sigma files are run-local noise-model state. They may be written and consumed inside a running refinement, but they are not registered as os_out handoff artifacts for a later refine3D execution.

12. Refactor Rules¶

Preserve the particle-domain versus volume-domain boundary.
Keep shared-memory and distributed workflows behaviorally aligned.
Keep volume assembly as the owner of assembled-reference work.
Do not move assembled-volume postprocessing into refine3D_strategy.
Do not merge probabilistic particle-update logic with volume postprocessing.
Do not introduce a second ambiguous source of matching references.
Preserve the single image-stack read per matcher batch.
Treat assignment maps, partition-local partials, state volumes, even/odd volumes, FSC files, automasks, and NU outputs as explicit workflow contracts.
Preserve the two-mode in-plane boundary: only new_legacy may attach the callback, and no inpl_cont=yes path may invoke it.
Initialize candidate-profiling joint solves with one callback-equivalent all-angle selection at the supplied shift; do not run the 5-by-5 coarse shift initializer.
Treat the final 3D hard assignment as authoritative: refine locally around its rounded in-plane cell and never perform a second global angle selection.
Keep probabilistic in-plane artifacts rounded and run durable fractional refinement only after final hard assignment.
Keep inpl_cont on shared-memory and distributed matcher/probability child commands while stripping it from non-matcher children.