Step 1 Project Configuration

Required inputs:

• Animal ID — numeric identifier for the animal
• Session ID
• Input directory — folder containing your video files
• Output directory — where results will be saved

Advanced (Dask) Settings: 8 workers, 200 GB memory limit, 100% video percentage by default. Reduce video percentage for testing.

Step 2 Data Preprocessing

2a · File Pattern Recognition

• Use regex to match your video files — the .avi pattern is pre-set.
• Tip: paste an example file path into an LLM and it will generate the correct regex.
• Downsampling is optional if your machine has sufficient resources.
• Line splitting detection is optional — see Advanced Features.

2b · Background Removal & Denoising

Denoising methods:

• Median — best for salt-and-pepper noise, preserves edges well. General-purpose default.
• Gaussian — smooth uniform blur; good when you need everything smooth but can lose sharp detail.
• Bilateral — edge-preserving blur; slower but keeps neuron boundaries sharp.
• Anisotropic — directional smoothing that follows structure shape; great for elongated neurons but slowest.

Background removal methods:

• Tophat — subtracts a morphologically opened version of the image. Robust and works well for most data.
• Uniform — rolling average subtraction; faster, best when background changes are smooth and gradual.

2c · Motion Correction

• Keep motion estimation on "frame" (default).
• This is one of the longer steps — expect ~3× your video size in RAM. For very large datasets, Step 7e may take comparable or longer time.
• Algorithm uses recursive estimation with phase correlation.

2d · Quality Control

• Threshold factor: how many standard deviations above the mean motion must be before a frame is dropped.

2e · Data Validation

• Keep fill value as zero (for NaNs). Keep all other options checked. Validates transformed data integrity.

2f · Preview Results

• Processes a subset of data — saves only statistics, not full output. Use as a gut-check before committing to the full run.

Step 3 Spatial Cropping & Initialization

3a · Define ROI

• Keep the crop as tight as possible — this directly impacts processing speed throughout the pipeline.
• Test on 10% of the video first, then return with the full video.
• Use a circular crop centered on your imaging field. Adjust offset if the field is not centered.

3b · NNDSVD Initialization

Think of NNDSVD like a conductor listening to an orchestra warm up — instead of isolating each instrument one by one, it instantly identifies the major sections from the combined sound. NNDSVD does the same: it decomposes the full video into a compact set of spatial components rapidly, giving the CNMF a strong starting point.

What it's actually doing: Non-Negative Double Singular Value Decomposition — a fast matrix factorization that expresses your video as a product of three simpler matrices. Unlike PCA, SVD reaches a near-equivalent answer much faster on large datasets. The NN ("non-negative") constraint adds the biological prior that fluorescence can't be negative, encouraging spatially compact, blob-like components.

Parameters:

• Number of components — Component zero typically captures background variation (the first singular vector reflects the dominant variance, which in miniscope recordings is usually neuropil/background fluorescence). Components 1+ are candidate neural signals, ordered roughly by variance explained. Even 20–50 components typically capture 99%+ of variance in a sparse brain region.
• Power iterations — refines the SVD (5 is usually sufficient).
• Sparsity threshold — (0.05 = keep signals 5% above noise floor). In regions with fewer neurons and lower overall activity, a lower threshold helps avoid discarding dim but genuine signals. In denser regions, a higher threshold excludes noise.
• Spatial regularization — encourages blob-like components. Increase for larger, spread-out neurons; decrease for very small, compact ones.
• Chunk size — decrease for less memory use at the cost of slower processing.

3c · Early Analysis Option

Some workflows stop here for preliminary results. The variance-explained plot shows cumulative % of data variance captured as components are added. For most miniscope datasets, 99%+ is typically accounted for within the first few components. You don't need 100% — the final fraction of a percent usually contains noise. If you're at 95–99%, your initialization is in good shape.

Step 4 Component Detection

4a · Watershed Parameter Search

Watershed segmentation identifies candidate neurons. Since it's better to overestimate at this stage, this step quickly finds optimal parameters.

• Min distances — minimum pixel distance between neuron centers (e.g., 10, 20, 30 tests different spacings).
• Threshold relativity — how much brighter than surroundings a peak needs to be (0.1 = 10% brighter). Think of this as the elevation cutoff on a topographic map — lowering it detects more candidates, including noise. The CNMF downstream will prune false positives, so over-seeding here is not catastrophic, just slower. If you get consistently fewer components than expected, try lowering this. If you then see overlapping or unclear components in the final output, you over-seeded.
• Sigma values — smoothing before peak-finding (1.0 = sharp, 2.0 = slightly blurry).
• Sample size — components to test (20 is usually enough).

4b · Apply Best Parameters

• Minimum region size — smallest neuron size you'll accept. Better to be permissive here; small components can be filtered later.

4c · Merging Units

Collapses spatially overlapping components that were over-segmented in 4b.

• Distance Threshold — how close (px) two centers need to be to consider merging (25 px is a good start).
• Size Ratio Threshold — won't merge if one component is much larger (5.0 = up to 5× size difference).
• Minimum Component Size — removes anything smaller than this (9 px = 3×3 minimum).

4d · Temporal Signal Extraction

Extracts calcium traces from each spatial component — going from "where are the neurons" to "what are they doing."

• Batch Size — components per batch (10 is safe).
• Frame Chunk Size — frames per load (10,000 works for most systems).
• Component Limit — for testing, process a subset first.
• Memory Management — keep on unless you have unlimited RAM.

4e · AC Initialization

Prepares the spatial (A) and temporal (C) matrices for CNMF. Skip component 0 (background).

• max — normalize to brightest pixel (default, works well).
• l1 — normalize by total brightness; use when component sizes vary a lot.
• l2 — normalize by energy; mathematical option.
• none — raw values; only if you know what you're doing.

4f · Final Component Preparation

Quality control before expensive processing. Remove NaN, empty, and flat components. These will break downstream steps if left in.

4g · Temporal Merging

Final cleanup — merges components that are probably the same neuron split across detections.

• Temporal Correlation Threshold — how similar traces need to be (0.75 = 75% similar).
• Spatial Overlap Threshold — how much components must overlap spatially (0.3 = 30% minimum).

Step 5 CNMF Preparation

5a · Noise Estimation

Estimates per-pixel noise across the field of view. Critical for CNMF — it tells the algorithm which parts of the signal to trust.

• Noise Scaling Factor — in active regions, noise is higher due to shot noise from calcium indicators. Default 1.5 scales up the estimate in bright areas.
• Smoothing Sigma — spatially smooths the noise map to avoid pixel-to-pixel variation (1.0 = gentle).
• Background Threshold — mean (good default), median (more robust to outliers), or custom.

5b · Validation & Setup

Quality control checkpoint before CNMF computation.

• Check for NaN/Inf — these will break CNMF. Always check (may be slow on huge datasets).
• Compute Full Statistics — detailed component stats for troubleshooting.
• Size Filtering — minimum 10 px, maximum 1,000 px. Components outside this range are likely artifacts or merged neurons.

Step 6 CNMF Processing

6a · YrA Computation

Computes YrA — the raw projection of the video data onto each component's spatial footprint. This gives the corresponding fluorescence signal at each timepoint for each neuron, before any denoising or spike inference. This is one of the computational bottlenecks of CNMF.

• Subtract Background — recommended.
• Use Float32 — cuts memory in half with minimal precision loss. Always recommended.
• Fix NaN Values — replace with zeros before computation.

6b · YrA Validation

Sanity check on the YrA computation. What YrA actually means: it is the raw projection of video data onto each spatial footprint — not a measure of variance explained. High YrA at a given time means strong fluorescence in that neuron's footprint at that moment. This projection feeds into the AR/sparsity optimization in Step 6d.

• Units to Analyze — 5 is usually enough to spot issues.
• Frame Selection — random, start/middle/end (highest_variance not yet implemented).
• Number of Frames — 1,000 is sufficient for validation.

6c · Parameter Suggestions for Temporal Update

Analyzes your data to suggest AR order, sparse penalty, max iterations, and zero threshold. It examines SNR distribution, temporal dynamics, AR coefficient strengths, and spike rates.

6d · Update Temporal Components

The core CNMF temporal update. Outputs: C (denoised calcium traces), S (inferred spike trains), b0/c0 (background), g (AR coefficients).

• AR Order (p)
  • AR=1 — simple linear decay model. If two spikes fire before the signal returns to baseline, they are counted as one event. Produces cleaner, sparser traces.
  • AR=2 — allows more nonlinear dynamics, can resolve closely spaced events, but produces noisier traces. AR order is conventionally an integer — use 1 or 2.
• Sparse Penalty — L1 penalty (sum of absolute spike amplitudes). Higher = fewer but more confident spikes. Lower = more spikes but higher noise risk. If you lower the sparse penalty and are concerned about noise, raise the zero threshold to compensate.
• Zero Threshold — values below this are set to zero. Useful complement to a lower sparse penalty: detects more events while suppressing low-amplitude noise.
• Max Iterations — 500 is usually sufficient.
• Chunk Size / Overlap — 5,000 frames per chunk, 100-frame overlap to avoid edge artifacts.
• Dask Settings — 8 workers is good for most systems; match threads per worker to your CPU core count.

6e · Filter & Validate

Removes components that didn't optimize well. Min spike sum, min calcium variance, and min spatial sum thresholds (all default 1e-6) catch dead or artifact components.

Step 7 Spatial Refinement

7a · Spatial Component Dilation

Expands spatial footprints for better coverage. CNMF footprints are often conservatively tight.

• Dilation Window Size — radius of structuring element (3 px typical). Larger = more expansion, risk of merging neighbors.
• Intensity Threshold — only dilate pixels above this fraction of component max (0.1 = 10%). Higher = more conservative.

7b · Component Clustering

Groups nearby components into clusters for efficient local processing.

• Max Cluster Size — 10 components default.
• Min Area / Min Intensity — thresholds to exclude trivially small or dim components from clustering.
• Overlap Threshold — 0.2 (20%) default.

7c · Component Boundary Calculation

Creates bounding boxes around clusters — rectangular regions used to limit the spatial update to relevant areas.

• Dilation Radius (10 px) — expands footprints before calculating bounds.
• Padding (20 px) — extra space around the dilated shapes; important near cluster edges.
• Minimum Size (40 px) — smallest allowed bounding box.
• Intensity Threshold (0.05) — what fraction of a component's max is considered "part of the neuron."

7d · Parameter Suggestions for Spatial Update

Analyzes temporal variability, spatial coherence, signal strength, and component shape to suggest minimum STD threshold, penalty scale, and maximum penalty for Step 7e. Treat these as an informed starting hypothesis.

Common scenarios:

• Neurons look fragmented → lower penalty values
• Neurons are merging → higher penalty values
• Missing dim neurons → use aggressive (lower) STD threshold
• Too much noise → use conservative (higher) STD threshold

7e · Spatial Update

Multi-penalty LASSO regression on local video regions. This is the longest step in the pipeline for large datasets.

• Min / Max Penalty — LASSO penalty range. Higher values → more compact, tightly defined footprints. Lower values → more expansive.
• Min STD — pixels below this variability are excluded from the update.

7f · Merging & Validation

Merges updated spatial components, handles cluster-boundary overlaps, and validates the final spatial result.

• Apply smoothing / Smoothing Sigma — optional Gaussian smoothing. Be cautious: too high a sigma causes footprints to bleed into large, diffuse blobs that may overlap neighboring neurons. Blocky or rectangular-looking components are more likely caused by bounding box geometry from the clustering step — reducing sigma will not fix that artifact. If components look unnaturally spread or merged, reduce sigma or turn off smoothing.
• Handle overlaps — keep on in almost all cases. Turning it off can produce inconsistent boundaries where cluster windows overlap.
• Save both versions — saves raw and smoothed versions for comparison.

Step 8 Final Processing & Export

8a · YrA Computation

Recomputes YrA using the refined spatial components from Step 7. This gives each neuron's temporal trace computed against better footprints than Step 6 had access to. Use step7f_A_merged as the spatial component source.

8b · Final Temporal Update

Re-runs the CNMF temporal update (CVXPY with AR modeling and sparsity constraints) on the improved YrA from Step 8a. See Step 6d for a full explanation of AR order and sparse penalty tradeoffs — the same logic applies here.

• Include background — incorporates background components (b, f).
• Chunk Size / Overlap — temporal chunking for memory efficiency; chunks are processed independently then merged.

8c · Final Filtering & Export

Final quality filtering (min component size, min SNR, min correlation), then export in multiple formats.

The export includes a timestamp and all processing parameters for full reproducibility.