I/O System#

Modular I/O system supporting multiple file formats through a common VideoReader/VideoWriter interface.

Factory Functions#

pyflowreg.util.io.factory.get_video_file_reader(input_source, buffer_size=500, bin_size=1, **kwargs)[source]#

Factory function to create appropriate reader based on input type. Mirrors MATLAB get_video_file_reader functionality.

Parameters:

input_source – Path to video file, numpy array, VideoReader instance, list of paths for multichannel, or folder for images
buffer_size – Buffer size for reading
bin_size – Temporal binning factor
**kwargs – Additional reader-specific arguments

Return type:

VideoReader

Returns:

Appropriate VideoReader subclass instance

pyflowreg.util.io.factory.get_video_file_writer(file_path, output_format, **kwargs)[source]#

Factory function to create appropriate writer based on output format. Mirrors MATLAB get_video_file_writer functionality.

Parameters:

file_path – Output file path
output_format – Output format string (e.g., ‘TIFF’, ‘HDF5’, ‘MAT’, ‘MULTIFILE_TIFF’, ‘ARRAY’, ‘NULL’, etc.) Special formats: - ‘ARRAY’: Returns ArrayWriter for in-memory accumulation - ‘NULL’: Returns NullVideoWriter that discards all frames (useful for callbacks only)
**kwargs – Additional writer-specific arguments

Return type:

VideoWriter

Returns:

Appropriate VideoWriter subclass instance

pyflowreg.util.io.factory.main()[source]#: Test wrapper implementations.

Base Classes#

class pyflowreg.util.io._base.VideoReader[source]#

Bases: ABC

Abstract base class for all video file readers.

Data is returned in (T, H, W, C) format: - T: Time/frames - H: Height - W: Width - C: Channels

This format is optimal for OpenCV operations and can be easily converted to PyTorch format (T, C, H, W) when needed.

abstract close()[source]#: Close file handles and clean up resources.

bin_frames(frames)[source]#

Apply temporal binning to reduce frame count.

Parameters:: frames – Input array with shape (T, H, W, C)
Return type:: ndarray
Returns:: Binned array with shape (T//bin_size, H, W, C)

read_batch()[source]#

Dataset Discovery#

class pyflowreg.util.io._ds_io.DSFileReader[source]#

Bases: object

A mixin class that provides a generic, multi-pass heuristic for finding the most likely data-containing datasets within a file.

class pyflowreg.util.io._ds_io.DSFileWriter(**kwargs)[source]#

Bases: object

A mixin class that provides logic for generating dataset names for writers. This is a direct port of the DS_file_writer.m functionality.

get_ds_name(channel_id, n_channels)[source]#

Gets the dataset name for a specific channel.

Parameters:

channel_id (int) – The 1-based index of the channel.
n_channels (int) – The total number of channels being written.

Return type:

str

Returns:

The dataset name as a string.

File Format Support#

HDF5#

class pyflowreg.util.io.hdf5.HDF5FileReader(file_path, buffer_size=500, bin_size=1, **kwargs)[source]#

Bases: DSFileReader, VideoReader

HDF5 video file reader with dataset discovery.

close()[source]#: Close HDF5 file.

class pyflowreg.util.io.hdf5.HDF5FileWriter(file_path, **kwargs)[source]#

Bases: DSFileWriter, VideoWriter

HDF5 video file writer with MATLAB compatibility.

Accepts frames in Python format (T, H, W, C) but stores them in MATLAB-compatible format as separate 3D datasets per channel with configurable dimension ordering.

Initialize HDF5 writer.

Parameters:

file_path – Output file path
dataset_names – Optional dataset naming pattern or list Default: ‘ch*’ (produces ch1, ch2, etc.)
dimension_ordering – Storage order for MATLAB compatibility Default: (0, 1, 2) for (H, W, T) storage
compression – HDF5 compression (‘gzip’, ‘lzf’, or None)
compression_level – Compression level for gzip (1-9)
chunk_size – Chunk size for temporal dimension (default: 1)

__init__(file_path, **kwargs)[source]#

Initialize HDF5 writer.

Parameters:

file_path – Output file path
dataset_names – Optional dataset naming pattern or list Default: ‘ch*’ (produces ch1, ch2, etc.)
dimension_ordering – Storage order for MATLAB compatibility Default: (0, 1, 2) for (H, W, T) storage
compression – HDF5 compression (‘gzip’, ‘lzf’, or None)
compression_level – Compression level for gzip (1-9)
chunk_size – Chunk size for temporal dimension (default: 1)

write_frames(frames)[source]#

Write frames to HDF5 file.

Parameters:: frames – Array with shape (T, H, W, C) or (T, H, W) or (H, W)

close()[source]#: Close the HDF5 file.

pyflowreg.util.io.hdf5.main()[source]#

pyflowreg.util.io.hdf5.reader_main()[source]#

TIFF#

class pyflowreg.util.io.tiff.TIFFFileReader(file_path, buffer_size=500, bin_size=1, **kwargs)[source]#

Bases: VideoReader

TIFF stack file reader with support for multi-page and multi-channel formats.

Supports: - Multi-page TIFF stacks (standard format) - Single-page multi-sample TIFFs (channels as samples per pixel) - Deinterleaved reading for formats like Suite2p - Memory-mapped reading for large files - Various data types (uint8/16/32/64, int32/64, float32/64)

Initialize TIFF reader.

Parameters:

file_path – Path to TIFF file
buffer_size – Number of frames per batch
bin_size – Temporal binning factor
deinterleave – Channel deinterleaving factor (1=none, >1 for interleaved formats)
use_memmap – Use memory mapping for large files (default: True)

__init__(file_path, buffer_size=500, bin_size=1, **kwargs)[source]#

Initialize TIFF reader.

Parameters:

file_path – Path to TIFF file
buffer_size – Number of frames per batch
bin_size – Temporal binning factor
deinterleave – Channel deinterleaving factor (1=none, >1 for interleaved formats)
use_memmap – Use memory mapping for large files (default: True)

close()[source]#: Close TIFF file.

get_metadata()[source]#

Get comprehensive metadata from TIFF file.

Return type:: dict

class pyflowreg.util.io.tiff.TIFFFileWriter(file_path, **kwargs)[source]#

Bases: VideoWriter

TIFF stack file writer with multi-page and compression support.

Features: - Multi-page TIFF writing - Suite2p format support (interleaved single-channel pages) - Various compression algorithms - ImageJ metadata compatibility - BigTIFF support for files >4GB (default: always enabled like MATLAB)

Initialize TIFF writer.

Parameters:

file_path – Output file path
format – ‘default’ or ‘suite2p’ (changes to interleaved single-channel pages)
compression – Compression type (‘none’, ‘lzw’, ‘zlib’, ‘jpeg’)
compression_level – Compression level for zlib (0-9)
bigtiff – Use BigTIFF format (default: True, matching MATLAB ‘w8’)
imagej – Write ImageJ-compatible metadata
metadata – Additional metadata dict to include

__init__(file_path, **kwargs)[source]#

Initialize TIFF writer.

Parameters:

file_path – Output file path
format – ‘default’ or ‘suite2p’ (changes to interleaved single-channel pages)
compression – Compression type (‘none’, ‘lzw’, ‘zlib’, ‘jpeg’)
compression_level – Compression level for zlib (0-9)
bigtiff – Use BigTIFF format (default: True, matching MATLAB ‘w8’)
imagej – Write ImageJ-compatible metadata
metadata – Additional metadata dict to include

write_frames(frames)[source]#

Write frames to TIFF file.

Parameters:: frames – Array with shape (T, H, W, C) or (T, H, W) or (H, W)

close()[source]#: Close the TIFF file.

pyflowreg.util.io.tiff.test_basic_functionality()[source]#: Test basic TIFF reading and writing.

pyflowreg.util.io.tiff.test_mdf_conversion()[source]#: Test MDF to TIFF conversion with proper binning.

pyflowreg.util.io.tiff.main2()[source]#

pyflowreg.util.io.tiff.main3()[source]#

MATLAB MAT#

class pyflowreg.util.io.mat.MATFileReader(file_path, buffer_size=500, bin_size=1, **kwargs)[source]#

Bases: DSFileReader, VideoReader

MAT video file reader with dataset discovery. Supports both traditional MAT files (v5, v7) and v7.3 (HDF5-based).

close()[source]#: Close MAT file.

class pyflowreg.util.io.mat.MATFileWriter(file_path, **kwargs)[source]#

Bases: DSFileWriter, VideoWriter

MAT video file writer with MATLAB compatibility.

Creates MAT files with separate 3D datasets per channel, stored in MATLAB-compatible dimension ordering.

Initialize MAT writer.

Parameters:

file_path – Output file path
dataset_names – Optional dataset naming pattern or list Default: ‘ch*’ (produces ch1, ch2, etc.)
dimension_ordering – Storage order for MATLAB compatibility Default: [0, 1, 2] for (H, W, T) in MATLAB
use_v73 – Force v7.3 format (HDF5-based) for large files

__init__(file_path, **kwargs)[source]#

Initialize MAT writer.

Parameters:

file_path – Output file path
dataset_names – Optional dataset naming pattern or list Default: ‘ch*’ (produces ch1, ch2, etc.)
dimension_ordering – Storage order for MATLAB compatibility Default: [0, 1, 2] for (H, W, T) in MATLAB
use_v73 – Force v7.3 format (HDF5-based) for large files

write_frames(frames)[source]#

Write frames to MAT file buffers.

Parameters:: frames – Array with shape (T, H, W, C) or (T, H, W) or (H, W)

close()[source]#: Close and write the MAT file.

pyflowreg.util.io.mat.main()[source]#: Test MAT file I/O.

MDF (Sutter MesaScope)#

class pyflowreg.util.io.mdf.MDFFileReader(file_path, buffer_size=500, bin_size=1, **kwargs)[source]#

Bases: VideoReader

MDF file reader for Windows using MCSX.Data COM interface.

Note: MDF files use 1-based indexing internally (MATLAB heritage). This reader transparently converts between 0-based Python indexing and 1-based MDF indexing.

Initialize MDF reader.

Parameters:

file_path – Path to .mdf file
buffer_size – Number of frames per batch
bin_size – Temporal binning factor
channel_idx – Optional list of channels to read (1-based)

__init__(file_path, buffer_size=500, bin_size=1, **kwargs)[source]#

Initialize MDF reader.

Parameters:

file_path – Path to .mdf file
buffer_size – Number of frames per batch
bin_size – Temporal binning factor
channel_idx – Optional list of channels to read (1-based)

close()[source]#: Release COM object and clean up.

reset_connection()[source]#: Reset the MDF COM connection if it becomes unresponsive. Useful when the COM server gets stuck.

get_metadata()[source]#

Get comprehensive metadata from MDF file.

Return type:: dict
Returns:: Dictionary with file metadata

Multi-File Handling#

Wrapper classes for video file I/O operations. Provides multi-file, multi-channel, and subset reading/writing capabilities.

class pyflowreg.util.io.multifile_wrappers.MULTIFILEFileWriter(filename, file_type='TIFF', **kwargs)[source]#

Bases: VideoWriter

File writer that writes one file per channel. Each channel is saved to a separate file with _ch{N} suffix.

Initialize multi-file writer.

Parameters:

filename – Base output filename or directory
file_type – Output format for each channel file
**kwargs – Additional parameters passed to individual writers

__init__(filename, file_type='TIFF', **kwargs)[source]#

Initialize multi-file writer.

Parameters:

filename – Base output filename or directory
file_type – Output format for each channel file
**kwargs – Additional parameters passed to individual writers

write_frames(frames)[source]#

Write frames to multiple files (one per channel).

Parameters:: frames – Array with shape (T, H, W, C) or compatible

close()[source]#: Close all channel writers.

class pyflowreg.util.io.multifile_wrappers.MULTICHANNELFileReader(input_files, buffer_size=500, bin_size=1, **kwargs)[source]#

Bases: VideoReader

Generic multichannel reader that reads from multiple video files and combines them into a single multichannel output.

Initialize multichannel reader.

Parameters:

input_files – List of input file paths
buffer_size – Buffer size for batch reading
bin_size – Temporal binning factor
**kwargs – Additional parameters passed to individual readers

__init__(input_files, buffer_size=500, bin_size=1, **kwargs)[source]#

Initialize multichannel reader.

Parameters:

input_files – List of input file paths
buffer_size – Buffer size for batch reading
bin_size – Temporal binning factor
**kwargs – Additional parameters passed to individual readers

close()[source]#: Close all file readers.

class pyflowreg.util.io.multifile_wrappers.SUBSETFileReader(video_file_reader, indices)[source]#

Bases: VideoReader

Reader that provides a subset of frames from another video reader. Useful for reading non-contiguous frame indices or reordering frames.

Initialize subset reader.

Parameters:

video_file_reader – Source video reader
indices – Frame indices to include in subset (0-based)

__init__(video_file_reader, indices)[source]#

Initialize subset reader.

Parameters:

video_file_reader – Source video reader
indices – Frame indices to include in subset (0-based)

close()[source]#: No-op as we don’t own the source reader.

pyflowreg.util.io.multifile_wrappers.main()[source]#: Test wrapper implementations.

Array I/O#

Private module for in-memory array I/O wrappers. Allows numpy arrays to be processed through the same pipeline as video files.

class pyflowreg.util.io._arr.ArrayReader(array, buffer_size=100, bin_size=1, inplace=False)[source]#

Bases: VideoReader

Wraps numpy arrays to provide VideoReader interface. Enables batch processing and binning for in-memory arrays.

Initialize array reader.

Parameters:

array – Input array with shape (T,H,W,C) or (H,W,C) or (T,H,W)
buffer_size – Number of frames per batch
bin_size – Temporal binning factor
inplace – If True, return views for memory efficiency (no copy). If False (default), return copies for safety with multiprocessing.

__init__(array, buffer_size=100, bin_size=1, inplace=False)[source]#

Initialize array reader.

Parameters:

array – Input array with shape (T,H,W,C) or (H,W,C) or (T,H,W)
buffer_size – Number of frames per batch
bin_size – Temporal binning factor
inplace – If True, return views for memory efficiency (no copy). If False (default), return copies for safety with multiprocessing.

close()[source]#: No-op for array reader.

class pyflowreg.util.io._arr.ArrayWriter[source]#

Bases: VideoWriter

Accumulates frames in memory instead of writing to file. Provides VideoWriter interface for array output.

Initialize array writer.

__init__()[source]#: Initialize array writer.

init(first_frame_batch)[source]#

Initialize writer from first batch following base class pattern.

Parameters:: first_frame_batch – First batch with shape (T,H,W,C) or (H,W,C) or even (H,W)

write_frames(frames)[source]#

Accumulate frames in memory.

Parameters:: frames – Array with shape (T,H,W,C), (H,W,C), or (H,W)

get_array()[source]#

Fetch accumulated frames as single array.

Return type:: Optional[ndarray]
Returns:: Concatenated frames or None if empty

close()[source]#: No-op for array writer.

Null Writer (Callback-Only Processing)#

Null video writer that discards frames without storage. Implements the Null Object Pattern for the VideoWriter interface.

class pyflowreg.util.io._null.NullVideoWriter[source]#

Bases: VideoWriter

A writer that discards all frames without storing or writing them.

Useful for running the motion correction pipeline when only intermediate computations (callbacks, displacement fields) are needed, without the overhead of actual I/O operations.

This implements the Null Object Pattern, allowing the pipeline to run normally without special case handling for “no output” scenarios.

frames_written#: Counter tracking total frames processed

batches_written#: Counter tracking total batches processed

Example

>>> from pyflowreg.util.io import NullVideoWriter
>>> writer = NullVideoWriter()
>>> frames = np.random.rand(10, 256, 256, 2)  # 10 frames
>>> writer.write_frames(frames)
>>> print(writer)  # NullVideoWriter(frames_written=10, batches=1)

Initialize the null writer with counters.

__init__()[source]#: Initialize the null writer with counters.

init(first_frame_batch)[source]#

Initialize writer properties from first batch.

Parameters:: first_frame_batch – First batch with shape (T,H,W,C), (H,W,C), or (H,W)

write_frames(frames)[source]#

Discard frames but track count for debugging/monitoring.

Parameters:: frames – Array with shape (T,H,W,C), (H,W,C), or (H,W) These frames are not stored, only counted.

close()[source]#: No-op for null writer - no resources to clean up.

ScanImage Support#

Utility functions for parsing and handling ScanImage TIFF metadata.

ScanImage stores microscopy metadata in TIFF files, including information about: - Multi-channel imaging - Z-stacks (3D volumes) - Time series - ROIs (Regions of Interest) - Acquisition parameters

This module provides tools to extract and interpret this metadata correctly.

pyflowreg.util.io._scanimage.parse_scanimage_metadata(file_path)[source]#

Parse ScanImage metadata from a TIFF file.

Parameters:

file_path – Path to the ScanImage TIFF file

Return type:

Dict[str, Any]

Returns:

Dictionary containing parsed metadata with keys –

is_scanimage: Whether this is a ScanImage file
version: ScanImage version
channels: Number of channels
volumes: Number of volumes/stacks
slices_per_volume: Number of Z slices per volume
frames_per_slice: Number of frames at each Z position
total_frames: Total number of 2D frames (volumes * slices_per_volume)
z_step: Z step size in microns (if available)
frame_rate: Acquisition frame rate (if available)
roi_data: ROI information (if available)
raw_metadata: Complete raw metadata dictionary

pyflowreg.util.io._scanimage.interpret_scanimage_dimensions(tif_shape, tif_axes, si_metadata)[source]#

Interpret dimensions of a ScanImage TIFF based on shape, axes, and metadata.

Parameters:

tif_shape – Shape tuple from tifffile
tif_axes – Axes string from tifffile (e.g., ‘TZYX’, ‘TYX’, ‘ZYX’)
si_metadata – Parsed ScanImage metadata dictionary

Return type:

Dict[str, Any]

Returns:

Dictionary with interpreted dimensions –

total_frames: Total 2D frames for processing
height: Image height
width: Image width
channels: Number of channels
volumes: Number of 3D volumes
z_planes: Number of Z planes per volume
true_time_frames: Actual time points (volumes)

pyflowreg.util.io._scanimage.format_scanimage_metadata_report(metadata)[source]#

Format ScanImage metadata into a human-readable report.

Parameters:: metadata – Parsed metadata dictionary
Return type:: str
Returns:: Formatted string report