GitHub YouTube Wiki QC Sample

The Diffusion MRI (DTI) Pre-Processing Pipeline is a lightweight, end-to-end workflow for diffusion brain MRI. It performs susceptibility correction (TOPUP), eddy current & motion correction (EDDY), within-session alignment/merge, registration to MNI, and diffusion tensor fitting (DIPY). The pipeline generates clear QC visualizations and summary tables (CSV/JSON/HTML) for downstream analysis. The design is config-driven, scalable across cohorts, and interoperable with common neuroimaging tools and AI pipelines.

• Distortion & Motion, FSL TOPUP (AP/PA) + EDDY with slice-to-volume (S2V) and outlier replacement (REPOL).
• Brain Extraction, FSL BET with montage QC and brain-volume range checks.
• Registration, FLIRT rigid (6 DOF) → affine (12 DOF) to MNI (trilinear for images, nearest-neighbour for masks).
• Gradient Handling, b-vector rotation synchronized with all applied transforms.
• Tensor Fitting, DIPY (WLS/OLS) producing FA, MD, RD, AD, eigenvectors, and tensor components.
• QC & Reports, Per-step PNG overlays, per-subject CSV/JSON, and an aggregated HTML summary.
• Metadata, CSV indices for corrected/registered outputs, and tensor maps for downstream modeling.

• Python ≥ 3.8

• Python packages: numpy, nibabel, dipy, matplotlib, tqdm, pandas

• FSL ≥ 6.0 (for TOPUP, EDDY, BET, FLIRT)

• ANTs

• DIPY

This pipeline is config-driven. Edit config.py once and reuse across all scripts (b0_correction.py, process_eddy.py, brain_extraction.py, run_reg_mni.py, run_dtifit_dipy.py, run_final_qc.py, dti_qc.py).

FSL environment

• FSL_HOME, FSL_BIN, FSL_ENV,  core FSL paths and env vars.
• setup_fsl_env(), helper to source FSL config into the current process.

Common / Shared

• DATASET_NAME, e.g., "OpenNeuro ds004215"
• INPUT_DIR, INPUT_SUBDIR, OUTPUT_DIR, TEMP_DIR, QC_DIR
• NUM_SCANS_PER_SESSION, number of DWI runs per session
• Logging: ENABLE_DETAILED_LOGGING, LOG_DIR, LOG_LEVEL, LOG_FORMAT
• Processing toggles: DICOM_TO_NIFTY, FORCE_REPROCESS

B0 field correction

• B0_CORRECTION_FOLDER, B0_CORRECTION ("Topup" | "Fieldmap" | None)
• Input patterns: DWI_FILE_PATTERNS, BVAL_FILE_PATTERNS, BVEC_FILE_PATTERNS, JSON_FILE_PATTERNS
• Reversed polarity: REVERSED_DWI_FILE_PATTERNS, REVERSED_BVAL_FILE_PATTERNS, REVERSED_BVEC_FILE_PATTERNS, REVERSED_JSON_FILE_PATTERNS
• B0_CORRECTION_QC_SLICES, slice indices for B0 QC

Skull stripping (BET)

• SKULL_STRIP_INPUT_FOLDER, SKULL_STRIP_OUTPUT_FOLDER
• SKULL_STRIP_INPUT_NAMES, SKULL_STRIP_OUTPUT_PATTERN
• FRACTIONAL_INTENSITY, BET threshold (default 0.4)
• QC: ENABLE_QC, QC_GENERATE_IMAGES, QC_SUMMARY_FILE, QC_BRAIN_VOLUME_MIN_ML / QC_BRAIN_VOLUME_MAX_ML (800–2000)
• QC visuals: QC_IMAGE_DPI, QC_IMAGE_FORMAT, QC_SLICE_VIEWS, QC_SLICE_NUMBERS

Eddy correction

• EDDY_CORRECTION_FOLDER
• SLICE_TO_SLICE_CORRECTION, enable slice-to-slice correction
• BASELINE_SLICE_ORDER_JSON, fallback template if JSON lacks slice order
• EDDY_CORRECTION_QC_SLICES

Registration to MNI

• TEMPLATE_PATH, e.g., tpl-MNI152NLin2009cAsym_res-01_desc-brain_T1w.nii.gz
• Inputs: REG_MNI_MASK_INPUT_FOLDER, REG_MNI_MASK_NAMES, REG_MNI_B0_INPUT_FOLDER, REG_MNI_B0_INPUT_NAMES, REG_MNI_INPUT_FOLDER, REG_MNI_INPUT_NAMES, REG_MNI_BVEC_INPUT_NAMES, REG_MNI_BVAL_INPUT_NAMES
• Outputs: REG_MNI_OUTPUT_FOLDER, REG_MNI_OUTPUT_PATTERN
• QC: MODALITY_PATTERNS, e.g., {"Diffusion": ["b0_reg*", "mask_bet_scan0_mask*"]}

DTI fit (DIPY)

• MASK_PATH, MASK_NAME
• DTIFIT_INPUT_FOLDER, DTIFIT_DWI_INPUT_NAME, DTIFIT_BVEC_INPUT_NAME, DTIFIT_BVAL_INPUT_NAME
• DTIFIT_OUT_FOLDER, DTIFIT_QC_SLICES

# !/usr/bin/env python
# -*- coding: utf-8 -*-

"""
Configuration file for neuroimaging pipeline
Contains settings for:
1. Common configurations (paths, logging, etc.)
2. Topup and Eddy correction
3. Skull stripping
4. DTI fit
5. Registration

Authors:
- Mohammad H Abbasi (mabbasi [at] stanford.edu)
- Gustavo Chau (gchau [at] stanford.edu)

Stanford University
Created: 2025
Version: 1.0.0
"""

import os
import multiprocessing
import subprocess

###############################################################################
#                         FSL Settings                                #
###############################################################################
FSL_HOME = "/path/u/user/fsl"
FSL_BIN = os.path.join(FSL_HOME,'share/fsl/bin')
# FreeSurfer environment variables
FSL_ENV = {
    "FSLOUTPUTTYPE": "NIFTI_GZ",
    "FSLDIR": FSL_HOME
}

def setup_fsl_env():
    """
    Set up FreeSurfer environment variables in the current Python process
    """
    os.environ.update(FSL_ENV)
    os.system(f". {FSL_HOME}/etc/fslconf/fsl.sh")

###############################################################################
#                         Common/Shared Settings                                #
###############################################################################
DATASET_NAME = "OpenNeuro ds004215"

INPUT_DIR = "/input/path"
INPUT_SUBDIR = ""
OUTPUT_DIR = "/output/path"
TEMP_DIR = "tmp"
QC_DIR = os.path.join(OUTPUT_DIR, "QC")
NUM_SCANS_PER_SESSION = 1

# Logging settings
ENABLE_DETAILED_LOGGING = True
LOG_DIR = "/logs/path"
LOG_LEVEL = "DEBUG"             # DEBUG, INFO, WARNING, ERROR, CRITICAL
LOG_FORMAT = '%(asctime)s - %(name)s - %(levelname)s - %(message)s' if ENABLE_DETAILED_LOGGING else '%(message)s'

# Processing settings
DICOM_TO_NIFTY = False
FORCE_REPROCESS = False

###############################################################################
#                         B0 field correction settings                        #
###############################################################################
B0_CORRECTION_FOLDER = os.path.join(OUTPUT_DIR, "B0_correction")
B0_CORRECTION = 'Topup' # Topup, Fieldmap or None

# File patterns for input files. Length of lists should match NUM_SCANS_PER_SESSION
DWI_FILE_PATTERNS  = ['*dMRI_PA.nii']
BVAL_FILE_PATTERNS = ['*dMRI_PA.bval']
BVEC_FILE_PATTERNS = ['*dMRI_PA.bvec']
JSON_FILE_PATTERNS = ['*dMRI_PA.json']

# File patterns for Reversed polarity input files. Length of lists should match NUM_SCANS_PER_SESSION
REVERSED_DWI_FILE_PATTERNS  = ['*dMRI_AP.nii']
REVERSED_BVAL_FILE_PATTERNS = ['*dMRI_AP.bval']
REVERSED_BVEC_FILE_PATTERNS = ['*dMRI_AP.bvec']
REVERSED_JSON_FILE_PATTERNS = ['*dMRI_AP.json']

B0_CORRECTION_QC_SLICES = [17,40]

###############################################################################
#                      Skull Stripping Settings                               #
###############################################################################
SKULL_STRIP_INPUT_FOLDER   = B0_CORRECTION_FOLDER
SKULL_STRIP_OUTPUT_FOLDER  = os.path.join(OUTPUT_DIR, "Skull_stripping")
SKULL_STRIP_INPUT_NAMES    = None  # if None, assume output of Topup
SKULL_STRIP_OUTPUT_PATTERN = None  # if None, use "mask_bet_scan{scan_num}"
FRACTIONAL_INTENSITY       = 0.4   # 0..1 (smaller => larger brain outline)

# Skull Stripping QC
ENABLE_QC         = True
QC_GENERATE_IMAGES= True
QC_SUMMARY_FILE   = "qc_summary.csv"

QC_BRAIN_VOLUME_MIN_ML = 800
QC_BRAIN_VOLUME_MAX_ML = 2000

QC_IMAGE_DPI    = 150
QC_IMAGE_FORMAT = "png"
QC_SLICE_VIEWS  = ["axial", "sagittal", "coronal"]
QC_SLICE_NUMBERS= {"axial": None, "sagittal": None, "coronal": None}

###############################################################################
#                         Eddy correction settings                            #
###############################################################################
EDDY_CORRECTION_FOLDER   = os.path.join(OUTPUT_DIR, "Eddy_correction")
SLICE_TO_SLICE_CORRECTION= True
BASELINE_SLICE_ORDER_JSON= None   # template if json lacks info
EDDY_CORRECTION_QC_SLICES= [17,40]

###############################################################################
#                         Registration to MNI                                 #
###############################################################################
TEMPLATE_PATH              = "/path/u/user/tpl-MNI152NLin2009cAsym_res-01_desc-brain_T1w.nii.gz"

REG_MNI_MASK_INPUT_FOLDER  = SKULL_STRIP_OUTPUT_FOLDER
REG_MNI_MASK_NAMES         = 'mask_bet_scan0_mask.nii.gz'

REG_MNI_B0_INPUT_FOLDER    = SKULL_STRIP_OUTPUT_FOLDER
REG_MNI_B0_INPUT_NAMES     = 'mask_bet_scan0.nii.gz'  # or None

REG_MNI_INPUT_FOLDER       = EDDY_CORRECTION_FOLDER
REG_MNI_INPUT_NAMES        = 'eddy_aligned_0.nii.gz'
REG_MNI_BVEC_INPUT_NAMES   = 'eddy_aligned_0.eddy_rotated_bvecs'
REG_MNI_BVAL_INPUT_NAMES   = 'dwi_merged_0.bval'

REG_MNI_OUTPUT_FOLDER      = os.path.join(OUTPUT_DIR, "Reg_MNI")
REG_MNI_OUTPUT_PATTERN     = None  # default "merged_dwi"

# QC thresholds/patterns
MODALITY_PATTERNS = {'Diffusion':['b0_reg*','mask_bet_scan0_mask*']}

###############################################################################
#                         DTIFIT using Dipy                                   #
###############################################################################
MASK_PATH               = REG_MNI_OUTPUT_FOLDER
DTIFIT_INPUT_FOLDER     = REG_MNI_OUTPUT_FOLDER
DTIFIT_DWI_INPUT_NAME   = None  # or explicit file name
DTIFIT_BVEC_INPUT_NAME  = None
DTIFIT_BVAL_INPUT_NAME  = None
MASK_NAME               = None  # or explicit mask name

DTIFIT_OUT_FOLDER       = os.path.join(OUTPUT_DIR, "Dtifit")
DTIFIT_QC_SLICES        = [75,90]

1) Topup Correction, b0_correction.py

Correct susceptibility-induced distortions using FSL TOPUP (Andersson et al., 2003). This step estimates a field map from images with opposite phase-encoding directions and applies it to unwarp distorted diffusion volumes.

Reference: Andersson, J.L.R., Skare, S., & Ashburner, J. (2003). NeuroImage, 20(2), 870–888.
https://pubmed.ncbi.nlm.nih.gov/14568458/

2) Eddy Current and Motion Correction, process_eddy.py

Correct distortions caused by eddy currents and subject motion using FSL EDDY (Andersson & Sotiropoulos, 2016). Includes slice-to-volume (S2V) correction and outlier replacement (REPOL).

Steps performed:

1. Load and merge diffusion-weighted images (AP/PA or reversed polarity)
2. Prepare acquisition parameters and B0 indices
3. Run FSL EDDY with slice-to-volume correction (optional)
4. Generate QC images comparing original vs corrected volumes

Reference: Andersson, J.L.R., & Sotiropoulos, S.N. (2016). NeuroImage, 125, 1063–1078.
https://pubmed.ncbi.nlm.nih.gov/26481672/

3) Brain Extraction, brain_extraction.py

Removes non-brain tissue from diffusion MRI data using FSL BET (Smith, 2002). This step performs skull stripping and generates quality control (QC) images and summary files to validate the extracted brain.

Steps performed:

1. Apply FSL BET to remove non-brain tissue
2. Generate brain mask and masked output
3. Compute brain volume (ml) and validate against the expected range
4. Create QC images (raw, brain-extracted, raw+mask, differences)
5. Write QC summary (CSV) for downstream review

Reference: Smith, S.M. (2002). Human Brain Mapping, 17(3), 143–155.
https://pubmed.ncbi.nlm.nih.gov/12391568/

4) Within-session registration & merge, reg_within_fsl.py

Aligns and merges multiple diffusion MRI runs using FSL FLIRT in case a session is broken down into multiple acquisitions. Rigid transformations are estimated between B0 reference images, applied to corresponding diffusion volumes, and propagated to b-vectors to ensure orientation consistency across runs.

Steps performed:

1. Register B0 images between runs using FLIRT rigid-body transform
2. Apply transforms to diffusion volumes (DWI)
3. Rotate b-vectors using the computed transformation matrix
4. Merge registered DWI volumes, b-vectors, and b-values
5. Save combined outputs for downstream processing

5) Registration to MNI Space, run_reg_mni.py

Registers diffusion images to the MNI152 template using a two-step approach with FSL FLIRT (rigid + affine). Corresponding b-vectors are rotated to preserve orientation consistency. The pipeline also applies the transforms to the diffusion volumes and masks, and prepares final outputs for downstream analysis.

Steps performed:

1. Register B0 to MNI (rigid, 6 DOF), save the matrix and registered B0
2. Refine B0→MNI with affine (12 DOF), save matrix, and registered B0
3. Apply rigid and then affine transforms to DWI
4. Rotate b-vectors using the previously computed transformations
5. Copy b-values (bval_final.bval) for consistency
6. Register mask with nearest-neighbour interpolation using the previously computed transformations

6) Diffusion Tensor Model Fitting, run_dtifit_dipy.py

Fits a diffusion tensor model using DIPY (Garyfallidis et al., 2014). This script takes preprocessed diffusion MRI data (DWI, bvec, bval, mask) and outputs standard DTI-derived measures including fractional anisotropy (FA), mean diffusivity (MD), radial diffusivity (RD), and axial diffusivity (AD). Eigenvectors of the tensor are also saved for visualization.

Steps performed:

1. Load diffusion MRI data, b-values, b-vectors, and brain mask
2. Construct gradient table using DIPY
3. Fit the diffusion tensor model voxel-wise
4. Extract tensor components (Dxx, Dxy, Dxz, Dyy, Dyz, Dzz)
5. Save DTI-derived tensor components and maps: FA, MD, RD, AD, and eigenvectors (V1, V2, V3)
6. Generate QC images showing a color FA map

References:

• Basser, P.J., Mattiello, J., & LeBihan, D. (1994). MR diffusion tensor spectroscopy and imaging. Biophysical Journal, 66(1), 259–267.
  https://pubmed.ncbi.nlm.nih.gov/8130344/
• Garyfallidis, E., Brett, M., Correia, M.M., Williams, G.B., & Nimmo-Smith, I. (2014). DIPY, a library for the analysis of diffusion MRI data. Frontiers in Neuroinformatics, 8, 8.
  https://pmc.ncbi.nlm.nih.gov/articles/PMC3931231/

7) Quality Control of Final Outputs, run_final_qc.py

Performs automated, session-aware QC checks on diffusion pipeline outputs. This script validates file existence and assesses registration quality using overlap metrics (Dice) between warped images and the MNI template. Results are written to per-session CSV reports for downstream review.

What it checks:

1. File existence validation (key outputs, matrices, QC images)
2. Registration to MNI (rigid + affine), Dice coefficient vs. MNI template
3. Within-session registration (if NUM_SCANS_PER_SESSION > 1) Dice on b0-to-b0
4. Session discovery and per-session reporting
5. Structured logging to file + console

Outputs (CSV per subject/session) (filenames as written by the current code):

• QC/file_existance.csv
• QC/within_subject_registraction_qc.csv (only if NUM_SCANS_PER_SESSION > 1)
• QC/mni_registraction_qc.csv

Notes: Uses thresholds for PASS/WARNING based on Dice (configurable in code). Relies on TEMPLATE_PATH and derivative folders from config.py. Designed to run after registration steps have completed.

Reference: Zou, K.H., Warfield, S.K., Bharatha, A., Tempany, C.M., Kaus, M.R., Haker, S.J., Wells, W.M., Jolesz, F.A., & Kikinis, R. (2004), 11(2), 178–189.
https://pubmed.ncbi.nlm.nih.gov/14974593/

8) Generation of HTML QC Reports, dti_qc.py

Generates automated, subject-level QC reports that summarize and visualize the outputs of the DTI preprocessing pipeline. This script reads existing QC CSVs and QC images from each step and compiles them into a clickable HTML report along with CSV/JSON summaries. If FA/MD maps are available, it also computes basic statistics.

Summarized in the report:

• Raw vs. corrected B0 (Topup), QC images (before/after)
• Eddy-corrected vs. uncorrected volumes, QC images
• Brain extraction evaluation, mask overlays + brain volume (mL)
• FA and color-FA maps, quick-look thumbnails and stats
• Registration metrics, Dice coefficients for within-session & MNI steps

Inputs (read-only) (filenames as written by the current code):

• Existing QC CSVs (e.g., file_existance.csv, within_subject_registraction_qc.csv, mni_registraction_qc.csv)
• QC images from each step (Topup, Skull stripping, Eddy, DTI fit)
• Optional DTI maps: dipy_fa.nii.gz, dipy_md.nii.gz
• Paths and flags from config.py (e.g., OUTPUT_DIR, NUM_SCANS_PER_SESSION)

Outputs:

• Per-subject JSON: <QC>/<subject_id>/<subject_id>_qc_results.json
• Per-subject CSV: <QC>/<subject_id>/<subject_id>_qc_summary.csv
• Per-subject HTML: <QC>/<subject_id>/<subject_id>_report.html
• Aggregated CSV: <QC>/all_subjects_summary.csv
• Aggregated HTML: <QC>/DTI_QC_Summary.html

Behavior & notes: Session-aware; read-only summarization; safely skips visualization if nilearn is missing; uses non-interactive Matplotlib backend (Agg).

Reference: Dice for overlap validation: Zou, K.H., et al. (2004). Academic Radiology, 11(2), 178–189.
https://pubmed.ncbi.nlm.nih.gov/14974593/

9) Utility Functions, utilities.py

Provides supporting functions used across the diffusion MRI preprocessing pipeline.

Key functionalities:

• Session Detection, automatically identifies session folders (e.g., YYYY-MM-DD).
• File Search & Matching, robust pattern matching to locate NIfTI, bvec/bval, and transform files even when filenames differ slightly.
• NIfTI Handling, inspects image dimensions and trims odd dimensions to ensure compatibility with FSL/ANTs.
• QC Image Generation, creates slice-wise PNG snapshots for quick quality checks.
• Logging Utilities initializes console and file logging with safe fallbacks if write access is restricted.

Quick Start

1) Edit configuration

• Update paths and file patterns in config.py to point to your dataset.
• Set NUM_SCANS_PER_SESSION according to your acquisitions.

2) Prepare subject list

Create a text file named subject_list.txt with one subject ID per line, e.g.:

• sub-0001
• sub-0002
• sub-0003

3) Run preprocessing

Submit array jobs (replace N with the number of subjects in your list):

sbatch --array=1-N run_b0_correction.sh

Pipeline order (per subject)

python b0_correction.py <subject_id>
python process_eddy.py <subject_id>
python brain_extraction.py <subject_id>
python reg_within_fsl.py <subject_id>
python run_reg_mni.py <subject_id>
python run_dtifit_dipy.py <subject_id>
python run_final_qc.py <subject_id>
python dti_qc.py <subject_id>

Check outputs

• Preprocessed data → <OUTPUT_DIR> (set in config.py)
• QC results → <OUTPUT_DIR>/QC/
• Open <OUTPUT_DIR>/QC/DTI_QC_Summary.html for an overview across all subjects
• Lab: Stanford University, STAI Lab (https://stai.stanford.edu)
• Created: 2025 | Version: 1.0.2 | Last update: 10 Sep, 2025

@software{abbasi2025dti,
  author       = {MohammadHassan Abbasi,  Ehsan Adeli, and Gustavo Chau},
  title        = {{DTI} {P}rocessing {P}ipeline: A reproducible framework for diffusion {MRI} preprocessing and quality control},
  year         = {2025},
  institution  = {Stanford Translational AI Lab, Stanford University},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.17503171},
  url          = {https://doi.org/10.5281/zenodo.17503171},
  note = {Zenodo, doi: 10.5281/zenodo.17503171},
 }