Chapter 18 ENIGMA DTI Preprocessing without TOPUP

Be sure to run ANTs Cortical Thickness before completing this step.

18.1 Job Script

On the remote computer create job script.

vi ~/scripts/ACAP/ENIGMA-preprocess-job.sh

Copy and paste.

#!/bin/bash

#SBATCH --time=24:00:00   # walltime
#SBATCH --ntasks=1   # number of processor cores (i.e. tasks)
#SBATCH --nodes=1   # number of nodes
#SBATCH --mem-per-cpu=16G  # memory per CPU core

# LOAD ENVIRONMENTAL VARIABLES
username=`id -un`
module load lib/openblas/0.2.20-gnu
module load fsl/6.0.0
export ANTSPATH=/home/${username}/apps/ants/bin/
PATH=${ANTSPATH}:${PATH}

# INSERT CODE, AND RUN YOUR PROGRAMS HERE
# CREATE SYMLINKS TO YOUR DWI AND PROCESSED T1 FILES
DATA_DIR=/work/${username}/data/ACAP/
BIDS_DIR=/work/${username}/BIDS/ACAP/
ANTS_DIR=${DATA_DIR}/mindboggle/ants_subjects
mkdir -p ${DATA_DIR}/enigma/dwiprep/${1}_${2}
cd ${DATA_DIR}/enigma/dwiprep/${1}_${2}
ln -s ${BIDS_DIR}/${1}/${2}/dwi/${1}_${2}_dwi.nii.gz dwi.nii.gz
ln -s ${BIDS_DIR}/${1}/${2}/dwi/${1}_${2}_dwi.bval dwi.bval
ln -s ${BIDS_DIR}/${1}/${2}/dwi/${1}_${2}_dwi.bvec dwi.bvec
ln -s ${ANTS_DIR}/${1}_${2}/antsExtractedBrain0N4.nii.gz t1.nii.gz

## CROP DWI IMAGE
c4d dwi.nii.gz -trim 5vox -pad 10% 10% 0 -o dwi_padded.nii.gz

## CREATE DWI BRAIN MASK FOR EDDY
bet dwi_padded.nii.gz dwi_padded -f 0.1 -g 0 -m -n

## SET acqparams.txt
printf "0 1 0 0.095" > acqparams.txt

## SET index.txt
myVar=($(wc -w dwi.bval))
indx=""
for ((i=1; i<=$myVar; i+=1)); do indx="$indx 1"; done
echo $indx > index.txt

## EDDY CORRECTION
eddy_openmp \
--imain=dwi_padded.nii.gz \
--acqp=acqparams.txt \
--index=index.txt \
--mask=dwi_padded_mask.nii.gz \
--bvecs=dwi.bvec \
--bvals=dwi.bval \
--out=dwi-eddy

## GENERATE B0
fslroi dwi-eddy.nii.gz b0.nii.gz 0 1

## RIGID ALIGN T1 TO B0 TO ELIMINATE MOTION BETWEEN THE TWO IMAGES
## BECAUSE YOUR B0 IMAGE IS FIXED YOU DO NOT NEED TO ADJUST BVEC FILE
antsRegistrationSyNQuick.sh \
-d 3 \
-f b0.nii.gz \
-m t1.nii.gz \
-o t1tob0- \
-t a

## NON-LINEARLY REGISTER B0 TO T1 TO FIX DISTORTIONS (STRETCHING AND SQUISHING)
## YOU ARE NOT MOVING THE B0 FILE AS A WHOLE
## YOU ARE ONLY ADJUSTING THE VOXELS THAT ARE OUT OF PLACE
## YOU DO NOT NEED TO ADJUST THE BVEC FILE
antsRegistrationSyNQuick.sh \
-d 3 \
-f t1tob0-Warped.nii.gz \
-m b0.nii.gz \
-o dwi-epicorr- \
-t so

## APPLY TRANSFORM TO EDDY-CORRECTED DW IMAGE
antsApplyTransforms \
--dimensionality 3 \
--input-image-type 3 \
--input dwi-eddy.nii.gz \
--output dwi-epicorr.nii.gz \
--reference-image t1tob0-Warped.nii.gz \
--interpolation Bspline \
--transform dwi-epicorr-1Warp.nii.gz

## FINAL DWI BRAIN MASK
bet dwi-epicorr.nii.gz dwi-epicorr -f 0.1 -g 0 -m -n

## FIT TENSORS
dtifit \
--data=dwi-epicorr.nii.gz \
--out=dti \
--mask=dwi-epicorr_mask.nii.gz \
--bvecs=dwi-eddy.eddy_rotated_bvecs \
--bvals=dwi.bval \
--save_tensor

18.2 Batch Script

First check that your subjid and ses are correct.

for i in $(find /work/ashley.ware/BIDS/ACAP/ -name "*b900_dwi.nii.gz"); do
  subjid=`echo $i | cut -d "/" -f6`
  ses=`echo $i | cut -d "/" -f7`
  echo $subjid
  echo $ses
done

If everything is correct then you can create your batch file.

Create batch script.

vi ~/scripts/ACAP/ENIGMA-preprocess-batch.sh

Copy and paste.

#!/bin/bash
curTime=`date +"%Y%m%d-%H%M%S"`
username=`id -un`
mkdir -p ~/logfiles/ACAP/${curTime}
ANTS_DIR=/work/${username}/data/ACAP/mindboggle/ants_subjects/
for i in $(find /work/${username}/BIDS/ACAP/ -name "*b900_dwi.nii.gz"); do
  subjid=`echo $i | cut -d "/" -f6`
  ses=`echo $i | cut -d "/" -f7`
  t1file=${ANTS_DIR}/${subjid}_${ses}/antsBrainSegmentation0N4.nii.gz
  if [ -f "$t1file" ]; then
    sbatch \
    -o ~/logfiles/ACAP/${curTime}/output_${subjid}_${ses}.txt \
    -e ~/logfiles/ACAP/${curTime}/error_${subjid}_${ses}.txt \
    ~/scripts/ACAP/ENIGMA-preprocess-job.sh \
    ${subjid} \
    ${ses}
    sleep 1
  fi
done

18.3 Submit Jobs with Batch Script

Submit jobs.

sh ~/scripts/ACAP/ENIGMA-preprocess-batch.sh

You can check the status of your jobs by checking if it is running or pending.

squeue

You can check the outputs.

tail -n 5 ~/logfiles/ACAP/<date>/output*

18.4 Download Directory / Sync Directory with Arc

Back on your local computer make sure you have the directory already.

mkdir -p /Volumes/bobo/data/ACAP/enigma/dwiprep/

Then sync the data from the remote computer to your local computer.

rsync -rauv \
--no-links \
ashley.ware@arc.ucalgary.ca:/work/ashley.ware/data/ACAP/enigma/dwiprep/ \
/Volumes/bobo/data/ACAP/enigma/dwiprep/

18.5 Clean Directory

If your dti_FA.nii.gz images look alright, then you can delete these files:

for i in $(ls /Volumes/bobo/data/ACAP/enigma/dwiprep/); do
  cd /Volumes/bobo/data/ACAP/enigma/dwiprep/$i
  mv dwi-eddy.eddy_rotated_bvecs dwi.bvec
  mv dwi-epicorr.nii.gz dwi.nii.gz
  mv dwi-epicorr_mask.nii.gz mask.nii.gz
  rm acqparams.txt
  rm b0.nii.gz
  rm dwi_padded*
  rm dwi-eddy*
  rm dwi-epicorr-*
  rm index.txt
  rm t1.nii.gz
  rm t1tob0*
done

To sync the changes on the Baker cluster:

rsync -rauv \
--no-links \
--delete \
--exclude=dwi.bval \
/Volumes/bobo/data/ACAP/enigma/dwiprep/ \
ashley.ware@arc.ucalgary.ca:/work/ashley.ware/data/ACAP/enigma/dwiprep/ \