Chapter 3 Programming and Tools

3.1 Tips for Neuroimage processing

  • before using -mas fucntion in FSL, make sure they have same FOV and resoluiton

  • Remove skull first then do the registration !!!

  • Make a good plan for testing/trials and errors to debug efficiently; e.g. name every file and folder in well-formated and structured way; to see if the pipeline is working or not by check output files

3.2 Useful Regex examples

  • How to find the content inside of Square bracket?

You can use the following regex globally: find content in square bracket

\[(.*?)\]
  • \[ : [ is a meta char and needs to be escaped if you want to match it literally.
  • (.*?) : match everything in a non-greedy way and capture it.
  • \] : ] is a meta char and needs to be escaped if you want to match it literally.

[3-7] will match a single digit in the range 3 to 7.

  • How to Match multiple digits
\d\d       will match 2 consecutive digits
\d+        will match 1 or more consecutive digits
\d*        will match 0 or more consecutive digits
\d{3}      will match 3 consecutive digits
\d{3,6}    will match 3 to 6 consecutive digits
\d{3,}     will match 3 or more consecutive digits

NOTICE! The \\d in the above examples can be replaced with a number range:

[3-7][3-7]    will match 2 consecutive digits that are in the range 3 to 7
[3-7]+        will match 1 or more consecutive digits that are in the range 3 to 7
[3-7]*        will match 0 or more consecutive digits that are in the range 3 to 7
[3-7]{3}      will match 3 consecutive digits that are in the range 3 to 7
[3-7]{3,6}    will match 3 to 6 consecutive digits that are in the range 3 to 7
[3-7]{3,}     will match 3 or more consecutive digits that are in the range 3 to 7

3.3 Useful Shell commands and examples

3.3.1 Find files and copy to target folder


cd /project/3015083.01/processed
procfolder=/project/3015083.01/processed
for ID in 036 090 090 140 161 190 200 221 286 296 300 308 363 366 367 404 434 439 444 485 507;
do
find -name sub*ivim-data.nii.gz -exec cp {} /project/3015083.01/mengfei/Munich_IVIM/IVIM \;
cp "${procfolder}"/sub-"${ID}"/ses-mri01/ivim/sub-"${ID}"_ses-mri01_ivim-data.nii.gz /project/3015083.01/mengfei/Munich_IVIM/IVIM/
done

3.3.2 Kill a range of running batch jobs

qdel {18280..18285}

3.3.3 sed

Using sed to change file names

> echo "./pentaray_run2/Trace_220560.dat" |  sed 's/.*Trace_//' | sed 's/.dat//'

> output:  220560.dat --> 220560 

`.*` is very important

3.3.4 tr

The tr command in UNIX is a command line utility for translating or deleting characters. It supports a range of transformations including uppercase to lowercase, squeezing repeating characters, deleting specific characters and basic find and replace. It can be used with UNIX pipes to support more complex translation. tr stands for translate.

Syntax :

$ tr [OPTION] SET1 [SET2] Options

-c : complements the set of characters in string.i.e., operations apply to characters not in the given set -d : delete characters in the first set from the output. -s : replaces repeated characters listed in the set1 with single occurrence -t : truncates set1

Sample Commands

  1. How to convert lower case to upper case

To convert from lower case to upper case the predefined sets in tr can be used.

$cat greekfile Output:

WELCOME TO GeeksforGeeks $cat greekfile | tr “[a-z]” “[A-Z]” Output:

WELCOME TO GEEKSFORGEEKS or

$cat geekfile | tr “[:lower:]” “[:upper:]” Output:

WELCOME TO GEEKSFORGEEKS

  1. How to translate white-space to tabs

The following command will translate all the white-space to tabs

$ echo “Welcome To GeeksforGeeks” | tr [:space:] ’ Output:

Welcome To GeeksforGeeks

  1. How to translate braces into parenthesis You can also translate from and to a file. In this example we will translate braces in a file with parenthesis.

$cat greekfile Output:

{WELCOME TO} GeeksforGeeks $ tr ‘{}’ ‘()’ newfile.txt

Output:

(WELCOME TO) GeeksforGeeks The above command will read each character from “geekfile.txt”, translate if it is a brace, and write the output in “newfile.txt”.

  1. How to use squeeze repetition of characters using -s

To squeeze repeat occurrences of characters specified in a set use the -s option. This removes repeated instances of a character. OR we can say that,you can convert multiple continuous spaces with a single space

$ echo “Welcome To GeeksforGeeks” | tr -s [:space:] ’ ’ Output:

Welcome To GeeksforGeeks

  1. How to delete specified characters using -d option

To delete specific characters use the -d option.This option deletes characters in the first set specified.

$ echo “Welcome To GeeksforGeeks” | tr -d ‘w’ Output:

elcome To GeeksforGeeks

  1. To remove all the digits from the string, use

$ echo “my ID is 73535” | tr -d [:digit:] Output:

my ID is

  1. How to complement the sets using -c option

You can complement the SET1 using -c option. For example, to remove all characters except digits, you can use the following.

$ echo “my ID is 73535” | tr -cd [:digit:] Output:

73535

for i in $(ls);
do
  mv $i ${i:(-3)};
done  # extract the final three characters

3.3.5 Batch job submit

list="/project/3015083.01/mengfei/script/psmd_2006_list.sh"

while IFS= read -r line
do
    #echo "this_is: ${line} "
    #echo "ID_is...."
    ID=$(echo ${line} | awk -F'[ ]' '{print $2}')   ## echo is important here; field separator is [space]
    echo "sub_ID: ${ID}.....calculating PSMD....."
    echo "$line -i" | qsub -N \'PSMD_${ID}\' -l 'nodes=1:ppn=4,mem=6gb,walltime=6:00:00'  # baclslash was used to escape the single quote, so that ${ID} can be contained.
done < $list

3.3.6 Copy files after checking file existence


cd /project/3015083.01/processed
list=$(ls | head -3 | awk -F'-' '{print $2}')   # for sub-001, to extract 011.
##ID=$(echo ${line} | awk -F'[ ]' '{print $2}') ## echo is important here; field separator is [space]
for ID in ${list};
do
    echo "this is ${ID}"
    cd /project/3015083.01/nifti/sub-"${ID}"/ses-mri01/anat/
    if [ -f "*flair.nii.gz" ] ; then
        /project/3015083.01/mengfei/raw_backup/nifti/resliced_to_2006

    else
        echo "FLAIR nifti does not exist."
    fi
done

3.4 ITK_SNAP command line tutorials

~$ itksnap -h

ITK-SnAP Command Line Usage:

 /Applications/ITK-SNAP.app/Contents/MacOS/ITK-SNAP \[options\] \[main\_image\]

Image Options:

 -g FILE              : Load the main image from FILE

 -s FILE              : Load the segmentation image from FILE

 -l FILE              : Load label descriptions from FILE

 -o FILE \[FILE+\]      : Load additional images from FILE

 :   (multiple files may be provided)

 -w FILE              : Load workspace from FILE

 :   (-w cannot be mixed with -g, -s, -l, -o options)

Additional Options:

 -z FACTOR            : Specify initial zoom in screen pixels/mm

 --cwd PATH           : Start with PATH as the initial directory

Debugging/Testing Options:

 --debug-events       : Dump information regarding UI events

 --test list          : List available tests.

 --test TESTID        : Execute a test.

 --testdir DIR        : Set the root directory for tests.

 --testacc factor     : Adjust the interval between test commands by factor (e.g., 0.5).

3.4.1 ITK-SNAP open serial images once for all

module load itk-snap
#cd /project/3015083.01/processed/
dir=/project/3015006.07/Data

opensubject(){
itksnap -g /opt/fsl/6.0.3/data/standard/MNI152_T1_2mm.nii.gz -o /project/flair_brain.nii.gz \

 &> /dev/null
}

while :
do
echo ""
echo "Enter ID (e.g. 005) for viewing or leave blank for rating of next subject. Press Ctrl+C to exit!"
read number
[ -z $number ] && runloop || opensubject
done

3.5 Compress files/folders with gzip and tar

procfolder=/project/3015083.01/processed

cd ${procfolder}  # this is home folder for processed folders
#/project/3015083.01/processed/sub-004/ses-mri01/SWI/
swi_folder="SWI"
for subject in $(ls);

do
    echo ${subject}
    cd "${procfolder}"/"${subject}"/ses-mri01
    if [ -d "${procfolder}"/"${subject}"/ses-mri01/SWI ]; then  ## space after "[" is very important
        cd "${procfolder}"/"${subject}"/ses-mri01/SWI
        echo "found SWI folder and compressing...."
        gzip *.nii
    else
        echo "there is no SWI folder"
        cd ${procfolder}
    fi
done
## examples for zip a folder [lotte]
tar -zcvf lotte.zip lotte/

Often, you’ll create a zip archive of a directory including the content of subdirectories. The -r option allows you to traverse the whole directory structure recursively:

zip -r archivename.zip directory_name

You can also add multiple files and directories in the same archive:

zip -r archivename.zip directory_name1 directory_name2 file1 file1