Computing

Computer Facilities:

Brain Imaging Core computing facilities consist of centralized computing infrastructure, a shared central compute resource, a walk-in image analysis bay, and desktop/laptop computers for BI lab personnel.

The centralized computing infrastructure is housed in a dedicated computer room with raised floor, air conditioning, and APC uninterruptable power supplies.  Central storage consists of 102 TB of network attached, 15k rpm Serial Attached SCSI RAID6 disk arrays. Ancillary file servers provide an additional 5.2TB of 15k rpm Serial Attached SCSI disks in a variety of RAID0 and RAID10 configurations. Total network storage is 107TB.  Data is made available to all Core workstations via nfs and smb/cifs protocols. Backups are provided by an HP StorageWorks MSL8096 library with three LTO5 tape drives which access the SAN fabric in a drive sharing configuration.  Total tape capacity is approximately 1.2 PetaByte.  Backups are controlled by EMC Legato enterprise backup software. The windows active directory domain is served by 2 quad-code Xeon servers with 4GB memory each and redundant power supplies.  A web application platform provides internet-based research paradigm presentation. Other servers provide web, database, monitoring/alerting, and other core network services.

A shared central computing resource is housed in the dedicated computer room.  This resource provides 142 Xeon and Athlon cores, 256GB of RAM, and 5TB of local scratch disk.  This resource is accessible to lab users both locally and over the internet, and has full connectivity to the Core storage, compute, and software resources.  The Condor high-throughput computing environment is used to manage resource allocation and scheduling.

The walk-in image analysis bay provides 13 shared OS X and windows workstations for Brain Imaging lab users.

The Brain Imaging Core also supports approximately 200 desktop, laptop, instrumentation, and data collection computers, running Scientific Linux, OS X, and windows operating systems. Computers within the facility have full connectivity to the Core storage, compute, and software resources.

Access to a wide variety of data analysis software is available throughout the Brain Imaging Core.  Most software packages are installed on every computer in the Core, including computers in offices as well as in the image analysis bay, which is designed for use by off-site researchers.  The following programming languages are actively used and supported: C/C++, Java, Perl, Python, TCL, IDL (Research Systems, Inc.), and Matlab (MathWorks).  The Core uses standard software for writing to CD/RW media, as well as a customized in-house program for performing robust data backup and archival to CD/RW disks.  Statistical analysis packages include SAS, SPSS, R, and Matlab.  Machine shop drawings use AutoCAD.  STELLA is used to model complex dynamic systems.  The Core includes several methodological research groups who vigorously produce in-house software to aid their investigations related to MRI, PET, and EEG, although were possible, the Core uses existing software obtained either commercially or from other academic research groups.

MRI:  Stimulus presentation is controlled by E-Prime software (Psychology Software Tools Inc.). The fMRI data are uploaded as DICOM files and preprocessed to correct for rigid body motion and image distortion caused by magnetic field inhomogeneity. Several software packages are available for analysis of fMRI data, including SPM99/SPM2/SPM5, SnPM, AFNI, fmristat, FSL, BrainVoyager, MedX, and VoxBo. Morphometric measurements can use AIR, FSL, Freesurfer, or SPM99/2/5 for coregistration. Both Freesurfer and in-house tools perform manual coregistration (BrainSqueezer) and distortion-based morphometry (DBM) measurements. Manual ROI drawing can use AFNI or an in-house tool (BrainMaker), automated ROI identification can be performed with Freesurfer. Diffusion Tensor Imaging (DTI) uses in-house software for analysis and display. Cortical flatmaps can be created using BrainVoyager and Caret.

PET: Manufacturers software is used for acquisition and reconstruction of PET data from the ECAT V7.2.2 and the Concorde/microPET; in-house software (Spamalize) is used for reconstruction of data from an offsite PET scanner. Images are coregistered using AIR, SPM99/2, or an in-house program (BrainSqueezer). ROI analysis is performed using an in-house tool (BrainMaker) for data from all three PET scanners. Blood flow studies are quantitated using in-house software. PET-FDG data are quantitated using Spamalize. Neurotransmitter and cardiac studies can be quantitated using Spamalize’s Patlak/Logan plot tool. SPM99/SPM2, fmristat, and SnPM are used for multiple subject data analysis.