Journal of Multimedia Theory and Applications (JMTA)

ISSN: 2368-5956

1. Introduction

Direct visualization of carotid atherosclerotic plaques provides useful insights on the natural history of atherosclerotic disease and aids in selecting appropriate treatments. Although several imaging modalities are available to assess atherosclerotic vessels, the two noninvasive techniques, mainly used are magnetic resonance imaging (MRI) and ultrasound (US). The challenge for imaging methods is to enable identification of patients with high-risk lesions that are vulnerable to thrombosis, so-called vulnerable plaques, before the occurrence of cerebrovascular complications. Noninvasive MRI has great potential to enable characterization of atherosclerotic plaque composition and morphology, which helps to assess plaque vulnerability. It differentiates plaque components on the basis of biophysical and biochemical parameters such as chemical composition and concentration, water content, physical state, molecular motion or diffusion [1]. Multispectral carotid MR imaging is able to characterize all major plaque components: active inflammation, thin cap with large lipid-necrotic core, endothelial denudation with superficial platelet aggregation, fissured plaque and stenosis by depicting particular combinations of signal intensities of each component on images obtained with different contrast weightings [2].

The limitations of MRI are that it is expensive, less accessible for use as a screening tool and lacks the temporal resolution necessary to evaluate instantaneously the dynamic characteristics of the carotid wall for estimation of tissue strain and biomechanical properties. Imaging of carotid dynamics is however possible with new US imaging systems. US imaging provides a convenient and inexpensive assessment of the carotid arteries and is currently being used in obtaining measurements of the total plaque area, plaque volume and vessel wall volume. It is less traumatic to the patient and offers a high temporal resolution enabling motion analysis to measure the distensibility of the artery and provides images that have higher in-plane resolution [3, 4].

Typically, only one of these examinations is performed, but, potentially the dynamic characteristics information and biomechanical characterization with US systems complement plaque composition information and high-resolution structural imaging with carotid MR. This motivates the use of multimodality approach to carotid plaque assessment. Spatial coregistration and combined visualization (fusion) of the images produced by MR and US modalities will allow a detailed comparison of both modalities in depicting the carotid bifurcation anatomy and lesions. Coregistration is an important first step in the use of complementary information obtained from fused images. It establishes spatial alignment between images from different modalities so that the exact point-to-point correspondence between image data sets is known. The techniques for multimodality registration fall into either feature-based, which establish correspondence between a limited set of identified salient points (landmarks) or intensity-based, which operate directly on image gray values. The applications of multimodality registration are abundant and diverse, predominantly diagnostic in nature. A coarse division would be into anatomical-anatomical registration, where images showing different aspects of tissue morphology are combined and functional-anatomical, where tissue metabolism and its spatial location relative to anatomical structures are related [5, 6].

2. Literature Review

Many researchers have developed intensity-based algorithms for multimodality fusion of carotid MR and US images. The first three-dimensional (3D) rigid registration algorithm for multimodal carotid images was reported by Slomka et al. [7], where they used voxel-based method to register 3D magnetic resonance angiography (MRA) images with 3D power Doppler (PD) US and indirectly with 3D B-mode US images. Subsequently, Chan et al. [8] assessed the accuracy of first non-rigid algorithm for fusion of 3D US and MR carotid data using the thin-plate-spline based deformable models. A biomechanical "twisting and bending" model was proposed by Nanayakkara et al. [9] for voxel-based non-rigid registration of multimodal carotid US and MR images. In order to correctly match two sets of images from different modalities, Rosas-Romero et al. [10] presented a novel approach for registration of 3D images based on optimal free-form rigid transformation. The reference objects for registration were the lumen of a carotid artery and the experiments were conducted to register sequences of MRI images to histology images of the carotid artery.

Feature-based approaches for multimodality registration have been explored to a lesser extent. The main difficulty with these approaches lies in the feature extraction step. Recently, Chiu et al. [11] developed the first geometric feature-based rigid registration algorithm for characterization of carotid plaque components. The algorithm used a surface-based iterative closest point (ICP) registration method to align surfaces reconstructed from US and MR images. A review paper dedicated to carotid image registration algorithms has been published by Gupta et al. [12]. During the past few years, efforts have been made to combine the advantages of both feature- and intensity-based approaches, resulting in a hybrid approach. A number of hybrid approaches have been developed and applied in the registration of brain and other images [13, 14 and 15]. However, the use of hybrid approaches for multimodality carotid image registration problem has not been reported, apart from the work by Carvalho et al. [16], who proposed a methodology to coregister freehand US and MR carotid images using a combination of point-based and intensity-based algorithms. Our recent preliminary communication [17] also described a novel hybrid approach, but that was intended for monomodality rigid registration of carotid US images. Table 1 gives an overview of multimodality carotid image registration methods discussed so far.

This work aims to coregister carotid MR and US images using a non-rigid methodology in a hybrid framework which combines the geometric and iconic features of both the images. The multimodality registration will allow the use of complementary information from different imaging modalities to establish a diagnosis or assist the clinician for a therapeutic gesture. To illustrate the gain in performance, quantitative and qualitative evaluations have been performed using a pilot set of clinical data.

3. Materials

Five subjects were selected for conducting the registration experiments. For each subject, MR and US images were obtained around the carotid bifurcation. The prior permissions were secured from all the patients for using their images in the experimental procedures and publishing the results on same for the purpose of this research work.

3.1. Magnetic Resonance Image Acquisition

All carotid MR scans were taken on a 1.5 Tesla system (Magnetom Avanto, Siemens Medical solutions, Erlangen, Germany) using custom-built matrix coils (head, neck and spine). Patients were positioned supine on the scanner table. The MR examinations for all patients included T1-, T2- and STIR (short inversion time inversion recovery)-weighted images acquired in the axial plane. The various imaging parameters TR/TE/flip-angle/slice-thickness/inter-slice-gap/in-plane-resolution/FOV were as follows: 600ms/20ms/150º/4mm/0.4mm/1.3 × 0.9 × 4mm³/240 × 240mm (T1), 4420ms/72ms/150º/4mm/0.4mm/ 0.8 × 0.5 × 4mm³/240 × 240mm (T2) and 6180ms/41ms/150º/4mm/0.4mm/1 × 0.9 × 4mm³/240 × 240mm (STIR) with time of inversion (TI) being 160ms.

Table 1. Overview of main parameters of the multimodality carotid image registration methods.

Reference	Imaging Modalities	Registration			Validation
		Similarity Measure	Transformation Model	Optimization	Quantitative	Qualitative
Slomka et al.[7]	3D MRA, 3D PD US and 3D B-mode US	Mutual Information (MI)	Voxel-based rigid registration algorithm	Simplex algorithm (Trilinear interpolation)	TRE (x,y,z) 0.32 ± 0.3 mm (rotational) 1.6 ± 2.1°	Visual verification: volume rendering and orthogonal slice display
Chan et al. [8]	US and MR	NMI	Rigid and non-rigid (Thin hyper-plate spline model)	Powell's direction set method (Trilinear interpolation)	Mean Absolute Distance (1.2 ± 0.16 mm)	Visual verification of orthogonal cut planes
Nanayakkara   et al. [9]	US and MR	MI	Twisting and bending model	Powell optimizer	Mean Registration Error (MRE)	Visual verification of overlapped vessels
Rosas-Romero et al. [10]	MR and Histology images	Distance function	Non-rigid using free-form deformations (Geometric approach)	Levenberg-Marquardt	-	Visual verification of registered data sets
Chiu et al. [11]	US and MR	Root-mean-square error	Rigid ICP (Geometric approach)	Quaternion based method	MRE (Phantom: 0.3 mm) (In vivo:    <1 mm)	Visual verification of superimposed slices
Carvalho et al. [16]	US and MR	Weighted sum of MI and Euclidean distance	ICP + Iconic (Hybrid approach)	Stochastic gradient descent (Linear interpolation)	MRE 0.05 mm (Dice similarity index)	Visual verification: volume rendering

3.2. Ultrasound Image Acquisition

Carotid US examinations were performed using the Voluson 730 Pro GE US machine having state-of-the-art user interface and exceptional image quality. A high frequency linear transducer SP 6-12 (Footprint: 38 x 4 mm, Bandwidth: 3-11 MHz and FOV: 37.4 mm) was used to acquire US images of the carotid artery by sweeping the neck of patients along the transverse plane.

3.3. Preparation of Datasets

All images were rescaled to be of size 512 × 512 pixels (135.50 mm height and 135.50 mm width) with pixel size 0.265 × 0.265 mm². Five experiments were performed to register a pair of US and MR images acquired from each of the five patients. The registration experiments were accomplished on a personal computer running Windows 7 operating system with an Intel Core I5-2450M processor 2.50 GHz and 8GB of RAM. The average time to complete the single registration was approximately 320 seconds for the experimental images reported in this work.

4. Proposed Hybrid Registration Methodology

The proposed hybrid algorithm divides the registration problem into two steps. In the first step, it uses the feature-based (geometric) approach to build explicit models of identifiable landmarks in the US image which are matched with their counterparts in the MR image. The second step uses the intensity-based (iconic) method to iteratively optimize a given similarity measure between the two images. Let M and U represent the carotid MR and US images to be registered, {X_M} represents the set of points in image M and {X_U} represents the corresponding set of points in image U. The purpose of proposed algorithm is to determine the registration transformation, T, which relates the position (x, y) of features in one image or coordinate space with the position (x′, y′) of the corresponding features in another image or coordinate space.

(1)

The block diagram in Fig. 1 gives an overview of the steps used for multimodality registration of carotid MR and US images. The non-rigid registration technique is required to account for differences in the patient's position and soft-tissue deformations between different image acquisition sessions.

4.1. Feature-Based Registration

The feature-based iterative closest point algorithm [18] has been employed that uses {X_M} as model point set and {X_U} as data point set to perform the initial rigid registration. The algorithm works by iteratively finding, for each point x_u, in the data set {X_U}, the closest point in the model set {X_M}. This is the point x_m in {X_M} for which the distance d between x_u and x_m is minimum

(2)

The set of closest points (one for each x_u) achieved is {P_U}. A least square rigid-body registration is then carried out between the two point sets {X_U} and {P_U} using the quaternion-based method [19]. The transformation obtained from this stage is applied to the set of data points {X_U} and then, the closest points are determined once again. The process iterates until the change in mean square error falls below a defined threshold of 1e-05. The final transformation from this stage, T_{G_rigid}, is used to create the roughly registered image U^T.

4.2. Intensity-Based Registration

Intensity-based approach takes the complete image data into account. The registration involves calculating a transformation between two images by iteratively optimizing some similarity measure calculated from all pixel values. The transformation in this stage consists of a global transformation, T_{I_rigid} and a local transformation, T_{I_non-rigid}. The global rigid transformation model has been used to map any point in the source image into the corresponding point in target image. In two dimensions, this model has three degrees of freedom: two translations and one rotation. To describe the local deformation, a free-form deformation (FFD) model based on B-splines [20] has been chosen because it is a powerful tool for modeling deformable objects and has been used successfully for registration of breast MR images [21].

The normalized mutual information (NMI) similarity metric [22] assessed the similarities between the MR and US images in both global and local transformation models since it is robust and appropriate for multimodality image registration. NMI is defined using image entropies as:

(3)

where H(M) and H(U^T) denote the Shannon-Wiener entropies of images M and U^T respectively and are defined as:    

(4)

(5)

The joint entropy H(M, U^T) of images M and U^T is defined as:

(6)

The joint probability p_MU^T(m,u) and the marginal probabilities p_M(m) and p_U^T(u) have been estimated from the 256 × 256 normalized joint histogram of the images. The optimal value of NMI similarity metric in global and local transformation models has been determined using Powell and limited-memory quasi-Newton optimizers respectively. When two images are geometrically aligned, NMI is maximal. Bicubic interpolation is employed to resample the US image data at non-grid positions. The proposed hybrid registration algorithm has been implemented in MATLAB (The MathWorks, Natick, MA).

5. Experiments and Results

5.1. Quantitative Evaluation

       The accuracy of proposed registration method is assessed using a geometric measure called target registration error (TRE), which is defined as the displacement between two corresponding points after registration [23]. Let p represents a point in the first image of a pair to be registered and q, a point in the second image. If T is the overall transformation that registers first image to the second, TRE is then given as:

(7)

The proposed registration procedure has been evaluated in five different configurations: geometric ICP rigid transformation (T_{G_rigid}), composite of geometric and iconic rigid transformations (T_{I_rigid} o T_{G_rigid}) and the complete method (T_{I_non-rigid} o T_{I_rigid} o T_{G_rigid}) with three different values of control point spacing (32 mm, 16 mm and 8 mm). Table 2 summarizes the average values of TRE for the intrasubject multimodality registration of carotid MR and US image datasets. The box plot in Fig. 2 shows the comparison of different registration approaches in terms of TRE. The first box shows the results of geometric ICP rigid registration procedure. The second box shows the results after applying the subsequent iconic rigid registration. The remaining three boxes show the results of complete method after applying B-spline based FFD with three different values of control point spacing.

Table 2. Comparison of different registration procedures in terms of the average values of TRE for the intrasubject multimodality registration of carotid MR and US image datasets.

Registration procedure	Transformation settings	TRE (mm)
No registration	-	9.00475
ICP rigid	TG_rigid	0.54436
ICP rigid + Iconic Rigid	TI_rigid o TG_rigid	0.41209
ICP rigid + Iconic rigid + B-spline (32 mm)	TI_non-rigid o TI_rigid o TG_rigid	0.34530
ICP rigid + Iconic rigid + B-spline (16 mm)	TI_non-rigid o TI_rigid o TG_rigid	0.32765
ICP rigid + Iconic rigid + B-spline (8 mm)	TI_non-rigid o TI_rigid o TG_rigid	0.10941

It is evident from the results that the registrations which are based on pure geometric transformations reduce TRE significantly and thus, can be used to provide the rough initial alignment between the images. Subsequently, the rigid iconic approach reduces the error slightly. Finally, the B-spline based FFD drops the error appreciably. The proposed hybrid method achieved a TRE of 0.10941 mm which is significantly less by 79.9% in comparison to pure geometric approach (TRE = 0.54436 mm) and by 73.4% as compared to the composite of rigid geometric and iconic approaches (TRE = 0.41209 mm). Another observation made from the study is that the hybrid method performs better as the resolution of the control point mesh of the spline-based FFD increases. The main reason for this is the increased flexibility of the spline-based FFD to describe local deformations of the carotid artery as the number of control points increases. Overall, the proposed hybrid method with control point spacing of 8 mm performs the best in terms of TRE.

5.2. Qualitative Assessment

For qualitative validation of the proposed hybrid registration method, all results were visually examined by two clinical radiologists. The quality of the results was rated as: "correctly aligned", "satisfactorily aligned" and "poorly aligned". Successful registration was defined as the registration for which the visual agreement was "correctly aligned".  The results of such assessment are presented in Table 3. Overall, 80% (4 out of 5) tests for five considered patients were judged to be "correctly aligned" after the application of the proposed algorithm. In the remaining 20% (1 out of 5) cases, the registration was judged to be satisfactory by both the radiologists.

Table 3. Qualitative assessment of the proposed hybrid algorithm for multimodality registration of carotid MR and US image datasets.

Rating	No. of cases
Correctly aligned	4
Satisfactorily aligned	1
Poorly aligned	0

An example of unregistered carotid MR and US images acquired from a patient in transverse view is shown in Fig. 3. The results obtained after coregistration of US image with the corresponding MR image have been shown in Fig. 4. The results are calculated using three different settings of transformation models (i) ICP-based geometric rigid, (ii) composite of geometric and iconic rigid and (iii) proposed hybrid. For the B-spline based non-rigid registration technique, a control point spacing of 8 mm was used since, this had provided the best results. The registered carotid US images are fused with the corresponding MR image for an integrated display with MR being shown in green channel and US in magenta channel.

The results in Fig. 4a clearly show that after the pure geometric registration, there are significant misalignments between MR and US lumen segmentations. Subsequently, the iconic rigid registration improves the results slightly, however, the double edges can still be seen (Fig. 4b). Finally, after applying the proposed hybrid method, the lumen segmentations are perfectly overlapped (Fig. 4c). Thus, the visual assessment of results divulges that the proposed hybrid method with control point spacing of 8 mm can eliminate misregistration artifacts completely and hence, aid clinical interpretation.

6. Conclusions and Discussion

MR imaging of carotid atherosclerotic plaques provides excellent soft-tissue characterization, but lacks the temporal resolution. On the other hand, US provides information on carotid dynamics, but lacks the spatial resolution of carotid MRI. Multimodality registration of MR and US data is essential for accurately correlating the findings obtained by both modalities for the purpose of diagnosis, treatment and basic sciences. It provides unique information which is often not available from independent analysis of each modality. In this work, a hybrid approach based non-rigid registration methodology has been developed and evaluated for the fusion of the carotid MR and US images. Five sets of intrasubject carotid MR and US images were coregistered using the proposed hybrid algorithm. The ICP algorithm has been used to provide the initial alignment of images. Subsequently, the intensity-based rigid transformation model optimizes NMI similarity measure using Powell optimizer. Finally, non-rigid registration is carried out using B-spline based FFD model to achieve more precise alignment. The registered images have been fused to create a composite image from two images. In order to evaluate the necessity of each step and to establish the optimal configuration, different registration approaches were compared quantitatively using TRE. The qualitative validation was performed by clinical radiologists using visual inspection scheme. The results showed that the proposed hybrid method successfully aligned MR and US images, allowing multimodal analysis of atherosclerotic plaque in the carotid artery.

Acknowledgements

Authors are highly grateful to radiologists Dr. Simmi Garg and Dr. Paramdeep Singh from Guru Gobind Singh Government Medical College and Hospital, Faridkot (Punjab, India) for their enormous co-operation and constant guidance during this study. Authors also acknowledge the valuable help given by Mr. Yashpaul Samberia, radiology supervisor and Mr. Anirudh Sharma, MRI technologist from the same institute.

References