|
|
1
|
|
|
Super-Resolution Reconstruction of Electrical Impedance Tomography Images
|
|
|
R. A. Borsoi, J. C. C. Aya, G. H. Costa, J. C. M. Bermudez
|
|
|
|
|
|
arXiv:1701.00031v2 [cs.CV] 23 Oct 2017
|
|
|
|
|
|
Abstract--Electrical Impedance Tomography (EIT) systems are becoming popular because they present several advantages over competing systems. However, EIT leads to images with very low resolution. Moreover, the nonuniform sampling characteristic of EIT precludes the straightforward application of traditional image super-resolution techniques. In this work, we propose a resampling based Super-Resolution method for EIT image quality improvement. Results with both synthetic and in vivo data indicate that the proposed technique can lead to substantial improvements in EIT image resolution, making it more competitive with other technologies.
|
|
|
Index Terms--Tomography, Image resolution, Image reconstruction
|
|
|
I. INTRODUCTION
|
|
|
Real-time imaging systems are particularly important in medicine as they provide useful means of monitoring the patient in intensive care or during medical procedures like intraoperative image guidance. However, most intraoperative imaging systems are expensive, offer radiation risks and do not operate in real-time [1]. Likewise, real-time monitoring can avoid lung overdistension and collapse in mechanical lung ventilation, reducing ventilator-associated pneumonia rates [2].
|
|
|
Electrical Impedance Tomography (EIT) is an imaging method that leads to low cost, highly portable, real-time and radiation-free imaging devices. These characteristics make it suitable for bedside diagnosis and intraoperative imaging. However, the lower image quality when compared to other tomography methods still keeps EIT from being widely applied in medical diagnosis and guidance.
|
|
|
The image resolution in tomography methods such as Computed Tomography (CT) and Positron Emission Tomography (PET) has significantly improved with the direct application of Super-Resolution Reconstruction (SRR) techniques. SRR consists basically in the reconstruction of a high resolution (HR) image by extracting non-redundant information from several low resolution (LR) images (usually acquired in the presence of motion) of the same object. SRR was originally a technique intended for overcoming physical limitations of optical imaging sensors. More recently, it has been successfully adapted for other imaging systems such as PET and CT, leading to the improvement of the resulting image quality [3].
|
|
|
Although SRR algorithms developed for optical systems require some adaptations for super-resolving tomographic
|
|
|
This work has been supported by the the National Council for Scientific and Technological Development (CNPq).
|
|
|
R. A. Borsoi and J. C. M. Bermudez are with the Department of Electrical Engineering, Federal University of Santa Catarina, Floriano<6E>polis, Brazil
|
|
|
J. C. C. Aya and G. H. Costa are with the Department of Mechanical Engineering, University of Caxias do Sul, Caxias do Sul, Brazil
|
|
|
|
|
|
images originating from PET or CT, the principle of operation of these systems is quite similar to that of optical image acquisition systems. An EIT system, on the other hand, is based on different operation principles. It employs diffuse electrical currents instead of a coherent X-ray beam to generate images, which makes the image acquisition process inherently nonlinear. Besides, the use of traditional finite element (FE) techniques for solving the EIT inverse problem leads to an image represented by a nonuniform grid of conductivity values, as opposed to the uniformly sampled images acquired by sensor grids used in optical systems. These characteristics preclude the direct application of traditional SRR methods to EIT.
|
|
|
Recently, a new SRR method has been developed for the reconstruction of optical images generated by hypothetical nonuniform pixel arrays called Penrose Pixels [4]. Penrose tiling is a method of aperiodic tiling that uses different geometric forms at different orientations to fill a plane [5]. Rhombus Penrose tiling is a specific type of aperiodic tiling that employs only two types of rhombus of equal side length. To allow the representation of the nonuniform pixel grid using matrix-vector notation, it is proposed in [4] that the LR images be upsampled to a regular HR grid as soon as they are acquired. The upsampling operation is dependent on the sensor layout map. The resulting image is called Intermediate HR (IHR) image. After upsampling, image registration and SRR algorithms can be applied on the IHR grid, where the processes of downsampling to and upsampling from the nonuniform grid are incorporated to the reconstruction algorithm by means of an acquisition model. This method has been shown to be capable of achieving larger magnification factors than those obtained using regular pixel arrays, as there are no constraints concerning (sub)pixel displacements between the LR observations because the grid has no translational symmetry. Unfortunately, however, such sensor arrays are currently not being manufactured, and this promising technique remains largely without practical application.
|
|
|
In this paper, a new super-resolution strategy for EIT images is proposed. By employing the resampling strategy used in [4] to deal with the aperiodic tiling characteristic of Penrose pixels, we developed a method for super-resolving images generated with arbitrarily irregular tiling such as EIT images, which are composed of non-uniformly spaced pixels forming a finite element mesh (FEM). The EIT data acquisition system is modeled as in an optical SRR problem with nonuniform sampling, and the image reconstruction is performed independently from the EIT inverse problem. This allows the proposed methodology to be used with arbitrary EIT algorithms and FEMs, which includes commercial EIT
|
|
|
|
|
|
2
|
|
|
|
|
|
systems already in operation. Quantitative and perceptual evaluations show substantial improvements in the resolution of the reconstructed images when compared to the original ones in LR. This observation remains consistent for different choices of algorithms used in the EIT inverse problem.
|
|
|
This paper is organzized as follows. In Section II the basic principles of EIT and SRR are explained. In Sections III and IV, the EIT imaging system is modelled as a nonuniform optical imaging system. Results illustrating the super-resolution of EIT images are presented in Section V. Finally, Section VI concludes this paper.
|
|
|
|
|
|
II. OPERATION PRINCIPLE OF EIT AND SRR
|
|
|
A. The EIT Working Principle
|
|
|
In electrical impedance tomography, the electrical conductivity (p) inside a domain is estimated from voltage and current measurements {V (p), I(p)} at its boundary , where p denotes the spatial position vector. The physical process relating the electrical conductivity of the domain with the measurements is described by the following partial differential equations [2]:
|
|
|
|
|
|
div (p)V (p) = 0 , p
|
|
|
|
|
|
V (p)
|
|
|
|
|
|
(1)
|
|
|
|
|
|
(p)
|
|
|
|
|
|
= j(p) ,
|
|
|
|
|
|
p
|
|
|
|
|
|
where div(<28>) is the divergence operator, is an outwardlypointing unitary vector normal to and j(p) is the current density on , satisfying j(p)dp = 0.
|
|
|
Since the voltage and current values are measured through a finite number of electrodes attached to the surface of the body, this discretization effect must be represented in some form [6]. One simple model for the influence of the electrodes is the gap model, where the boundary condition of (1) is described as:
|
|
|
|
|
|
V (p)
|
|
|
|
|
|
(p)
|
|
|
|
|
|
=
|
|
|
|
|
|
|
|
|
|
|
|
Il/Al , p El , l = 1, . . . , L 0 , p / Ll=1El
|
|
|
|
|
|
(2)
|
|
|
|
|
|
where L is the number of electrodes, and Il and Al are, respectively, the current and the area of the l-th electrode,
|
|
|
|
|
|
which is denoted by El. The region of the domain boundary
|
|
|
|
|
|
occupied by the l-th electrode is denoted by El , and
|
|
|
|
|
|
the voltage V (p) at the center of El is denoted by Vl. The
|
|
|
|
|
|
condition j(p)dp = 0 is replaced with
|
|
|
|
|
|
L l=1
|
|
|
|
|
|
Il
|
|
|
|
|
|
=
|
|
|
|
|
|
0,
|
|
|
|
|
|
and
|
|
|
|
|
|
the integral of j(p) over El equals Il.
|
|
|
|
|
|
Besides (1) and (2), one must also consider the effect of the
|
|
|
|
|
|
high conductivity of the electrodes in the measured voltages.
|
|
|
|
|
|
This can be performed either by simple models like the shunt
|
|
|
|
|
|
model, which considers Vl to be constant on El , or using more elaborate models which take the contact impedance
|
|
|
|
|
|
between the electrode and the body into account [7]. Along
|
|
|
|
|
|
with the choice of a ground level such as
|
|
|
|
|
|
L l=1
|
|
|
|
|
|
Vl
|
|
|
|
|
|
=
|
|
|
|
|
|
0,
|
|
|
|
|
|
this
|
|
|
|
|
|
set
|
|
|
|
|
|
of considerations constitute the so-called complete electrode
|
|
|
|
|
|
model [7].
|
|
|
|
|
|
With this model, one can fully describe the behavior of
|
|
|
|
|
|
the voltages VF = [V1, . . . , VL] in the electrodes at the object/domain boundary El given the conductivity of the medium (p) and a set of injected currents IF = [I1, . . . , IL].
|
|
|
|
|
|
This process is described in compact form by a nonlinear forward operator defined by F((p), IF ) = VF , and constitutes the EIT Forward Problem [2].
|
|
|
Given a set of L-1 linearly independent voltage and current measurements IkF = [I1k, . . . , ILk ] and VFk = [V1k, . . . , VLk], k = 1, . . . , L - 1, the goal of the EIT inverse problem is to find the approximate conductivity y = ^(p) such that the voltages predicted through the forward problem V = F(^(p), I1F , . . . , ILF-1) 1 are as close to the actual measurements VM = [VF1 , . . . , VFL-1] as possible [2].
|
|
|
Many methods exist for solving the EIT inverse problem
|
|
|
(including for example neural networks [8] and the D-Bar
|
|
|
method [9]). One of the most prominent choices consist of
|
|
|
employing Finite Element (FE) techniques for the numerical
|
|
|
computation of the EIT forward model based on a discrete approximation of the domain . The EIT inverse problem can
|
|
|
then be described by the following optimization problem [2]:
|
|
|
|
|
|
y
|
|
|
|
|
|
=
|
|
|
|
|
|
arg min (p)
|
|
|
|
|
|
VM - F((p), I1F , . . . , ILF-1) 2
|
|
|
|
|
|
(3)
|
|
|
|
|
|
+ EIT R((p)) ,
|
|
|
|
|
|
where R(^(p)) is a regularization functional that includes additional information about the solution. Common choices for the regularization term include the Thikonov and Total Variation regularizations [10], [2]. The scalar EIT is the regularization parameter.
|
|
|
In many applications of EIT such as lung monitoring, nonidealities like breathing motion and electrode misplacement can introduce substantial artifacts in images obtained as solutions to optimization problem (3). Therefore, it is a common practice to employ linearized difference imaging in medical applications to reduce the occurrence of these image artifacts [2]. In this case, first a set of voltages Vref corresponding to a reference conductivity distribution ref(p) is acquired beforehand. Afterwards, for the measured voltages VM we try to reconstruct the conductivity change (p) = ^(p) - ref(p) in the body for a linearized version of the forward operator F (<28>), corresponding to the optimization problem
|
|
|
|
|
|
y
|
|
|
|
|
|
=
|
|
|
|
|
|
arg min (p)
|
|
|
|
|
|
V - J(p)
|
|
|
|
|
|
2 + EIT R((p)) ,
|
|
|
|
|
|
(4)
|
|
|
|
|
|
where V = VM - Vref is the measured voltage difference and J is the Jacobian of F(<28>). The optimization problems (3) and (4) can be solved in various ways like, for instance, either by using Newton-Rhaphson or Gauss-Newton techniques [2]. More advanced techniques are employed in the case of nonsmooth regularization terms [10].
|
|
|
It is important to note that more advanced methods in EIT image reconstruction consider optimization problems that might differ from (3). These works mainly search for measures of similarity for predicted and measured voltages, which leads to an increased robustness to modeling errors, and use regularization terms that promote sparsity and steep variations in the solution [11], [12], [13], [10].
|
|
|
|
|
|
1Note that the forward operator F((p), I1F , . . . , IkF ) describes an arbitrary number of sets of voltage measurements VF1 , . . . , VFk, depending on the number k of input current measurements I1F , . . . , IkF .
|
|
|
|
|
|
3
|
|
|
|
|
|
B. Traditional Super-Resolution Reconstruction
|
|
|
The objective of SRR techniques is to obtain one or more images of high resolution (HR) from a set of observed low resolution (LR) images. This is usually done by solving an inverse problem, which is formulated based on a mathematical model for the relationship between the different HR and LR images. The super-resolution of images deriving from conventional imaging sensors has received a lot of interest, and is already a consolidated methodology [14]. In this case, the acquisition process of an image at the t-th discrete time instant can be modeled as [15]
|
|
|
|
|
|
y(t) = DHx(t) + e(t) ,
|
|
|
|
|
|
(5)
|
|
|
|
|
|
where y(t) (N 2<>1) and x(t) (M 2<>1) are the lexicographic (i.e.
|
|
|
vectorized) representations of the degraded (LR) and original (HR) images, respectively. Matrix D (N 2 <20> M 2) models the subsampling taking place at the sensor, and H (M 2 <20> M 2)
|
|
|
models the blurring in the acquisition process. These matrices are assumed to be known in advance. Vector e(t) (N 2 <20> 1)
|
|
|
models the observation noise in the sensor. In the case of video sequences, the relationship between
|
|
|
each pair of HR images can be modelled by [16]
|
|
|
|
|
|
x(t) = G(t)x(t - 1) + s(t) ,
|
|
|
|
|
|
(6)
|
|
|
|
|
|
where G(t) is a warp matrix that represents the relative displacement between the HR images x(t - 1) and x(t) at adjacent time instants. The motion between the images is estimated from the LR observations using image registration techniques, and can be described either by simple parametric models (such as global rotations and translations) or by a dense motion field, where each pixel of the image can move independently of the others [17], [18]. Vector s(t) models the innovations in x(t), which consists of unpredictable changes happening in the video sequence between time t - 1 and t, i.e. the information in x(t) which cannot be represented as a linear combination of the pixels in x(t - 1), usually due to scene changes such as occlusions.
|
|
|
The reconstruction of the t-th image based on L previously observed frames is performed by minimizing some metric of the difference between the degraded HR (estimated) image and the LR observations. For the case of the squared L2 norm, the minimization problem is formulated as [15]:
|
|
|
|
|
|
x^(t) = arg min
|
|
|
x(t)
|
|
|
|
|
|
L-1
|
|
|
y(t - l) - DH Mt-l,t x(t) 2
|
|
|
|
|
|
l=0
|
|
|
|
|
|
+ SRR Sx(t) 2
|
|
|
|
|
|
(7)
|
|
|
|
|
|
where Mt-l,t represents the total motion from the t-th to the (t - l)-th image (which is obtained from the dynamical model (6)). The last term is a regularization that incorporates additional a priori information about the desired solution. This is usually performed by assuming that the images are smooth, which results in the matrix S being a high pass filter (e.g. a Laplacian). The scalar SRR is the regularization parameter.
|
|
|
Note that the SRR process described considers imaging sensors consisting of regular arrays of square pixels. EIT images from a finite element mesh, on the other hand, consist of a nonuniform array of irregular elements. This renders the direct
|
|
|
|
|
|
application of the SRR methodology described above unfeasible. In the following, we shall represent the EIT image formation process as an optical imaging system with nonuniform pixels. This will allow for the application of a SRR methodology previously designed for the reconstruction of images originating from hypothetical sensors composed by Rhombus Penrose tiling [4], [5].
|
|
|
III. RESAMPLING-BASED SUPER-RESOLUTION FOR EIT
|
|
|
The EIT image formation process, which is derived from the EIT inverse problem in (3) or (4), does not resembles an optical image acquisition system as described in (5). In this case, each acquired image consists of a nonuniform mesh (i.e. a finite element mesh - FEM) composed of elements of arbitrary sizes and shapes, each one presenting a constant conductivity value. Moreover, the diffuse nature of the electrical currents injected in the boundary of the domain and the ill-posedness of the inverse problem inflict severe distortions on the resulting image. We will assume these distortions to be appropriately represented by a combination of a space invariant blurring and a nonlinear, spatially variant distortion. Hence, the typical mathematical modeling of the EIT image formation is quite different from the model used in optical SRR algorithms.
|
|
|
In this work we propose to model the EIT image acquisition process using a formulation similar to that used for traditional image systems. An overall illustration of the proposed methodology can be seen in Figure 1. By considering each element of the finite element mesh as a pixel of a nonuniform sensor, we propose to employ the Penrose SRR strategy to super resolve the low resolution tomography images. The EIT images are upsampled from the finite element mesh into a uniform grid (IHR) before being processed by a super resolution algorithm. For the t-th time instant, this process is represented as y(t) = y(t), where y(t) denotes the upsampled image and (<28>) the usampling operator. As will be discussed later, the reconstruction method also requires the conjugate downsampling process to be performed. It consists on the transition from the IHR to the LR grid, which is done by assigning to each element of the nonuniform mesh the average value of all the pixels within the corresponding area of the IHR grid. This process is illustrated in Figure 2.
|
|
|
It should be noted that by assigning a position to the center of each pixel in the regular IHR grid (which encloses the whole FEM domain), the value of each IHR pixel can be computed straightforwardly during the upsampling process by determining which element of the FEM circumscribes its position. This is performed using the geometric information about the FEM, which is available with the LR EIT images [2]. Since the position of the center of each IHR pixel yi(t) lies inside an unique LR image element yp (t) for some p, the upsampling process is identical to the uniform impulsive sampling of the nonuniform LR grid, where we assign to yi(t) the value of yp (t).
|
|
|
We also propose to model the non-linear distortions caused by the inverse problem using a linear filtering approximation (such as the typical optical blurring model), thereby only addressing the space invariant and more well-behaved part of the blurring process. Although the nonlinear and spatially
|
|
|
|
|
|
4
|
|
|
|
|
|
Fig. 1. Illustration of the proposed SRR methodology for EIT images. First row: real (HR) images of a physical body subject to motion, from which the voltage and current measurements are derived. Second row: The EIT (LR) images are reconstructed from the measurements using FEM techniques in the EIT inverse problem. Third row: The LR FEM images are upsampled to an uniform grid, resulting in the IHR images. Fourth row: The proposed SRR technique is applied to estimate the HR images from the IHR observations.
|
|
|
|
|
|
Upsample (uniform grid)
|
|
|
|
|
|
IV. THE PROPOSED EIT IMAGING MODEL
|
|
|
|
|
|
The proposed model for the EIT image formation can be mathematically described as
|
|
|
|
|
|
Downsample (nonuniform grid)
|
|
|
|
|
|
(a)
|
|
|
|
|
|
(b)
|
|
|
|
|
|
Fig. 2. Illustration of the resampling process. (a) EIT image given by a nonuniform mesh, with constant conductivity value per element. (b) IHR image on the uniform grid. The upsampling process consists of sifting the finite element mesh in order to generate an uniform IHR. The downsample process averages every pixel on the uniform grid that falls within the area of a single element of the mesh, assigning this value for the corresponding element on the nonuniform grid.
|
|
|
|
|
|
varying portion of the distortion is not accounted for, it shall nevertheless be shown that the resulting SRR performance employing this model is good enough for practical purposes. This new formulation allows the direct application of image SRR techniques to EIT images.
|
|
|
|
|
|
y(t) = HD(t)HB(t)x(t) ,
|
|
|
|
|
|
(8)
|
|
|
|
|
|
where t is the temporal index, x(t) is the lexicographically ordered desired uniform HR image, and the vector y(t), with the same dimension of x(t), is the lexicographically ordered uniform IHR image. Notice that the EIT LR image is the downsampled IHR image y (t) = y(t), where (<28>) represents the downsampling operation illustrated in Figure 2. The resulting LR image, y (t), is a mesh composed of nonuniformly distributed elements (pixels), each with constant conductivity.
|
|
|
The matrix HB(t) models the EIT distortion with the linear approximation previously discussed, which can be computed in the form of a convolution operator. Matrix HD(t) is a spatially variant kernel convolution matrix performing the transformation from the HR to the IHR grid. Differently from the decimation performed in traditional SRR, HD(t) in (8) is a square matrix which is constructed based on the finite element mesh. When
|
|
|
|
|
|
5
|
|
|
|
|
|
NOSER TV TS
|
|
|
NOSER TV TS
|
|
|
|
|
|
Results for high SNR and finer mesh Volume Ratio Hausdorff Distance MASD (<28>10-2)
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
0.46 0.52/0.48 0.12 0.09/0.11 0.99 0.73/0.96
|
|
|
|
|
|
0.41 0.41/0.39 0.13 0.12/0.14 1.24 1.21/1.42
|
|
|
|
|
|
0.43 0.48/0.46 0.14 0.11/0.12 1.20 0.90/1.10
|
|
|
|
|
|
Results for low SNR and coarser mesh Volume Ratio Hausdorff Distance MASD (<28>10-2)
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
0.38 0.50/0.46 0.21 0.10/0.12 1.70 0.83/1.10
|
|
|
|
|
|
0.38 0.42/0.40 0.24 0.13/0.15 1.70 1.29/1.47
|
|
|
|
|
|
0.39 0.46/0.44 0.16 0.12/0.13 1.52 1.03/1.22
|
|
|
|
|
|
NOSER TV TS
|
|
|
NOSER TV TS
|
|
|
|
|
|
Results for high SNR and finer mesh
|
|
|
|
|
|
Volume Ratio
|
|
|
|
|
|
Hausdorff Distance MASD (<28>10-2)
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
0.78 0.91/0.83 0.083 0.033/0.036 0.23 0.05/0.11
|
|
|
|
|
|
0.70 0.87/0.80 0.174 0.086/0.081 0.81 0.30/0.31
|
|
|
|
|
|
0.83 0.92/0.84 0.064 0.030/0.035 0.13 0.04/0.10
|
|
|
|
|
|
Results for low SNR and coarser mesh
|
|
|
|
|
|
Volume Ratio
|
|
|
|
|
|
Hausdorff Distance MASD (<28>10-2)
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
0.51 0.81/0.75 0.154 0.076/0.081 1.14 0.25/0.33
|
|
|
|
|
|
0.46 0.79/0.73 0.216 0.085/0.087 1.85 0.31/0.39
|
|
|
|
|
|
0.69 0.84/0.78 0.100 0.064/0.067 0.46 0.18/0.24
|
|
|
|
|
|
TABLE I EXAMPLE 1: AVERAGE ERROR METRICS FOR TRANSLATING T-SHAPED OBJECT. SRR RESULTS ARE PRESENTED FOR THE CASE CONSIDERING KNOWN MOTION AND USING A REGISTRATION ALGORITHM (SIDE BY SIDE).
|
|
|
|
|
|
TABLE II EXAMPLE 2: AVERAGE ERROR METRICS FOR SYNTHETIC LUNG OBJECT. SRR RESULTS ARE PRESENTED FOR THE MOTION ESTIMATED FROM THE KNOWN HR IMAGES AND FROM THE LR OBSERVATIONS (SIDE BY SIDE).
|
|
|
|
|
|
processed by this operator, each pixel yi(t) of the resulting IHR image lying within the area of the p-th nonuniform LR image element y p (t) gets assigned the value corresponding to the average of all pixels in the HR image xj(t) that falls within the area of that specific element of the nonuniform LR image. This process is illustrated in Figure 2, where matrix HD(t) corresponds to downsampling the HR image to the LR grid and upsampling the resulting LR image back to the IHR grid, which basically amounts to averaging the pixel values in each LR region.
|
|
|
The relationship between the HR images is described in the uniform HR grid, and does not depend on the finite element mesh or reconstruction algorithm employed in the EIT inverse problem. Therefore, it can be described in the same way as in the case of conventional SRR, by using the dynamical model (6). In this case, the image registration (motion estimation) is performed performed based on the IHR images [17], [18].
|
|
|
Since the imaging model proposed for the EIT system given by (8) and (6) is now in the same form as the models encountered for traditional optical systems, any traditional SRR algorithm such as those described in Section II-B can be applied to super-resolve the EIT images, using the proposed observation model and the upsampled EIT images as input.
|
|
|
V. EXPERIMENTAL SETUP AND RESULTS
|
|
|
In this section we present experimental results illustrating the super resolution of EIT images using the imaging model proposed in Section IV for both synthetic and real data. Three examples were devised to illustrate the application of the proposed method under different circumstances. The first example, depicted in Figures 3 and 4, presents the reconstruction of images of a T-shaped object subject to translational displacements. The algorithm was tested both with motion known a priori and with motion estimated by a registration algorithm. The second example, depicted in Figures 5 and 6, extends the first one to EIT images formed by ellipsoids of varying volume, which emulates the imaging of a breathing thorax. The motion estimation was performed twice, using both the synthetic HR images and the observed
|
|
|
|
|
|
IHR images as input, resulting in different levels of motion estimation errors. In both Examples 1 and 2 the LR EIT images were generated considering two different cases: one with small and another with large measurement noise, using a finer and a coarser FE mesh, respectively. This allows for a better assessment of the performance of the SRR methodology. Finally, a third example, depicted in Figures 7 and 8, present the super resolution of EIT images of real in vivo data, considering two different experiments with voltage measurements acquired from healthy adult subjects breathing.
|
|
|
The EIT inverse problem was solved in EIDORS [19], using three different algorithms, namely, the Gauss-Newton solver with the NOSER image prior [20], the Total Variation (TV) regularization [10] and the temporal regularization (TS) [21], all considering difference imaging with the reference voltage set as the average of all measurements in the time window. These algorithms include both a simple solution (NOSER) and sophisticated methodologies (TV and TS), which help to illustrate the performance of the SRR methodology for input images originating from a variety of algorithms, emphasizing its independence to the EIT inverse problem. The FEMs referred to in all examples are standard meshes available in EIDORS [19]. The mesh was also displayed over the reconstructed results in the figures for ease of comparison. The hyperparameters of the algorithms were manually selected in order to achieve a good performance. For the examples with synthetic images, these were generated with a background conductivity of 1 S/m and inclusions of 2 S/m.
|
|
|
We have applied the Least Mean Squares SRR [16] algorithm to super resolve the images in all the examples, with step size <20> = 0.01 and K = 100 iterations for each input image. The motion estimation was performed using the Horn & Schunk Optical Flow algorithm of [22], [18]2 using IHR image pairs as input. In all examples, matrix HD(t) corresponded to the space-variant kernel associated with the nonuniform mesh of each LR EIT image, which was assumed to be timeinvariant. The distortion matrix HB(t) was modeled as a space-
|
|
|
|
|
|
2With parameters pyramid_spacing=2.
|
|
|
|
|
|
lambda=106, pyramid_levels=4,
|
|
|
|
|
|
6
|
|
|
|
|
|
t=10
|
|
|
|
|
|
t=15
|
|
|
|
|
|
t=20
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
Fig. 3. Example 1-a: Translating T-shaped object with high measurement SNR and finer FEM. Column groups indicate time instants (t=10, t=15, t=20). First row: Synthetic HR (desired) images (used in the EIT direct problem). Second, third and fourth row: LR EIT image and super resolved results (side by side) for the NOSER, TV and TS algorithms, respectively. Due to space limitations, only the super-resolved images considering motion estimated from the LR observations is displayed.
|
|
|
|
|
|
and time-invariant 60 <20> 60 Gaussian convolution mask with standard deviation = 20 and Neumann (symmetric) boundary conditions. The nonuniform LR EIT images were upsampled to a 200 <20> 200 IHR uniform grid before processing, where the region outside the nonuniform mesh domain was padded with zeros.
|
|
|
Since the image sequences of Examples 1 and 2 were synthetically generated, the desired HR images were available, allowing a quantitative assessment of the SRR performance. To evaluate the resolution of the LR and reconstructed images with respect to the HR images, we resort to quantitative metrics that evaluate how close the shapes of the objects in these images are to one another, using metrics employed in the evaluation of medical image segmentation algorithms [23], [24], described in detail below.
|
|
|
Denote the HR and the upsampled or super-resolved images as x and x^, respectively. We create a binary map for the shapes by thresholding the upsampled and super-resolved EIT images at 25% of their maximum image value, i.e.
|
|
|
|
|
|
[x^bin]ij = 1 , [x^]ij 0.25 max(x^)
|
|
|
|
|
|
0
|
|
|
|
|
|
otherwise.
|
|
|
|
|
|
This binary image representing the shape of the reconstructed object is then compared with the binary image xbin representing
|
|
|
|
|
|
the shape of the true object in the HR image, using the following metrics [23], [24]:
|
|
|
1) Volume overlap fraction: Measures the relative overlap of the true and estimated volumes, scoring 1 for perfect agreement and 0 for complete disagreement
|
|
|
|
|
|
O(x^bin, xbin) =
|
|
|
|
|
|
x^bin xbin x^bin xbin
|
|
|
|
|
|
.
|
|
|
|
|
|
Note that the larger O(x^bin, xbin) is, the closer the estimated binary image x^bin is to the desired (true) binary image xbin.
|
|
|
2) Hausdorff distance: Measures the largest distance that occurs between two surfaces. Denote by dmin(x, B) = inf{d(x, b), b B} the minimal distance between a point x and a surface B. The Hausdorff distance is the symmetric extension of the largest distance from all points x xbin and the surface x^bin, given by
|
|
|
|
|
|
H(x^bin, xbin) = max{ sup{dmin(x, x^bin), x xbin}, sup{dmin(x, xbin), x x^bin}} .
|
|
|
|
|
|
3) Mean absolute surface distance (MASD): Measures how much the two surfaces differ on average, consisting of a border positioning measure integrated along the entire surface. Denoting the average of the minimal distance between two
|
|
|
|
|
|
7
|
|
|
|
|
|
t=10
|
|
|
|
|
|
t=15
|
|
|
|
|
|
t=20
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
Fig. 4. Example 1-b: Translating T-shaped object with low measurement SNR and coarser FEM. Column groups indicate time instants (t=10, t=15, t=20). First row: Synthetic HR (desired) images (used in the EIT direct problem). Second, third and fourth row: LR EIT image and super resolved results (side by side) for the NOSER, TV and TS algorithms, respectively. Due to space limitations, only the super-resolved images considering motion estimated from the LR observations is displayed.
|
|
|
|
|
|
surfaces
|
|
|
|
|
|
A
|
|
|
|
|
|
and
|
|
|
|
|
|
B
|
|
|
|
|
|
by
|
|
|
|
|
|
d<EFBFBD>min(A, B)
|
|
|
|
|
|
=
|
|
|
|
|
|
1 |A|
|
|
|
|
|
|
aA dmin(a, B), the
|
|
|
|
|
|
MASD is defined as
|
|
|
|
|
|
M ASD(xbin, x^bin)
|
|
|
|
|
|
=
|
|
|
|
|
|
1 2
|
|
|
|
|
|
[
|
|
|
|
|
|
d<EFBFBD>min(xbin,
|
|
|
|
|
|
x^bin)
|
|
|
|
|
|
+
|
|
|
|
|
|
d<EFBFBD>min(x^bin,
|
|
|
|
|
|
xbin)]
|
|
|
|
|
|
.
|
|
|
|
|
|
A. Example 1
|
|
|
In this example, a sequence of 20 images of a T-shaped object subject to random translational motion inside a circular body was generated. The amplitude of the motion was determined by samples of a zero-mean white Gaussian variate with variance 0.3, truncated so that the resulting position was limited to the interval [-0.15, 0.15] to assure that the object lied entirely inside the body.
|
|
|
The synthetic (desired) EIT images were generated using the mesh "d2d1c" with 32 electrodes for the direct problem, and random noise was added to the voltage measurements. The inverse problem was then solved for two situations, one considering a measurement signal to noise ratio (SNR) of 10dB and using finer mesh ("c2c"), displayed in Figure 3, and another considering a SNR of -5dB and using a coarser FEM ("a2d3c"), displayed in Figure 4.
|
|
|
The super resolution reconstruction was performed twice, first considering the motion to be known a priori and afterwards estimating it using a registration algorithm [22], [18]. The synthetic desired HR images, their corresponding EIT observations
|
|
|
|
|
|
(LR images y (t)), and the super resolution results for the case of estimated motion are depicted in Figures 3 and 4. The corresponding quantitative results are shown in Table I.
|
|
|
Although the object is already well identifiable in the LR images in the case of a high measurement SNR, a slight improvement in the representation of the object's shape can be generally noticed in the reconstructed images, specially for the NOSER and TS algorithms. This is also reflected in the quantitative results with a consistent improvements across all evaluated metrics, except for the case of the TV algorithm, where the results considering estimated motion did not provide a good quantitative performance.
|
|
|
The improvements in the reconstructed results are much more noticeable when a lower measurement SNR is employed, in which case the shape of the object in the reconstructed images is much closer to that of the desired one (in the original HR EIT image) when compared to the LR counterparts. This observation is consistently reflected in the quantitative performance metrics.
|
|
|
Note that, in both cases a slight degradation in the performance of the quantitative metrics is observed when a registration algorithm is used, which is a well known fact in SRR literature. Nevertheless, a significant improvement in resolution can still be generally verified in the super resolved images when compared with their LR counterparts, which indicates that the proposed approach is effective in practical scenarios.
|
|
|
|
|
|
8
|
|
|
|
|
|
t=10
|
|
|
|
|
|
t=15
|
|
|
|
|
|
t=20
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
Fig. 5. Example 2-a: Synthetic lung object with high measurement SNR and finer FEM. Column groups indicate time instants (t=10, t=15, t=20). First row: Synthetic HR (desired) images (used in the EIT direct problem). Second, third and fourth row: LR EIT image and super resolved results (side by side) for the NOSER, TV and TS algorithms, respectively. Due to space limitations, only the super-resolved images considering motion estimated from the LR observations is displayed.
|
|
|
|
|
|
B. Example 2
|
|
|
In this example, we performed the reconstruction of a sequence of 20 images of a synthetic lung phantom. A thorax was emulated by a circular body, with ellipses of varying area acting as the lungs, and an additional static circle denoting the spine. Like in Example 1, synthetic (desired) EIT images were generated using the mesh "d2d1c" with 32 electrodes for the direct problem, and random noise was added to the voltage measurements. The inverse problem was again solved for two situations, one considering a measurements SNR of 10dB and using finer mesh ("c2c"), displayed in Figure 5, and another considering a SNR of -5dB and using a coarser FEM ("a2d3c"), displayed in Figure 6.
|
|
|
In order to evaluate the effects of the registration accuracy in the SRR algorithm, we preformed the super resolution reconstruction twice, once estimating the motion from the HR images, and another estimating the motion from the LR observations [22], [18]. The synthetic desired HR images, their corresponding EIT observations (LR images y (t)), and the super resolution results for the case of motion estimated from the LR observations are depicted in Figures 5 and 6. The corresponding quantitative results are shown in Table II.
|
|
|
The performance improvement in the reconstruction results for this example is more significant than in the previous one. Both in the case with higher and lower SNR, the shape of the
|
|
|
|
|
|
objects in the reconstructed images is generally much closer to the desired one in the reconstructed images, especially when the SNR is lower. The quantitative results are consistent and also corroborate with the perceptual results, and a degradation in performance is again noticed when a less accurate registration information is used.
|
|
|
This example also points to the better ability of the temporal solver [21] to operate in low SNR conditions, since it exploits the correlation between the images in adjacent time instants. Nevertheless, a clear improvement is noted for the SRR results in the case of a lower measurement SNR.
|
|
|
C. Example 3
|
|
|
In this example the proposed technique is employed in superresolving sequences of lung EIT images of adult subjects from two different data sets, both available through EIDORS [19]. In both cases, we use the "c2t2" mesh with 16 electrodes in the inverse problem, which represents the shape of the adult human thorax more accurately. The motion between the frames was estimated using a registration algorithm [22], [18]. Since the HR (desired) images were not available, the quantitative performance metrics could not be evaluated for this example. Figures 7 and 8 show the LR observations (EIT image y (t)) and the corresponding reconstruction results for frames selected at the end of expiratory cycles.
|
|
|
|
|
|
9
|
|
|
|
|
|
t=10
|
|
|
|
|
|
t=15
|
|
|
|
|
|
t=20
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
Fig. 6. Example 2-b: Synthetic lung object with low measurement SNR and coarser FEM. Column groups indicate time instants (t=10, t=15, t=20). First row: Synthetic HR (desired) images (used in the EIT direct problem). Second, third and fourth row: LR EIT image and super resolved results (side by side) for the NOSER, TV and TS algorithms, respectively. Due to space limitations, only the super-resolved images considering motion estimated from the LR observations is displayed.
|
|
|
|
|
|
For the results of the first data set depicted in Figure 7, it can be noted that the lungs are already well represented in the LR EIT images. Nevertheless, the boundary of the lungs appears to be slightly clearer in the reconstructed images. Besides, the size of the lungs seems more consistent across the super resolved images generated using the different EIT algorithms, as opposed to their LR counterparts.
|
|
|
For the results of the second data set depicted in Figure 8, more pronounced improvements are obtained in the super resolved images. The lungs (objects of interest) are more well defined and more easily identifiable from the image background, with a reduced amount of artifacts when compared to the observed LR EIT images.
|
|
|
|
|
|
that a significant increase in the visual quality and resolution of EIT images obtained using different finite element meshes and EIT algorithms can be achieved with the proposed method. This indicates the possibility of mitigating the severe deficiency in image quality that currently prevents EIT imaging systems to be more widely used. The visual quality of the resulting image was significantly improved, and quantitative performance metrics show that the shapes of the objects in the reconstructed images are substantially closer to those in the HR images when compared to the LR observations, indicating that super-resolution effectively has been obtained. Significant improvements were verified for both idealized and practical scenarios.
|
|
|
|
|
|
VI. CONCLUSION
|
|
|
In this paper we have proposed a new method for superresolution reconstruction of EIT images. The proposed method uses a new model for the EIT image formation system, which is based on the Penrose Pixels approach developed for optical image acquisition systems. Although the Penrose Pixels Super-Resolution methodology still finds no practical application due to the unavailability of appropriate sensors, we have shown that its essence can be adapted to handle the non-conventional imaging sensors typical of EIT images. Simulation results employing the LMS-SRR algorithm show
|
|
|
|
|
|
REFERENCES
|
|
|
[1] Patrick Helm, Robert Teichman, Steven Hartmann, and David Simon. Spinal navigation and imaging: History, trends, and future. Medical Imaging, IEEE Transactions on, 34(8):1738<33>1746, 2015.
|
|
|
[2] David S Holder. Electrical impedance tomography: methods, history and applications. CRC Press, 2004.
|
|
|
[3] John Kennedy, Ora Israel, Alex Frenkel, Rachel Bar-Shalom, Haim Azhari, et al. Super-resolution in PET imaging. Medical Imaging, IEEE Transactions on, 25(2):137<33>147, 2006.
|
|
|
[4] Moshe Ben-Ezra, Zhouchen Lin, Bennett Wilburn, and Wei Zhang. Penrose pixels for super-resolution. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 33(7):1370<37>1383, 2011.
|
|
|
[5] R. Penrose. The role of aesthetics in pure and applied mathematical research. Bull. Inst. of Math. and its Appl., 10:266<36>271, 1974.
|
|
|
|
|
|
10
|
|
|
[19] Andy Adler and William RB Lionheart. Uses and abuses of EIDORS: an extensible software base for EIT. Physiol. Meas., 27(5):S25<32>S42, 2006.
|
|
|
[20] Margaret Cheney, David Isaacson, Jonathan C Newell, S Simske, and J Goble. NOSER: An algorithm for solving the inverse conductivity problem. Int. J. Imag. Syst. Tech., 2(2):66<36>75, 1990.
|
|
|
[21] Andy Adler, Tao Dai, and William RB Lionheart. Temporal image reconstruction in electrical impedance tomography. Physiological measurement, 28(7):S1, 2007.
|
|
|
[22] Berthold KP Horn and Brian G Schunck. Determining optical flow. Artificial intelligence, 17(1-3):185<38>203, 1981.
|
|
|
[23] Ingrid Sluimer, Mathias Prokop, and Bram Van Ginneken. Toward automated segmentation of the pathological lung in ct. IEEE transactions on medical imaging, 24(8):1025<32>1038, 2005.
|
|
|
[24] Guido Gerig, Matthieu Jomier, and Miranda Chakos. Valmet: A new validation tool for assessing and improving 3d object segmentation. In Medical Image Computing and Computer-Assisted Intervention<6F>MICCAI 2001, pages 516<31>523. Springer, 2001.
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
Fig. 7. Example 3-a: Real lung image for adult shallow breathing (data available through EIDORS), for the 19-th frame (end of an inspiratory cycle). First, second and third rows: LR EIT image and super resolved results (side by side) for the NOSER, TV and TS algorithms, respectively.
|
|
|
|
|
|
[6] Kuo-Sheng Cheng, David Isaacson, JC Newell, and David G Gisser. Electrode models for electric current computed tomography. IEEE Transactions on Biomedical Engineering, 36(9):918<31>924, 1989.
|
|
|
[7] Margaret Cheney, David Isaacson, and Jonathan C Newell. Electrical impedance tomography. SIAM review, 41(1):85<38>101, 1999.
|
|
|
[8] Andy Adler and Robert Guardo. A neural network image reconstruction technique for electrical impedance tomography. IEEE Transactions on Medical Imaging, 13(4):594<39>600, 1994.
|
|
|
[9] David Isaacson, Jennifer L Mueller, Jonathan C Newell, and Samuli Siltanen. Reconstructions of chest phantoms by the d-bar method for electrical impedance tomography. IEEE Transactions on Medical Imaging, 23(7):821<32>828, 2004.
|
|
|
[10] Andrea Borsic, Brad M Graham, Andy Adler, and William RB Lionheart. In vivo impedance imaging with total variation regularization. IEEE transactions on medical imaging, 29(1):44<34>54, 2010.
|
|
|
[11] Eric T Chung, Tony F Chan, and Xue-Cheng Tai. Electrical impedance tomography using level set representation and total variational regularization. Journal of Computational Physics, 205(1):357<35>372, 2005.
|
|
|
[12] Andrea Borsic and A Adler. A primal<61>dual interior-point framework for using the L1 or L2 norm on the data and regularization terms of inverse problems. Inverse Problems, 28(9):095011, 2012.
|
|
|
[13] Bangti Jin, Taufiquar Khan, and Peter Maass. A reconstruction algorithm for electrical impedance tomography based on sparsity regularization. International Journal for Numerical Methods in Engineering, 89(3):337<33> 353, 2012.
|
|
|
[14] Kamal Nasrollahi and Thomas B. Moeslund. Super-resolution: A comprehensive survey. Machine Vision & Applications, 25(6):1423<32>1474, 2014.
|
|
|
[15] Sung Cheol Park, Min Kyu Park, and Moon Gi Kang. Super-resolution image reconstruction: a technical overview. Signal Processing Magazine, IEEE, 20(3):21<32>36, 2003.
|
|
|
[16] Michael Elad and Arie Feuer. Superresolution restoration of an image sequence: adaptive filtering approach. Image Processing, IEEE Transactions on, 8(3):387<38>395, 1999.
|
|
|
[17] Lisa Gottesfeld Brown. A survey of image registration techniques. ACM Comput. Surv., 24(4):325<32>376, December 1992.
|
|
|
[18] Deqing Sun, Stefan Roth, and Michael J Black. Secrets of optical flow estimation and their principles. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conf. on, pages 2432<33>2439. IEEE, 2010.
|
|
|
|
|
|
11
|
|
|
|
|
|
t=1s
|
|
|
|
|
|
t=5s
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
LR
|
|
|
|
|
|
SRR
|
|
|
|
|
|
Fig. 8. Example 3-b: Real lung images for tidal breathing (data available through EIDORS), for the end of two inspiratory cycles (t=1 s and t=5 s). First, second and third rows: EIT LR image and super-resolved results (side by side) for the NOSER, TV and TS algorithms, respectively.
|
|
|
|
|
|
|