Spatial frequency domain imaging technology based on Fourier single-pixel imaging

Huiming Ren; Guoqing Deng; Peng Zhou; Xu Kang; Yang Zhang; Jingshu Ni; Yuanzhi Zhang; Yikun Wang

doi:10.1117/1.JBO.27.1.016002

24 January 2022 Spatial frequency domain imaging technology based on Fourier single-pixel imaging

Huiming Ren, Guoqing Deng, Peng Zhou, Xu Kang, Yang Zhang, Jingshu Ni, Yuanzhi Zhang, Yikun Wang

Author Affiliations +

Journal of Biomedical Optics, Vol. 27, Issue 1, 016002 (January 2022). https://doi.org/10.1117/1.JBO.27.1.016002

Abstract

Significance: Optical properties (absorption coefficient and scattering coefficient) of tissue are the most critical parameters for disease diagnosis-based optical method. In recent years, researchers proposed spatial frequency domain imaging (SFDI) to quantitatively map tissue optical properties in a broad field of contactless imaging. To solve the limitations in wavebands unsuitable for silicon-based sensor technology, a compressed sensing (CS) algorithm is used to reproduce the original signal by a single-pixel detectors. Currently, the existing single-pixel SFDI method mainly uses a random sampling policy to extract and recover signals in the acquisition stage. However, these methods are memory-hungry and time-consuming, and they cannot generate discernible results under low sampling rate. Explorations on high performance and efficiency single-pixel SFDI are of great significance for clinical application.

Aim: Fourier single-pixel imaging can reconstruct signals with less time and space costs and has fewer reconstruction errors. We focus on an SFDI algorithm based on Fourier single-pixel imaging and propose our Fourier single-pixel image-based spatial frequency domain imaging method (FSI-SFDI).

Approach: First, we use Fourier single-pixel imaging algorithm to collect and compress signals and SFDI algorithm to generate optical parameters. Given the basis that the main energy of general image signals is concentrated in the range of low frequency of Fourier frequency domain, our FSI-SFDI uses a circular-sampling scheme to sample data points in the low-frequency region. Then, we reconstruct the image details from these points by optimization-based inverse-FFT method.

Results: Our algorithm is tested on simulated data. Results show that the root mean square error (RMSE) of optical parameters is lower than 5% when the data reduction is 92%, and it can generate discernible optical parameter image with low sampling rate. We can observe that our FSI-SFDI primarily recovers the optical properties while keeping the RMSE under the upper bound of 4.5% when we use an image with 512 × 512 resolution as the example for calculation and analysis. Not only that but also our algorithm consumes less space and time for an image with 256 × 256 resolution, the signal reconstruction takes only 1.65 ms, and requires less RAM memory. Compared to CS-SFDI method, our FSI-SFDI can reduce the required number of measurements through optimizing algorithm.

Conclusions: Moreover, FSI-SFDI is capable of recovering high-quality resolvable images with lower sampling rate, higher-resolution images with less memory and time consumed than previous CS-SFDI method, which is very promising for clinical data collection and medical analysis.

1. Introduction

Recent advances in optical imaging show promising applications in biomedical areas due to the potential correlation between the optical properties of biological tissue and biochemical composition.¹^–³ Among these technologies, spatial frequency domain imaging (SFDI) has warranted closer attention in that SFDI is effective to derive tissue optical parameters in a noncontact manner.⁴^,⁵ The key of SFDI is acquiring the absorption and scattering coefficients of the tissues by computing on the pattern image that is captured by the hyperspectral imaging cameras on spatially modulated light.⁶

However, the significant issues in SFDI research are as follows: The existing SFDI methods use a camera for data collection, which relies on electronics integration (silicon) and is limited by CCD and CMOS digital technology. In wavebands unsuitable for silicon-based sensor technology, imaging is considerably more complicated, such as the infrared or deep ultraviolet. Cameras with the required resolution at wavelengths where silicon is blind are more expensive. Recently, several attempts have been tried to fix the issue (1) by introducing an SFDI system that is based on compressed sensing theory (CS), named CS-SFDI.⁷ CS method uses a single-pixel detector to collect the images and then reconstructs the images with fewer sampling points. The most frequently used method of CS is to reconstruct the image with optimization-based algorithms.⁸^–¹⁰ CS-SFDI replaces the camera with a single-pixel photo detector and collects the measurement matrix of human tissue to reconstruct and demodulate images.

Theoretically, CS-SFDI method passively drops some useful information thus result in high errors in reconstructing images. To improve CS-SFDI method, CS-based parameter recovery algorithm¹¹ is proposed to compress the demodulated images before image reconstruction. Unfortunately, in some cases of clinical data with high heterogeneity, CS-based algorithm often produces results that are not distinguishable under low sampling rate.

However, compared with traditional optical imaging technology, single-pixel imaging still has the following two disadvantages: on the one hand, the image quality is far from the level of the current traditional optical imaging system, the image resolution, and the signal-to-noise ratio is low. On the other hand, compression sensing algorithm often requires a lot of computing time and is difficult to reconstruct the details of complex objects. Consequently, most studies in the field of CS-SFDI only focus on collecting small-scaled images.

In this work, we examine the emerging role of single-pixel imaging based on Fourier spectrum in the context of the SFDI image acquisition (FSI-SFDI). First, we replace the binary basis patterns with the grayscale harmonic sinusoid patterns for the single-pixel imaging system, which is proved in our experiments to improve the estimation accuracy of optical coefficients. Second, the proposed method takes full advantage of the sparseness of natural signals in the Fourier domain and recovers higher resolution images with fewer samples. Given the prior knowledge that the main information of an image is concentrated in the low frequency part of the Fourier spectrum,¹²^,¹³ our method can reconstruct the large-size image by sampling less data point of low-frequency information. To acquire the pattern images, we superimpose Fourier base pattern images on spatial frequency domain images to simulate structured illumination and compressive detection on tissue surfaces: one tissue with sinusoidal light patterns illuminations at two frequencies and three different phases⁷ and another tissue with four-step phase-shifting sinusoid patterns detection.¹² After obtaining the pattern images of tissue using IFFT method, we use approximate lookup table (LUT) algorithm based on Monte Carlo simulation¹⁴ to map and demodulate optical properties.

We conduct experiments to evaluate our method. Using the APP-SFDI data set for simulation,¹⁵^,¹⁶ the clearer and more distinguishable images of the optical properties can be generated, and the root mean square error (RMSE) of the optical properties is $< 5 %$ and SSIM of the optical properties is $> 0.7$ for a 92% reduction in measurements. Beyond that, keeping the same number of samples, high quality and large-size images can be restored with higher time efficiency and the results of our method are more accurate than that of CS-SFDI. It means that FSI-SFDI can use less sampling rate in large-size images to obtain optical properties images with higher definition.

2. Approach

2.1.

Spatial Frequency Domain Imaging

Many researchers utilized SFDI to measure optical coefficients, such as absorption ( $μ_{a}$ ) and scattering ( $μ_{s}^{'}$ ) properties.⁴ The first step in this process is using illumination of tissue by structured light to generate modulated images (MI).¹⁷^,¹⁸ MI offers an effective way to acquire the spatial modulation transfer function of a turbid medium, which can represent the characteristics of the optical system. After the measurement of optical, we can capture and figure out two spatial frequency images, for example, a DC image (e.g., $0 {mm}^{- 1}$ ) and an AC image (e.g., $0.2 {mm}^{- 1}$ ), to determine the diffuse reflectance of the sample. Then, SFDI uses diffusion theory or LUT based on Monte Carlo simulations¹⁴ to extract the optical coefficients from diffuse reflectance images typically.

There are two main types of components used to collect spatial frequency images. One DMD projects 0 and $0.2 {mm}^{- 1}$ frequency, three-phase spatial structured light to the surface of tissue, and then, one camera collects MIs of the tissue. The AC amplitude $M_{ac} (x, f_{x})$ and the DC amplitude $M_{dc} (x, f_{x})$ is solved pixel-by-pixel:

Eq. (1)

M_{ac} (x, f_{x}) = \frac{\sqrt{2}}{3} {[{(I_{ac 1} - I_{ac 2})}^{2} + {(I_{ac 2} - I_{ac 3})}^{2} + {(I_{ac 3} - I_{ac 1})}^{2}]}^{1 / 2},

Eq. (2)

M_{dc} (x, f_{x}) = \frac{1}{3} (I_{dc 1} + I_{dc 2} + I_{dc 3}),

where

I_{ac 1}

,

I_{ac 2}

,

I_{ac 3}

are the backscattered light intensity corresponding to the phases

0 π, 2 / 3 π, 4 / 3 π

, with

0.2 {mm}^{- 1}

frequency and

I_{dc 1}

,

I_{dc 2}

,

I_{dc 3}

are the backscattered light intensity corresponding with

0 {mm}^{- 1}

frequency, respectively.

The diffuse reflectance of the sample $R_{d} (x, f_{x})$ can be calculated by measuring a standard reflector with existing diffuse reflectance as a reference:

Eq. (3)

R_{d_ac} (x, f_{x}) = \frac{M_{ac} (x, f_{x})}{M_{ac, ref} (x, f_{x})} \cdot R_{d_ac, ref} (f_{x}),

Eq. (4)

R_{d_dc} (x, f_{x}) = \frac{M_{dc} (x, f_{x})}{M_{dc, ref} (x, f_{x})} \cdot R_{d_dc, ref} (f_{x}),

where

M_{ac, ref} (x, f_{x})

and

M_{dc, ref} (x, f_{x})

are the AC and DC amplitude of a standard reflector with known diffuse reflectance.

Finally, we employ the precomputed LUT to map the relationship between $R_{d} (x, f_{x})$ and optical properties ( $μ_{s}^{'}$ and $μ_{a}$ ) by Monte Carlo algorithm.

2.2.

Fourier-Based Single-Pixel Imaging

Many prior works have been devoted to single-pixel imaging that often captures a scene without a direct line of sight light.¹²^,¹³ To increase the imaging quality, Fourier-based method for single-pixel imaging is often adopted. Due to the sparsity of the image in the Fourier domain, our proposed method adopts two-dimensional (2D) Fourier single-pixel imaging, using simulated patterns to capture the Fourier spectrum of the scene signal. Therefore, images can be reconstructed through Fourier inverse transform from Fourier spectrum. 2D Fourier transform¹⁹ and the inverse transform are given as

Eq. (5)

C (f_{x}, f_{y}) = \int_{- \infty}^{+ \infty} \int_{- \infty}^{+ \infty} I (x, y) \exp [- j \cdot 2 π (f_{x} x + f_{y} y)] d f_{x} d f_{y},

Eq. (6)

I (x, y) = \int_{- \infty}^{+ \infty} \int_{- \infty}^{+ \infty} C (f_{x}, f_{y}) \exp [j \cdot 2 π (f_{x} x + f_{y} y)] d x d y,

where

I (x, y)

represents 2D image,

C (f_{x}, f_{y})

is the Fourier spectrum of 2D image,

x, y

is the Cartesian coordinates in the spatial image domain,

f_{x}

,

f_{y}

is the Cartesian coordinates in the Fourier domain, and

j

is the imaginary unit.

Any 2D image is the result of the linear superposition of a series of Fourier base patterns. The weight corresponding to each Fourier base pattern is the Fourier coefficient.¹² To acquire the Fourier base patterns for 2D inverse Fourier transform (IFT), we use the existing dataset¹⁵ from APP-SFDI for simulation, superimpose the generated Fourier base pattern on the images, and obtain the final signals of single pixel detector through simulation computation. After the prior study,²⁰ the signal detected by a single pixel detector in a simulated environment is the sum of all the pixel intensities. First, to obtain the Fourier coefficients of the object image, a computer is used to generate a Fourier base pattern. The Fourier base pattern is a series of cosine distribution patterns with different spatial frequencies and different initial phases. The Fourier base pattern projected on the tissue surface can be formulated as

Eq. (7)

P (x, y; f_{x}, f_{y}, Ø) = a + b \cdot \cos (2 π f_{x} x / M + 2 π f_{y} y / N + Ø),

where

a

is the average light intensity,

b

is the contrast,

x

and

y

are the rectangular coordinates of the plane in which the target object is located,

M

and

N

are the dimensions of the image,

f_{x}

and

f_{y}

are the spatial frequencies corresponding to the

x

and

y

directions, and

Ø

is the initial phase.

Therefore, the scattered reflected light $D_{Ø} (f_{x}, f_{y})$ that collected from target object $O (x, y)$ can be expressed as follows:

Eq. (8)

D_{Ø} (f_{x}, f_{y}) = \sum_{x, y} P (x, y; f_{x}, f_{y}, Ø) \cdot O (x, y) .

To obtain the Fourier spectrum information of a single point in the image, the most commonly used method is the four-step phase-shifting method. Four-step Fourier spectrum acquisition methods are used frequently. 0, $1 / 2 π$ , $π$ , and $3 / 2 π$ are assigned to the values of $Ø$ , respectively. Based on the four-step phase-shift algorithm, the Fourier coefficients $C (f_{x}, f_{y})$ can be estimated from four values obtained as follows:

Eq. (9)

C (f_{x}, f_{y}) = (D_{0} (f_{x}, f_{y}) - D_{π} (f_{x}, f_{y})) + (D_{\frac{π}{2}} (f_{x}, f_{y}) - D_{\frac{3 π}{2}} (f_{x}, f_{y})) .

Traditionally, as the four-step phase-shift algorithm is adopted, a complex value in a particular spectrum can be assessed by measuring four projection samples. This means that an $M \times N$ pixel image needs to be captured $4 \times M \times N$ times. Because the Fourier spectrum of the image is conjugate symmetric, an $M \times N pixel$ image only needs to measure $2 \times M \times N$ projection samples. Previous studies¹³^,²¹^,²² have confirmed that the main information about objects is concentrated in the lower frequencies of the Fourier spectrum. Therefore, circular acquisition of low-frequencies is the common procedures for reducing sampling rate. However, discarding the spectrum of high frequencies introduces the reduction of the details of the image. To combat this, Wenwen et al. study sparse Fourier single-pixel imaging,²⁰ based on variable-density sparse sampling patterns. The probability of sampled distribution is formulated as follows:

Eq. (10)

ρ (r) = {\begin{matrix} 1 & r \leq R \\ {(1 - r)}^{p} & r > R \end{matrix},

where

p

is the polynomial coefficient,

R

is predefined radius of a circular sample in the low frequency, and

r

represents the Fourier frequency. In the experiment, we measure

r

by the Euclidean distance from the center point to other points in the image and is normalized to (0,1). Figure 1 shows the forms of three sampling matrices at a sampling rate of 10%.

Fig. 1

Three sample schemes: (a) circular sampling scheme, (b) random sampling scheme, and (c) variable-density random sampling scheme.

Sparse Fourier single-pixel imaging can reconstruct high-resolution images using CS optimization algorithms. The optimization algorithms solution of the following equation:

Eq. (11)

\arg \min {‖ F t - T^{'} ‖}_{2}^{2} + λ_{1} {‖ t ‖}_{1} + λ_{2} TV (t),

where TV is the total variation operator,

λ_{1}

,

λ_{1}

are the loss weight, and

t

is the object image to be reconstructed. The matrix

T^{'}

is the acquired undersampled spectral data.

2.3.

Fourier Single-Pixel Imaging-Based SFDI

Previous CS-SFDI methods estimate optical properties in a variety of ways,⁷^,¹¹ which is based on traditional CS algorithm. However, their method generates optical parameters with limited accuracy and fails to produce resolvable images at low sampling rates. To solve this problem, the FSI-SFDI algorithm is proposed in this paper. FSI-SFDI performs SFDI using Fourier single-pixel imaging. To acquire the data, the Fourier base patterns are superimposed on dataset image, which consists of hand images at 0 and $0.2 {mm}^{- 1}$ frequency, three-phase spatial structured light. The following equation shows the modulated patterns:

Eq. (12)

F (x, y; f_{x}, f_{y}; f_{k}, Ø_{1}, Ø_{2}) = \cos (2 π f_{k} + Ø_{1}) \cdot P (x, y; f_{x}, f_{y}, Ø_{2}),

where

f_{k}

is the frequency of SFDI,

Ø_{1}

is the phase of SFDI, and

P (x, y; f_{x}, f_{y}, Ø_{2})

is the Fourier base pattern. The scattered reflected light

C_{Ø 1, Ø 2} (f_{x}, f_{y}, f_{k})

is collected from target object

O (x, y)

, and can be expressed as follows:

Eq. (13)

D_{Ø 1, Ø 2} (f_{x}, f_{y}, f_{k}) = \sum_{x, y} F (x, y; f_{x}, f_{y}; f_{k}, Ø_{1}, Ø_{2}) \cdot O (x, y) .

According to Eq. (9), we can compute the Fourier spectrum of the image at a certain spatial frequency and phase. With the Fourier spectrum of the tissues, we utilize IFT to reconstruct the MIs of SFDI. Figure 2 shows an example of image simulation and reconstruction in FSI-SFDI. Because of the large number of patterns required for Fourier single-pixel imaging, it will cost a lot of time and space. To speed up imaging and computing process, a proper solution is to compress the sampling of tissue information. We can take full advantage of the fact that more of the image energy is concentrated in the low frequency of the Fourier spectrum. So we can use circular low frequency sampling, variable density random undersampling, and so on. In the experimental section, we present the efficiency comparison of these sampling methods.

Fig. 2

Illustration of image acquisition and reconstruction. The simulated structured pattern images are superimposed on the dataset of SFDI. Single pixel imaging signal is obtained through calculation, which is used to simulate the signal collected by the single pixel detector. After collecting the signal, an MI at a certain spatial frequency and phase can be obtained from the signal by IFT method.

As shown in Fig. 3, to measure the optical characteristic, we reconstruct the two spatial frequencies and three-phases MIs and calculate AC and DC by Eqs. (1) and (2). Finally, the optical properties: absorption ( $μ_{a}$ ) and scattering ( $μ_{s}^{'}$ ) properties, can be calculated according to the LUT algorithm or diffusion theory.

Fig. 3

Pipeline of SFDI algorithm. After FSI algorithm which is used to reconstruct multiple modulation images with different phases and frequencies, SFDI can be used to recover optical coefficients.

2.4.

Data Simulation

To better verify the effectiveness of the algorithm, we first use the data set to simulate the data on the computer. The data set consists of multiple frequency and phase dorsal patterns. Among the current open resources, APP-SFDI has provided SFDI software and images that are convenient for analysis and testing.¹⁵ The CS parameter reconstruction algorithm has used this data for experiments and analysis. Therefore, testing on this dataset is reasonable for comparing the performance of the algorithms.

2.5.

Error Metrics

To verify the error of the results, we choose RMSE for evaluation metrics of our method. Calculation equation of RMSE is obtained as

Eq. (14)

RMSE = \sqrt{\frac{\sum {(\bar{A} - \bar{B})}^{2}}{N}} * 100,

where

\bar{A}

and

\bar{B}

are the pixel values of the estimated result and the original image and

N

is the number of the images.

In addition, to evaluate the accuracy of our results, we adopt structural similarity index to measure the similarity of the two images. Calculation equation of SSIM is

Eq. (15)

SSIM = \frac{(2 \times \bar{A} \times \bar{B} + C_{1}) (2 \times σ_{A B} + C_{2})}{({\bar{A}}^{2} + {\bar{B}}^{2} + C_{1}) (σ_{A}^{2} + σ_{B}^{2} + C_{2})},

where

σ_{A}

and

σ_{B}

are the standard deviation of the estimated result and the original image.

σ_{A B}

is the covariance of the estimated result and the original image.

C_{1}

and

C_{2}

are the constant terms, and calculation equation of

C_{1}

and

C_{2}

is

Eq. (16)

{\begin{matrix} C_{1} = {(K_{1} \times L)}^{2} \\ C_{2} = {(K_{2} \times L)}^{2} \end{matrix},

where

K_{1}

and

K_{2}

are the adjustable constant terms. In general,

K_{1}

is equal to 0.01 and

K_{2}

is equal to 0.03.

L

is the data range of the input gray image, where

L

is 255 for uint8 encoding in our experiments.

3. Results

The simulation data use the data set in the APP-SDFI source file, and the source file includes multiple phase and frequency hand modulated illumination images. To compare with the existing methods, this paper uses three phases and two frequencies of dataset as simulations, which are $0 π$ , $1 / 3 π$ , $2 / 3 π$ , and 0 and $2 {mm}^{- 1}$ , respectively. And the images are resized to $256 \times 256 pixels$ .

3.1.

Comparison of Different Sampling Schemes

Since different sampling methods introduce different impacts on the reconstructed MI. To compare the differences between sample schemes, we use different sampling methods at 10% sampling rate to reconstruct the MIs and then generate the optical characteristic map. Three types of sampling schemes and two reconstruction methods have been used to restore the object information. The sampling rate is the ratio of the number of acquisition patterns to the number of image pixels for the convenience of comparison. The results of FSI-SFDI are shown in Fig. 4. Although the sparse Fourier single-pixel imaging has been proved to be effective in recovering the detail information of objects, it is not efficient enough for the situation where the image of skin tissue is flatter. Moreover, the reconstructed images by sparse Fourier single-pixel imaging drop some details of the modulated light, while using circular sampling is more effective to recover better details that are close to the original image.

Fig. 4

Restoration results of different frequencies and the optical characteristic at a sampling rate of 10%. As presented in this figure, uniform density random undersampling fails to reconstruct resolvable images at low sampling rate. Using variable density random undersampling is better than uniform density random undersampling but cannot significantly improve the accuracy of clinical data with high heterogeneity. On the contrary, it increases the error of the MI when $f_{k}$ is equal to $2 {mm}^{- 1}$ . Circular sampling scheme helps to reconstruct better details with less visible errors than other sampling methods when compared with full sampling.

3.2.

High Efficient Reconstruction Performance under Different Sampling Rates

It is very important to obtain more accurate reconstruction results with fewer details losses for exact optical parameter estimation. We have comprehensively compared the performance of our method with that of the CS-SFDI method at different pattern number (i.e., 5242 to 26,214), corresponding to different sampling rates (i.e., 8% to 40%). Figure 5 shows the optical characteristic diagram with pattern number of 5242 to 26,214 by FSI-SFDI and CS-SFDI algorithms. Obviously, at low sampling rate, the image reconstructed by FSI-SFDI method has clear edges and less information loss. At high sampling rate, this method is slightly better than the competitive method. The most interesting aspect of this graph is at 5242 patterns. FSI-SFDI algorithm can clearly preserve the outline of the hand, while the result of CS-SFDI which does not meet the needs of clinical scenarios cannot be distinguished. With lower sampling rate, the FSI-SFDI method is also capable of recovering good details that are clearer than CS-SFDI method.

Fig. 5

The SFDI based on Fourier single-pixel imaging recovery algorithm image panel by 5242 to 26,214 patterns. Pictures of hand are compressed into $256 \times 256 pixels$ .

According to Fig. 5, the lower sampling rate results in higher reconstruction errors of the MIs, leading to larger estimation error of the optical properties. To evaluate the performance of our method, we measure and compare the RMSE between the estimated optical properties and the original with CS-SFDI method. As shown in Fig. 6, our method achieves much lower errors than CS-SFDI in terms of $μ_{a}$ and $μ_{s}^{'}$ , especially when we use a low sampling rate. The RMSE of FSI-SFDI is $< 3 %$ ( $μ_{a}$ ) and $< 5 %$ ( $μ_{s}^{'}$ ) and the SSIM of FSI-SFDI is $> 0.9$ ( $μ_{a}$ ) and $> 0.8$ ( $μ_{s}^{'}$ ), respectively.

Fig. 6

Results comparison of optical properties estimation. (a), (c) The RMSE and SSIM metrics results of $μ_{a}$ with pattern numbers range from 5242 to 26,214. (b), (d) The RMSE and SSIM metrics results of $μ_{s}^{'}$ with pattern numbers range from 5242 to 26,214. Pictures of hand are resized into $256 \times 256 pixels$ . Comparison between the original data and reconstructed images for increasing pattern numbers.

Fig. 7

The SFDI based on Fourier single-pixel imaging recovery algorithm image panel with pattern numbers range from 5242 to 26,214. Pictures of hand are compressed into $512 \times 512 pixels$ .

To evaluate the performance of FSI-SFDI on large-size images, we use an image with $512 \times 512$ resolution as the example for calculation and analysis. As the number of modes increases, the computational cost increases. We use the same pattern number 5242 to 26,214 for testing. In qualitative evaluation, as shown in Fig. 7, FSI-SFDI recovers favorable optical characteristic map with few details missed even at 5242 patterns when compared with the results of 100% sampling rate. Quantitatively, we measure the RMSE of the estimated optical properties $μ_{a}$ and $μ_{s}^{'}$ under different sampling rate. From the curve of Fig. 8, we can observe that our FSI-SFDI primely recovers the optical properties while keeping the RMSE under the upper bound of 4.5%, and the $SSIM > 0.65$ .

Fig. 8

(a) FSI-SFDI RMSE results of data set compressed into $512 \times 512 pixels$ . (b) FSI-SFDI SSIM results of data set compressed into $512 \times 512 pixels$ . RMSE for each optical property map obtained using the FSI-SFDI algorithm, compared with the noncompression-based ground truth results.

Fig. 9

Result of homogeneous two-tone tissue-mimicking phantoms. Comparison between the original data and reconstructed images for increasing pattern numbers.

To investigate the ability of FSI-SFDI algorithm in estimating optical properties, we utilize homogeneous two-tone tissue-mimicking phantoms. The experiment settings and data of phantom measurement are kept same as Mellors et al.¹¹ Specifically, phantom measurements can be simulated using 0 and $0.2 {mm}^{- 1}$ spatial frequencies images to generate the MI. These simulated data sets have $256 \times 256$ resolution, and there are three different optical property anomalies. Figure 9 shows the final anomaly varying and result of the phantom measurements. From the data in Fig. 10, it is apparent that when the data are reduced by 92%, RMSE is lower than 2.5%, and SSIM is greater than 0.9. As can be seen from the histogram in Fig. 11, under the low sampling rate, the results of FSI-SFDI still have relatively high similarity.

Fig. 10

FSI-SFDI RMSE and SSIM results of homogeneous two-tone tissue-mimicking phantoms. (a) RMSE for each optical property map obtained using the FSI-SFDI algorithm, and (b) SSIM for each optical property map obtained using the FSI-SFDI algorithm, compared with the noncompression-based ground truth results.

Fig. 11

Histogram results of homogeneous two-tone tissue-mimicking phantoms. (a)–(c) Histogram results of $μ_{a}$ with pattern numbers range from 5242 to 26,214. (c)–(e) Histogram results of $μ_{s}^{'}$ with pattern numbers range from 5242 to 26,214.

3.3.

Memory and Time Efficiency with Large Resolutions

To compare the efficiency of different reconstruction algorithms, we also evaluate the time efficiency and memory cost of our method when restoring the image signal with 2% sampling rate at $128 \times 128$ resolution and $256 \times 256$ resolution. The experiment is conducted on a computer server with an Intel Xeon(R) Gold-6124M 2.60 GHz CPU and 128 GB RAM. Table 1 presents the comparison results of memory required and the computation time between two image reconstruction methods. For an image with $256 \times 256$ resolution, the CS method takes more than 67,100 ms, while the IFFT method takes only 1.65 ms. For space cost, the CS method requires more than 7000 MB, and IFFT requires 175 MB to reconstruct the image signal.

Table 1

Comparison of speed and memory cost for the inversion from Fourier spectrum to images.

Data points	Reconstruction method	Memory (MB)	Time (ms)
$128 \times 128$	CS	3709.22	17,655
$128 \times 128$	IFFT	170.43	1.43
$256 \times 256$	CS	7090.22	67,100
$256 \times 256$	IFFT	175.59	1.65

In Fig. 12, we present the memory cost required by the two methods during the computation. Analysis of the figure shows that CS-SFDI method requires the similar amount of computer memory as our FSI-SFDI method when the image resolution is no more than $64 \times 64$ . When dealing with higher image resolution, CS-SFDI method requires more RAM memory (more than 10 times memory cost of FSI-SFDI) to perform optimization task. In other words, our FSI-SFDI method is technically superior to CS-SFDI in memory cost when applied in cheaper devices, e.g., laptop computer, cell phone, etc.

Fig. 12

Comparison of memory for CS-SFDI and FSI-SFDI from different data points.

4. Discussion

This work presents a method that uses Fourier single pixel imaging to reconstruct the pattern image based on SFDI theory. We superimpose the two-frequency, three-phase MI onto the image of the spatial frequency domain to simulate the detection of Fourier single pixel imaging on the tissue. Then we reconstruct the optical characteristic map by based method, which effectively improves the accuracy and reduces the time and memory cost compared with previous CS methods.

However, using two-frequencies and three-phase modulated light to acquire six images is still time-consuming. Existing single snapshot algorithms²³^–²⁵ proposed for reducing the time of acquisition, e.g., one-frequency and one-phase, but downgrade the result accuracies heavily. In the future, it will be possible to consider employing machine learning methods and single snapshot algorithm to reconstruct the optical characteristic map with less patterns.²⁶^–²⁸ Moreover, in practical terms, DMD projects grayscale images is slower than binary patterns, further increasing the time cost for collecting information. To alleviate this burden, a programmable DMD operating in binary modes can be used to generate binary (stripe) patterns and then convert the patterns to grayscale (fringe) patterns by employing the defocusing techniques or the spatial low-pass filtering, and the research work of Zhang et al. has suggested the feasibility of path.

Another weakness of our methods is that the high-frequency details are not very clear when the method is dealing with extremely low sampling rate. As Fig. 13 shows, the maximum error is greater than 50% in the area around the edge of the hand due to the loss of high frequency information and the errors in surface curvature. A potential solution is to introduce better and flexible sampling method, which can not only be used to recover the details from low-frequency domain but also high-frequency details. Wenwen et al.²⁰ proposed variable density sampling matrix and CS algorithm to achieve super-resolution imaging. Although it has improved the accuracy of the image with more accurate details, the computational efficiency is decreased and it does not apply to MIs. It may in fact demonstrate even greater potency and will be considered in future research.

Fig. 13

Pixelwise RMSE for 2% sample rate using the FSI parameter recovery algorithm.

5. Conclusion

In this study, we propose an effective approach for optical parameters estimation via single pixel detector. Our method achieves single-pixel SFDI and achieves the state-of-the art performance. The scope of this contribution is introducing Fourier single pixel imaging method to SFDI (FSI-SFDI), which is more advantageous to generate large size optical characteristic than previous methods. Our system replaces hyperspectral imaging cameras with cheaper single-pixel detector, which is more effective without clear performance reduction. The proposed takes advantage of the sparsity of natural signals in the frequency domain, and collects spectrum data using Fourier single-pixel imaging instead of the CS method, which reduces the computation time and equipment cost for real deployment in a clinical environment.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Acknowledgments

This work was supported, in part, by the Natural Science Foundation of Anhui Province of China, under Grant No. 1908085QH365; the Key Research and Development Plan of Anhui Province of China, under Grant No. 202004a07020016; and the President Foundation of the Hefei Institutes of Physical Science, Chinese Academy of Sciences, under Grant No. YZJJZX202009.

References

1.

Y. Zhao et al., “Angle correction for small animal tumor imaging with spatial frequency domain imaging (SFDI),” Biomed. Opt. Express, 7 (6), 2373 –2384 (2016). https://doi.org/10.1364/BOE.7.002373 BOEICL 2156-7085 Google Scholar

2.

R. H. Wilson et al., “Quantitative short-wave infrared multispectral imaging of in vivo tissue optical properties,” J. Biomed. Opt., 19 (8), 086011 (2014). https://doi.org/10.1117/1.JBO.19.8.086011 JBOPFO 1083-3668 Google Scholar

3.

Y. Xu, A. Giljum and K. F. Kelly, “A hyperspectral projector for simultaneous 3D spatial and hyperspectral imaging via structured illumination,” Opt. Express, 28 (20), 29740 –29755 (2020). https://doi.org/10.1364/OE.402812 OPEXFF 1094-4087 Google Scholar

4.

S. Gioux, A. Mazhar and D. J. Cuccia, “Spatial frequency domain imaging in 2019: principles, applications, and perspectives,” J. Biomed. Opt., 24 (7), 071613 (2019). https://doi.org/10.1117/1.JBO.24.7.071613 JBOPFO 1083-3668 Google Scholar

5.

Y. Zhao et al., “Shortwave-infrared meso-patterned imaging enables label-free mapping of tissue water and lipid content,” Nat. Commun., 11 (1), 5355 (2020). https://doi.org/10.1038/s41467-020-19128-7 NCAOBW 2041-1723 Google Scholar

6.

D. J. Cuccia et al., “Quantitation and mapping of tissue optical properties using modulated imaging,” J. Biomed. Opt., 14 (2), 024012 (2009). https://doi.org/10.1117/1.3088140 JBOPFO 1083-3668 Google Scholar

7.

M. Torabzadeh et al., “Compressed single pixel imaging in the spatial frequency domain,” J. Biomed. Opt., 22 (3), 030501 (2017). https://doi.org/10.1117/1.JBO.22.3.030501 JBOPFO 1083-3668 Google Scholar

8.

E. J. Candes and M. B. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag., 25 (2), 21 –30 (2008). https://doi.org/10.1109/MSP.2007.914731 ISPRE6 1053-5888 Google Scholar

9.

L. Bian et al., “Multispectral imaging using a single bucket detector,” Sci. Rep., 6 24752 (2016). https://doi.org/10.1038/srep24752 SRCEC3 2045-2322 Google Scholar

10.

M. P. Edgar, G. M. Gibson and M. J. Padgett, “Principles and prospects for single-pixel imaging,” Nat. Photonics, 13 (1), 13 –20 (2018). https://doi.org/10.1038/s41566-018-0300-7 NPAHBY 1749-4885 Google Scholar

11.

B. O. L. Mellors et al., “Applications of compressive sensing in spatial frequency domain imaging,” J. Biomed. Opt., 25 (11), 112904 (2020). https://doi.org/10.1117/1.JBO.25.11.112904 JBOPFO 1083-3668 Google Scholar

12.

Z. Zhang, X. Ma and J. Zhong, “Single-pixel imaging by means of Fourier spectrum acquisition,” Nat. Commun., 6 6225 (2015). https://doi.org/10.1038/ncomms7225 NCAOBW 2041-1723 Google Scholar

13.

Z. Zhang et al., “Fast Fourier single-pixel imaging via binary illumination,” Sci. Rep., 7 (1), 12029 (2017). https://doi.org/10.1038/s41598-017-12228-3 SRCEC3 2045-2322 Google Scholar

14.

K. Binder and D. W. Heermann, Monte Carlo Simulation in Statistical Physics: An Introduction, Springer, Cham (2019). Google Scholar

15.

“OpenSFDI analysis programme,” (2019). http://opensfdi.org/appsfdi/ Google Scholar

16.

M. Applegate et al., “OpenSFDI: an open-source guide for constructing a spatial frequency domain imaging system,” J. Biomed. Opt., 25 (1), 016002 (2020). https://doi.org/10.1117/1.JBO.25.1.016002 JBOPFO 1083-3668 Google Scholar

17.

D. J. Cuccia et al., “Modulated imaging: quantitative analysis and tomography of turbid media in the spatial-frequency domain,” Opt. Lett., 30 (11), 1354 –1356 (2005). https://doi.org/10.1364/OL.30.001354 OPLEDP 0146-9592 Google Scholar

18.

T. D. O’Sullivan et al., “Diffuse optical imaging using spatially and temporally modulated light,” J. Biomed. Opt., 17 (7), 071311 (2012). https://doi.org/10.1117/1.JBO.17.7.071311 JBOPFO 1083-3668 Google Scholar

19.

R. Tao, F. Zhang and Y. Wang, “Research progress on discretization of fractional Fourier transform,” Sci. China Ser. F: Inf. Sci., 51 (7), 859 –880 (2008). https://doi.org/10.1007/s11432-008-0069-2 Google Scholar

20.

M. Wenwen et al., “Sparse Fourier single-pixel imaging,” Opt. Express, 27 (22), 31490 –31503 (2019). https://doi.org/10.1364/OE.27.031490 OPEXFF 1094-4087 Google Scholar

21.

L. H. Bian et al., “Efficient single pixel imaging in Fourier space,” J. Opt.-UK, 18 (8), 085704 (2016). https://doi.org/10.1088/2040-8978/18/8/085704 Google Scholar

22.

J. Huang et al., “Computational-weighted Fourier single-pixel imaging via binary illumination,” Opt. Express, 26 (13), 16547 –16559 (2018). https://doi.org/10.1364/OE.26.016547 OPEXFF 1094-4087 Google Scholar

23.

E. Aguenounon et al., “Single snapshot of optical properties image quality improvement using anisotropic two-dimensional windows filtering,” J. Biomed. Opt., 24 (7), 071611 (2019). https://doi.org/10.1117/1.JBO.24.7.071611 JBOPFO 1083-3668 Google Scholar

24.

M. van de Giessen, J. P. Angelo and S. Gioux, “Real-time, profile-corrected single snapshot imaging of optical properties,” Biomed. Opt. Express, 6 (10), 4051 –4062 (2015). https://doi.org/10.1364/BOE.6.004051 BOEICL 2156-7085 Google Scholar

25.

J. Vervandier and S. Gioux, “Single snapshot imaging of optical properties,” Biomed. Opt. Express, 4 (12), 2938 –2944 (2013). https://doi.org/10.1364/BOE.4.002938 BOEICL 2156-7085 Google Scholar

26.

S. Panigrahi and S. Gioux, “Machine learning approach for rapid and accurate estimation of optical properties using spatial frequency domain imaging,” J. Biomed. Opt., 24 (7), 071606 (2018). https://doi.org/10.1117/1.JBO.24.7.071606 JBOPFO 1083-3668 Google Scholar

27.

Y. Zhao et al., “Deep learning model for ultrafast multifrequency optical property extractions for spatial frequency domain imaging,” Opt. Lett., 43 (22), 5669 –5672 (2018). https://doi.org/10.1364/OL.43.005669 OPLEDP 0146-9592 Google Scholar

28.

M. T. Chen et al., “GANPOP: generative adversarial network prediction of optical properties from single snapshot wide-field images,” IEEE Trans. Med. Imaging, 39 (6), 1988 –1999 (2020). https://doi.org/10.1109/TMI.2019.2962786 ITMID4 0278-0062 Google Scholar

Biography

Hui M. Ren received her BS degree in computer science and technology from Anhui University of Chinese Medicine in 2019 and is a candidate for the MS degree in computer science and technology at the Institute of Material Science and Information Technology, Anhui University. Her research interest is within the field of spatial frequency-domain imaging and graphics algorithms.

Guoqing Deng received his MS degree in optics from Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences in 2001 and his PhD in optics from Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences in 2006. He is a senior engineer and doctoral supervisor in Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences. He is mainly engaged in the research of new cell and nucleic acid detection technology based on microfluidic chip.

Peng Zhou is a visiting staff at the 940th Hospital of Joint Logistic Support Force of Chinese People’s Liberation Army, Department of Sports Medicine, Gansu, China. Prior to his current position, he received his BS degree in clinical medicine from Shihezi University, China. And he is a candidate for the MS degree in orthopedics at the Central South University in China. He is mainly engaged in the research of sports medicine.

Xu Kang received his BS degree in physics from Anhui University in 2019 and is a candidate for the MS degree in optics at the Institute of Material Science and Information Technology, Anhui University.

Yang Zhang received his MS degree in basic medicine from Huazhong University of Science and Technology and his PhD in electronic information from Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, in 2021. His research interests include biomedical photonics and basic medicine.

Jingshu Ni received his MS degree in pharmacology from Suzhou University in 2015 and is a candidate for the PhD in biophysics at Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences.

Yuanzhi Zhang is a researcher at Anhui Institute of Optics and Fine Mechanics, Chinese Academy of Sciences. His research focuses on the application and transformation of tissue spectroscopy in noninvasive detection of chronic diseases. Prior to his current position, he received his MS degree in optics and his PhD in optics from the University of Chinese Academy of Sciences.

Yikun Wang is an associate research fellow and master supervisor at Anhui Institute of Optics and Fine Mechanics. His main research fields are medical spectroscopy, spectral imaging, and medical photonics. He has published more 50 papers in his research fields and has received support from many grants, such as from the National Science and Technology Support Program of China.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Huiming Ren, Guoqing Deng, Peng Zhou, Xu Kang, Yang Zhang, Jingshu Ni, Yuanzhi Zhang, and Yikun Wang "Spatial frequency domain imaging technology based on Fourier single-pixel imaging," Journal of Biomedical Optics 27(1), 016002 (24 January 2022). https://doi.org/10.1117/1.JBO.27.1.016002

Received: 12 September 2021; Accepted: 27 December 2021; Published: 24 January 2022

Access the abstract

JOURNAL ARTICLE
15 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

KEYWORDS

Optical properties

Reconstruction algorithms

Tissue optics

Image restoration

Imaging technologies

Image resolution

Computer simulations

1.

Introduction

2.

Approach

2.1.

Spatial Frequency Domain Imaging

Eq. (1)

Eq. (2)

Eq. (3)

Eq. (4)

2.2.

Fourier-Based Single-Pixel Imaging

Eq. (5)

Eq. (6)

Eq. (7)

Eq. (8)

Eq. (9)

Eq. (10)

Fig. 1

Eq. (11)

2.3.

Fourier Single-Pixel Imaging-Based SFDI

Eq. (12)

Eq. (13)

Fig. 2

Fig. 3

2.4.

Data Simulation

2.5.

Error Metrics

Eq. (14)

Eq. (15)

Eq. (16)

3.

Results

3.1.

Comparison of Different Sampling Schemes

Fig. 4

3.2.

High Efficient Reconstruction Performance under Different Sampling Rates

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

3.3.

Memory and Time Efficiency with Large Resolutions

Table 1

Fig. 12

4.

Discussion

Fig. 13

5.

Conclusion

Disclosures

Acknowledgments

References

Biography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years