Imaging dynamic scenes with a spatio-temporally channeled polarimeter

Jiawei Song; Israel J. Vaughn; Andrey S. Alenin; J. Scott Tyo

doi:10.1364/OE.27.028423

1. Introduction

Passive polarimeters are devices that measure the polarization information in light, and they represent a powerful tool for optical sensing [1–4]. The Stokes formalism is commonly used to describe polarization measurements because it is more suitable for incoherent light description than other formulations such as the Jones and coherency formalisms [5]. The four Stokes parameters arranged in vector form are [5,6]

(1)$$\underline {\textbf S}(x,y,t) = \begin{bmatrix} s_0(x,y,t)\\ s_1(x,y,t)\\ s_2(x,y,t)\\ s_3(x,y,t) \end{bmatrix} = \begin{bmatrix} I_0\\ I_H-I_V\\ I_{45^\circ}-I_{135^\circ}\\ I_{LCP}-I_{RCP} \end{bmatrix}.$$

The parameter $s_0$ is proportional to the irradiance of light, $s_1$ is the prevalence of horizontal polarization over vertical, $s_2$ is the prevalence of the $45^\circ$ polarization over $135^\circ$, and $s_3$ is the prevalence of left-handed circular polarization over right. Here we have explicitly denoted the dependence of $\underline {\textbf S}(x,y,t)$ on space and time, but it may also depend on additional independent variables including wavenumber and angle of incidence.

Many different types of polarimeters have been investigated over the past decades, and using the nomenclature of a well-known review article [1], the most common types are division of amplitude polarimeters (DoAmP) [7], division of aperture polarimeters (DoAP) [8], division of focal-plane polarimeters (DoFP) [9] and division of time polarimeters (DoT) [10]. In addition, the class of beam-exchange polarimeters can be related to other classes, but uses the concept of beam exchange to help innoculate the system to residual calibration error [11].

More recently, we have advocated for a different categorization of polarimeters that separates instruments into two classes. The first class is what we have termed “wavefront division” polarimeters, and includes DoAmp and DoAp instruments, as well as many types of beam-exchange polarimeters. These split the incoming light into multiple copies using beamsplitters or optical assemblies, each of which is analyzed independently by a separate detector array or sub-array with its associated polarization optics. The second class includes the “snapshot polarimeters” that have grown in popularity since the 2006 review was published. These devices introduce polarization-dependent carriers in some combination of independent variables (e.g. space [12], wavenumber [13–16], angle of incidence [17,18], etc.) and measure the modulated carrier signals with a single detector (or detector array). The carrier signals create a set of side bands, or channels, carrying polarization information in the associated frequency domains. We refer to this class of polarimeters as modulated and/or channeled polarimeters in this manuscript.

Under the traditional categorization, DoT and DoFP polarimeters are usually thought of in the wavefront division context. DoT polarimeters independently analyze multiple measurements of the optical field in time, then use the data reduction matrix (DRM) to invert those intensity measurements to estimate polarization [6]. DoFP devices are most commonly treated using a superpixel formalism, where independent measurements from a set of four polarized pixels are combined using a DRM to estimate the Stokes parameters at the center of the superpixel [19–21]. However, a series of work over the past decade has shown that these instruments are better thought of as modulated polarimeters when they are designed specifically with periodic modulation in mind. Furthermore, processing DoT and DoFP devices using the conventional DRM formalism leads to reconstructed images with unnecessary artifacts. This was demonstrated by Tyo, et al. for DoFP polarimeters [22] and generally by LaCasse, et al., for DoT polarimeters [23], though the earlier work of Diner, et al., demonstrated a similar concept by augmenting the instrument matrix of a DoT polarimeter with extra rows in order to guarantee that particular artifacts (such as linear gradients in time) were in the null space of the DRM [24,25]. Diner’s work was the first, to our knowledge, that leveraged degrees of freedom in an over-determined DoT polarimeter to suppress particular modes of error in the reconstructed time series. Some classes of beam-exchange polarimeters combine the concepts of DoAmp and DoT by “swapping” the two channels on successive time samples in order to ensure the artifacts are identical in each of the two channels of the system.

Here we treat both DoT and DoFP instruments as channeled systems, but the modulated and DRM formalisms must be fundamentally the same, and this was demonstrated by LaCasse, et al. [26], who showed that they are just two examples of ways of choosing the null space of the data inversion (Diner’s method represents a third). [24,25].

Wavefront division polarimeters are able to acquire polarimetric images at the full spatial and temporal resolution of the underlying imager, but their construction, calibration and maintenance tend to be costly because each optical path must be equalized, aligned, cross-calibrated and temporally synchronized [1]. They have their known strengths and weaknesses, and discussion of those is outside the scope of this paper. In contrast, the alignment and synchronization for modulated polarimeters are more easily achieved since the scenes are detected by a single detector array. However, modulated polarimeters suffer from resolution loss because the native bandwidth of the detector is divided up to measure multiple polarization parameters [23,27].

To combat this resolution loss LeMaster and Hirakawa [28] compared DoFP polarimeters to the color filtering case and improved the image resolution by optimizing the separation between the side bands in the spatial frequency plane. Alenin, et al [29,30], further improved the spatial only case, and introduced a class of multi-snapshot systems that specifically traded temporal bandwidth off to improve the potential spatial resolution (the $2\times 2\times 2$ and $2\times 2\times 3$ strategies in those references for linear and full Stokes polarimeters, respectively).

We have examined the concept of hybrid-domain modulation that creates channels simultaneously in two or more of the dimensions mentioned above, e.g. space and time [31,32]. Following the methods laid out by Vaughn, et al [33], the modulations can be designed in a way that allows certain channels to cancel, opening up bandwidth in the resulting frequency space. In recent work, we demonstrated theoretically and numerically a specific set of modulation strategies that trade off spatial and temporal bandwidth to create a system that out-performs spatial or temporal modulation alone over a broad range of scene conditions [32]. In this paper, we verify these predictions by comparing reconstructions using spatial-only, temporal-only and a spatio-temporal processing, and we demonstrate quantitatively the tradeoff between temporal and spatial bandwidth with dynamic scenes of varying spatial content.

The remainder of this paper is organized as follows. Section 2. presents the theoretical channel structure of the experimenal system. Section 3. describes the experimental apparatus and measurements. Section 4. discusses the experimental results demonstrating the space-time resolution of the system, and conclusions are drawn in 5..

2. Channel structure

We have fully developed the general theory of channeled polarimeters elsewhere [34], but we reproduce the basic elements here for the reader’s convenience. The irradiance measured by a channeled polarimeter can be expressed as the product of the system analyzer and the Stokes parameters of the incident light

(2)$$I_0(\vec{\theta}) = \underline {\textbf A}_{\textrm{sys}}^T\cdot\underline {\textbf S}_{\textrm{in}} = \displaystyle\sum_{i=0}^{3} a_i(\vec{\theta})\cdot s_i(\vec{\theta}),$$

where $\vec {\theta }=\{x,y,t,\lambda ,etc\}$ is a vector that includes all the independent variables over which the polarimeter modulates, and $\underline {\textbf A}_{\textrm {sys}}=[a_0,a_1,a_2,a_3]^T$ describes the system’s polarization analyzer as a function of the variables $\vec {\theta }$. In this paper, $\vec {\theta }=\{x,y,t\}$. In the Fourier domain equation (2) becomes

(3)$$\tilde{I}_0(\tilde{\vec{\theta}})=\tilde{\underline {\textbf A}}_{\textrm{sys}}^T\ast\tilde{\underline {\textbf S}}_{\textrm{in}}=\displaystyle\sum_{i=0}^{3}\tilde{a}_i(\tilde{\vec{\theta}})\ast\tilde{s}_i(\tilde{\vec{\theta}}),$$

where the tilde indicates the functions being Fourier transformed, $\tilde {\vec {\theta }}=\{ \xi ,\eta ,\nu \}$ denotes the frequency variables that correspond to $\vec {\theta }$ in the respective Fourier domain, and the symbol $\ast$ denotes convolution. The carrier signals that make up the analyzer vector in the direct domains will transform into a set of side bands in the Fourier domain (i.e. the Fourier transform of a product of sinusoids is a set of $\delta$-functions). The complete set of side bands form the channel structure of the modulated polarimeter [34]. The data distribution in the channel structure can be described by $\underline {\underline {\textbf Q}}$ which is introduced by Alenin and Tyo [34]. This is a $N_c\times 4$ matrix that maps the Fourier transformed Stokes parameters into the channels

(4)$$\underline {\textbf C} = \underline{\underline {\textbf Q}}\cdot\tilde{\underline {\textbf S}},$$

where $N_c$ denotes the number of channels, $\underline {\textbf C}$ is a vector that contains the Fourier transform of the measured data at baseband in each of the $N_c$ channels, and $\tilde {\underline {\textbf S}}$ is the Fourier transformed incident Stokes vector at base band. The Stokes parameters can be extracted from the measured data in Eq. 2 by demodulating the channels down to base band and inverting $\underline {\underline {\textbf Q}}$ to unmix the demodulated channels.

Consider a conventional DoFP system, which we treat here as spatially modulated (Fig. 1(a)). The micropolarizer array (MPA) is tiled with a $2\times 2$ pattern as shown in Fig. 1(a) was first introduced by Chun [19]. Other MPA tilings (such as the $2\times 4$ by LeMaster and Hirakawa [28] and $2\times L$ by Alenin, et al [29]) providing wider channel separations have since been designed, but here only the conventional MPA is included since it is the only type that is commercially available. The periodically varying angle pattern of the conventional MPA generates a system analyzer with a set of harmonic carriers

(5)$$\underline {\textbf A}_{\textrm{sys}} = \underline {\textbf A}_{\textrm{MPA}}(m,n) = \frac{1}{4}\begin{bmatrix} 2 \\ \cos\pi m + \cos\pi n \\ \cos\pi m - \cos\pi n \\ 0 \end{bmatrix} ,(m,n \in \mathbb{Z}),$$

where $m$ and $n$ are the sampling indices on the MPA in the $x$- and $y$-directions, respectively. Note that we are considering ideal point sampling in both space here and in time below, which is not strictly accurate since the pixels have finite area and the camera will have a finite integration time [33,35]. However, it is straightforward to add these low-pass filtering effects to the analysis, and they do not affect the general understanding [23]. The channel structure generated by the conventional MPA is obtained by taking the discrete-space Fourier transform of $\underline {\textbf A}$ and is shown in Fig. 2(A). The channels containing $s_1 + s_2$ and $s_1-s_2$ are located at $\pm 0.5$ cycles per pixel on the spatial frequency plane ($\nu =0$) because the MPA samples the scene at the Nyquist rate in both $x$ and $y$ [22].

Fig. 1. System structure diagrams. LP is the linear polarizer. RHWP is the rotating half waveplate

Download Full Size | PDF

Fig. 2. Channel structures in the error-free case: (A) Spatial-only configuration (B) Hybrid spatio-temporal configuration (C) Temporal-only configuration. Blue triangles denote the $\tilde {s}_0$ channel; purple triangles denote the $\tilde {s}_1+\tilde {s}_2$ channels; yellow triangles denote the $\tilde {s}_1-\tilde {s}_2$ channels; green triangles denote $s_1 \pm j s_2$ channels. The size of each triangle indicates the amplitude of the channel. The units on $\xi$ and $\eta$ are cycles per pixel, and the units on $\nu$ are cycles per frame.

Download Full Size | PDF

Next consider a conventional rotating half-wave-plate (HWP) DoT polarimeter as shown in Fig. 1(c). When the waveplate rotates $60^\circ$ per frame in front of a fixed, spatially constant linear polarization analyzer oriented at $0^\circ$, the analyzer vector of the polarimeter is

(6)$$\underline {\textbf A}_{0^\circ} = \frac{1}{2} \begin{bmatrix} 1 & \cos\frac{4}{3}\pi k & \sin\frac{4}{3}\pi k & 0 \end{bmatrix}^{\mathrm{T}},(k \in \mathbb{Z}),$$

where $k$ is the temporal sample index. The channel structure of the temporal domain modulated Stokes polarimeter is shown in Fig. 2(C). In this case, the polarization-carrying side bands are located at 0.33 cycles-per-sample along the temporal frequency axis. We cannot place these sidebands at 0.5 cycles per frame because it is not possible to reconstruct both the in-phase ($s_1$) and quadrature ($s_2$) information in the polarization side-bands if we sample exactly at the Nyquist rate. For the spatial case in Fig. 2(A), we can sample exactly at the Nyquist rate because we capitalize on the 2-dimensional symmetry of the spatial frequency plane.

Our recently simulated hybrid linear-Stokes polarimeter is shown in Fig. 1(b) [32]. This system uses both the MPA and a rotating half waveplate to generate spatio-temporal hybrid modulation. The effective analyzer is rotated at $45^\circ$ per frame, so the system analyzer is

(7)$$\underline {\textbf A}_{\textrm{sys}} = \frac{1}{2} \begin{bmatrix} 1\\ \frac{1}{2}\cos\pi m\cos\pi k + \frac{1}{2}\cos\pi n \cos\pi k \\ \frac{1}{2}\cos\pi m\cos\pi k - \frac{1}{2}\cos\pi n \cos\pi k \\ 0 \end{bmatrix}.$$

The effect of the rotating HWP is to expand the spatial frequency structure of Fig. 2(A) to the channel structure illustrated in Fig. 2(B).

Figure 2 illustrates the following properties of the three instrument designs. The spatially modulated system (Fig. 2(A)) can reconstruct the Stokes parameters $s_0$, $s_1$, and $s_2$ at every frame with a temporal bandlimit of 0.5 cycles per camera frame. However, the presence of the side bands at $(\xi ,\eta ,\nu )=(\pm 0.5,0,0)$ and $(\xi ,\eta ,\nu )=(0,\pm 0.5,0)$ means that the spatial bandwidth is reduced to $|\xi |,|\eta |\le 0.25$ cycles per pixel, as is well understood [22]. The DoT polarimeter in Fig. 2(C) has the full native spatial resolution of the detector array ($|\xi |,|\eta |\le 0.5$), but the required three samples in time limits the temporal resolution to one-third of that of the native imaging system. The hybrid polarimeter (Fig. 2(B)) can be thought of as a “multi-snapshot” system a la Alenin and Tyo [29,34]. Any setting of the HWP results in a snapshot linear polarimeter, but the reference axes of the system rotate with the HWP. In the frequency domain, this corresponds to moving the spatial domain side bands that contain polarization information from the $\nu =0$ plane to the $\nu =\pm 0.5$ planes. With this configuration, there is no interference between the $s_0$ and the polarized channels in either the spatial frequency plane $(\nu =0)$ or along the temporal frequency axis $(\xi ,\eta =0)$, so the bandwidth is dictated by the separation between the channels along diagonal directions. The system can reconstruct a temporally band-limited scene at the full resolution of the detector array, it can reconstruct a spatially band-limited scene at the full temporal resolution of the camera, or it can reconstruct a scene that violates both sampling criteria simultaneously, so long as there is no overlap in the 3-dimensional frequency volume. This last point is important; the hybrid system can in fact be processed either as a spatial-only system by considering each frame independently, or as a DoT system by treating each pixel independently. As a result, it is the choice of how the data is filtered in the frequency domain that ultimately dictates how the overall space-time bandwidth of the instrument is used to reconstruct polarization information. Figure 3 illustrates $\underline {\underline {\textbf Q}}$ for the three cases in Fig. 2

Fig. 3. Channel structures in the error-free case, shown as the $4 \times 27$ $\underline {\underline {\textbf Q}}^{T}$ matrix. Each column represents a potential Fourier-domain channel at the frequencies labeled at the bottom of the figure. Not all channels are used in all systems, but all are included for consistency. The baseband $(0,0,0)$ channel is depicted in the center column. $\nu _{max/min}$ is $\pm 0.5$ for system A/B and $\pm 0.33$ for system C. The colors of the squares indicate the coefficients: oe-27-20-28423-i001 = 1, oe-27-20-28423-i002 = −1, oe-27-20-28423-i003 = −i, oe-27-20-28423-i004 = i, the vertical height of the rectangle represents the relative signal strength in the corresponding channels

Download Full Size | PDF

3. System design

3.1 Physical system

The prototyped system uses a 4D Technologies PolarCam camera with a 1008 x 1008 FPA and a Pentax lens (Pentax 200mm f/2.8 lens). The HWP is a Meadowlark precision achromatic waveplate for 720nm with a 50.8mm diameter mounted in a IntelLiDrives rotation stage.

The MPAs in the PolarCam device vary significantly on a pixel-to-pixel basis, and Fig. 4 shows the histograms of the angles and extinction ratios of our instrument as determined in the laboratory. The analysis presented above assumes that each pixel is as ideally designed, but the true values will result in incomplete channel cancellation, which creates small, interfering channels in between the designed channel locations due to the fact that the spatial and temporal modulations are obtained independently in this system. We have proposed a method to eliminate these channels that requires non-separable modulations schemes where the space and time modulations are inherently mixed [36], but have not yet developed a method to prototype that design.

Fig. 4. Histogram of angles (a) and extinction ratios (b) of the MPA polarizers.

Download Full Size | PDF

The intended channel structure requires a frame for every $45^\circ$ of waveplate rotation. When the stage is rotating, it outputs a digital encoder pulse train of 540,000 pulses per rotation. Therefore, the camera must be triggered every 67,500 pulse counts. Because of its fast clock speed (84MHz), an ARDUINO Due is used to monitor the pulse train coming from the rotary stage and send out triggering signals to the camera through the camera link card.

3.2 Channel filters

The inversion algorithm for channeled polarimeters requires that the frequency domain data to be filtered and then demodulated. Choice of the channel filter is therefore an important part of the system design. The filters extract the channels from the frequency space so $\underline {\underline {\textbf Q}}$ can separate the mixed Stokes parameters. The modulation frequencies determine the center locations of the various channels, but the data will exist in a distribution around those centers dictated by the spatial and temporal polarimetric bandwidth of the scene being imaged. When the data are truly band-limited and noise-free in all sampled dimensions, then it is straightforward to design a filter to separate the sidebands. However, with realistic data, choice of filter can impact SNR, channel crosstalk, and the accuracy of reconstruction. In principle, the filter should be chosen based on the data being sampled (or its statistics) [37], and adaptive filtering provides great promise and is the subject of ongoing research [38].

In the systems presented below, the filters are designed based on the Planck-taper window function because it is infinitely differentiable and the transition from 0 to 1 for each dimension is controllable by a single parameter. The Planck-taper window equals 1 near the center, 0 outside the window, and is defined inside the window as

(8)$$f(r,\epsilon) =\frac{1}{1+\exp{(\epsilon W[\frac{1}{W-r}+\frac{1}{(1-\epsilon)W-r}])}}, (1-\epsilon)W\leq r \leq W,$$

where $W$ is the width of the window, $r$ is the distance to the center and $\epsilon$ controls the width of the slope. Note that the filter parameters can be independently selected for each dimension, though we choose them to be identical for the spatial frequency dimensions $\xi$ and $\eta$.

A 1951 USAF resolution test chart rotating at multiple angular speeds is used to create the scenes presented below. However, since the “truth” data of the real scenes cannot be known exactly, the channel filters are optimized using simulated scenes of a rotating resolution chart. Channel filters were chosen using a particle swarm optimization routine that minimizes mean square error of the reconstructions. Using the hybrid system as an example, the Fourier transformed data have a channel structure as shown in Fig. 5(a). The base band filter used in the hybrid linear-Stokes polarimeter is illustrated in Fig. 5(b). The parameters of the three dimensions ($\xi$, $\eta$, $\nu$) of the filters are optimized with an algorithm based on particle swarm optimization. In Table 1, the optimized filter parameters for the three situations presented in section 4. are listed as examples.

Fig. 5. (a) Fourier transform of the data showing the channel structure of the hybrid system. (b) Channel filter for the baseband ($s_0$) channel. (c) Filtered version of (a).

Download Full Size | PDF

Table 1. Filter parameters

View Table

3.3 Data Collection

To quantitatively compare the performances of the three different types of Stokes polarimeters, a $2^{\prime \prime }\times 2^{\prime \prime }$ positive (chrome pattern on clear background) 1951 USAF resolution test chart is used as the detection target. The resolution test chart is illuminated with an Epson E-TORL lamp with a diffuser in front and then polarized by a linear polarizer. The resolution test chart is mounted on a controllable rotary stage rotating at multiple angular speeds in order to create scenes with different temporal bandwidths. However, for each scene, although all the line pairs are rotating at the same angular speed, they have different tangential velocities due to their locations relative to the rotation center. The high spatial frequency line pairs closer to the center of the scene move more slowly than the low spatial frequency pairs. For any given rotation speed, the MTF is computed for each spatial frequency, and this will correspond to a given tangential velocity. Note that the tangential velocity must be perpendicular to the line pair to be meaningful. By collecting image sequences at many different rotation speeds, the system performance can be tabulated for a range of spatial frequencies and tangential velocities. Figure 6 shows the angular rotation speeds that are used to compute the MTF at 0.15.

Fig. 6. Illustration of the angular speed images that allow the MTF to be computed at a tangential velocity of 0.15 pixels per frame for each of the spatial frequencies. The range of tangential velocities at each spatial frequency arises from the fact that the blocks in the resolution chart have finite spatial extent.

Download Full Size | PDF

4. Results

4.1 Image reconstruction comparison

In this section, the reconstructed $s_1$ and DoLP images of three scenes where the resolution test chart rotating at $0^\circ$ per frame, $0.23^\circ$ per frame and $1.13^\circ$ per frame are presented in order to give an intuitive illustration about the advantages and disadvantages of the three modulation strategies.

When the scene is static (top rows of Figs. 7 and 8), it has no temporal bandwidth. Theoretically both the temporally modulated system and the hybrid modulated system can reconstruct the scene at full spatial resolution. In contrast, the snapshot system has to sacrifice spatial bandwidth as the channels containing the polarization information are all squeezed into the 2D spatial frequency plane where $\nu =0$. In this case, the spatial-only processing strategy has a clear drawback in reconstructing the high spatial frequency line pairs compared with the other two strategies (see Fig. 7, top row, line pair group 2). Additionally, as evident in Fig. 8, the DoFP polarimeter has strong edge artifacts even when the scene is static, and these are largely mitigated in the other two reconstructions.

Fig. 7. The reconstructed $S_1$ images of the three systems in three situations. Column A contains images reconstructed by the snapshot system, column B the hybrid system, column C the DoT system. Row 1 contains $s_1$ images reconstructed from the static scene, row 2 contains reconstructed $s_1$ images where the resolution target is rotating at 0.23$^\circ$ per frame and row 3 at 1.13$^\circ$ per frame.

Download Full Size | PDF

Fig. 8. The DoLP images of the three systems in three situations as in Fig. 7.

Download Full Size | PDF

The center rows of Figs. 7 and 8 are the reconstructed images where the resolution test chart is rotating at the angular velocity of $0.23^\circ$ per frame. Since the snapshot system uses a single frame to estimate the polarization properties, the motion of the scene has essentially no impact (ignoring tiny amounts of motion blur). The performance of the DoT polarimeter is clearly degraded because the scene’s temporal bandwidth increases cross-talk between the channels. The hybrid modulation strategy maintains approximately the same performance as the static scene situation for reconstructing high spatial frequency line pairs (see line pair group 2). However, there is clear ringing at low spatial frequencies for the hybrid and DoT systems because the low spatial frequency line pairs are further from the center of rotation and they move more quickly than the high spatial frequency line pairs which are closer to the center. The ringing results because the temporal bandwidth violates the sampling criterion, even though the spatial bandwidth does not.

The bottom rows of Figs. 7 and 8 show the reconstructed images from the scene where the resolution test chart is rotating at the angular speed of $1.13^\circ$ per frame. In the fast moving scene, the temporal bandwidth of the scene is too broad, and neither the hybrid modulated nor the temporal modulated system can reconstruct the polarization image properly due to the significant channel crosstalk. Meanwhile, the quality of images reconstructed by the snapshot system remains consistent because the angular velocity of the resolution target is slow enough given the short exposure time of the camera.

4.2 MTF distribution

Multiple image sequences are used to measure the MTF as a function of line pair tangential velocity and spatial frequency. Figure 9 shows the 3-dimensional distribution of these data, and Visualization 1 shows a dynamic rotation of this plot to further aid the reader’s understanding. Figure 9(a),(c),(e) show cross sections that demonstrate how the MTFs vary with spatial frequency when the line pairs are moving at the same tangential velocities. Figure 9(b),(d),(f) show cross sections that illustrate how the MTFs of line pairs of same spatial frequencies vary with tangential velocity.

Fig. 9. MTF distribution of the three different systems, the size of the markers are related to their MTF value. The blue/orange/yellow markers indicates the MTF values of line pair images reconstructed by the snapshot/hybrid/DoT system. Plots (a)/(c)/(e) demonstrate how MTF values vary with image resolution when the line pairs are static/moving slowly/moving quickly. Plots (b)/(d)/(f) indicate how the MTF values change with the line pair moving speed where the resolution of the line pairs are low, medium, or high spatial frequency.

Download Full Size | PDF

When the line pairs are static, the three systems have similar performance for low spatial frequencies as we see below 0.14 lp/px in Fig. 9(a). However, in the high frequency region (above 0.18 lp/px), the DoFP system performance falls off first, and the highest resolution it can reach is approximately 0.25 lp/px (as expected). The other two systems achieve similar image reconstructing performance for line pairs of up to 0.36 lp/px.

In Fig. 9(b), When the low spatial frequency line pairs move, the DoT system performance degrades first, then the hybrid system falls away (see tangential velocity below 0.53 pixels/frame). When all the line pairs are moving at the same but relatively slow tangential velocity (Fig. 9(c)), the performance of the DoT system is obviously lower than the other two. For the hybrid system, although its performance drops slightly compared with the static situation, it still out performs the other two systems for reconstructing high spatial frequency line pairs (above 0.18 lp/px).

As the velocities of the line pairs increases, the MTFs of the DoT system and hybrid system keep decreasing due to the sampling criteria violation. When all the line pairs are moving at the tangential velocity of 0.79 px/frm (Fig. 9(e)), the DoFP system still has a similar performance as slower speeds (compare frequency above 0.18 lp/px). However, the hybrid and DoT system have similar reconstruction limit at about 0.23 lp/px, but the hybrid system still has a better performance over the DoT system for line pairs below spatial frequency of 0.14 lp/px.

For line pairs of medium and high spatial frequencies, as the spatial frequency increases, the hybrid and DoT systems out perform the snapshot system in the low tangential speed region (Fig. 9(f) tangential speed near 0 pixel/s). As the tangential velocities increase (Fig. 9(d)), the DoT system and the hybrid system have similar sensitivity to motion, while the DoFP system is immune to the motion in the scene.

5. Conclusion and future work

In this work, we constructed a spatio-temporal modulated linear-Stokes polarimeter previously designed by our group [32,33]. We demonstrated the predictions of the earlier work that a hybrid modulated system allows spatial and temporal bandwidth to be traded off. We compared the hybrid polarimetric system to a DoFP system and a DoT system constructed using identical hardware. The results presented show the advantage of the DoFP system for measuring fast moving scenes, the advantage of the DoT system for measuring static scenes and the advantage of our hybrid system for measuring scenes with moderate spatial and temporal bandwidth as the previous theoretical work suggested. Notably, the DoFP and DoT images were actually constructed using the same PolarCam camera. The DoFP data were obtained by processing frame-by-frame, the DoT data were obtained processing pixel-by-pixel, and the hybrid data was obtained by using a spatio-temporal inversion. The most important aspect of these results is the possibility of constructing a Stokes polarimeter which is able to adaptively switch between the strategies based on the scene. Moreover, in future work, the spatio-temporal polarimeter performance may be further improved by using different reconstruction strategies in different parts of the image at the same time to create a fully adaptive polarimetric imager.

Funding

Asian Office of Aerospace Research and Development (FA2386-15-1-4098).

References

1. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef]

2. J. Stenflo, “Solar magnetic fields as revealed by stokes polarimetry,” The Astron. Astrophys. Rev. 21(1), 66 (2013). [CrossRef]

3. A. G. Ushenko, D. N. Burkovets, S. B. Yermolenko, A. Arkhelyuk, V. P. Pishak, A. Yuzko, O. V. Pishak, L. Plaviuk, A. Peresunko, and V. Znack, “Stokes polarimetry of biotissues,” in Fourth International Conference on Correlation Optics, vol. 3904 (International Society for Optics and Photonics, 1999), pp. 527–534.

4. D. C. Louie, L. Tchvialeva, S. Kalia, H. Lui, and T. K. Lee, “One-shot stokes polarimetry for low-cost skin cancer detection,” in Optics and Biophotonics in Low-Resource Settings V, vol. 10869 (International Society for Optics and Photonics, 2019), p. 1086904.

5. R. M. A. Azzam and N. M. Bashara, Ellipsometry and Polarized Light (North-Holland, New York, 1977).

6. R. A. Chipman, “Handbook of optics,” Polarimetry, 2nd ed. M. Bass, ed. McGraw Hill, New York2 (1995).

7. E. Compain and B. Drevillon, “Broadband division-of-amplitude polarimeter based on uncoated prisms,” Appl. Opt. 37(25), 5938–5944 (1998). [CrossRef]

8. J. L. Pezzaniti and D. B. Chenault, “A division of aperture mwir imaging polarimeter,” in Polarization Science and Remote Sensing II, vol. 5888 (International Society for Optics and Photonics, 2005), p. 58880V.

9. G. Myhre, W.-L. Hsu, A. Peinado, C. LaCasse, N. Brock, R. A. Chipman, and S. Pau, “Liquid crystal polymer full-stokes division of focal plane polarimeter,” Opt. Express 20(25), 27393–27409 (2012). [CrossRef]

10. P. Markowicz, W. Law, A. Baev, P. Prasad, S. Patskovsky, and A. Kabashin, “Phase-sensitive time-modulated surface plasmon resonance polarimetry for wide dynamic range biosensing,” Opt. Express 15(4), 1745–1754 (2007). [CrossRef]

11. M. Semel, J.-F. Donati, and D. Rees, “Zeeman-doppler imaging of active stars. 3: Instrumental and technical considerations,” Astron. Astrophys. 278, 231–237 (1993).

12. M. W. Kudenov, M. J. Escuti, N. Hagen, E. L. Dereniak, and K. Oka, “Snapshot imaging mueller matrix polarimeter using polarization gratings,” Opt. Lett. 37(8), 1367–1369 (2012). [CrossRef]

13. K. Oka and T. Kato, “Spectroscopic polarimetry with a channeled spectrum,” Opt. Lett. 24(21), 1475–1477 (1999). [CrossRef]

14. K. Oka and T. Kaneko, “Compact complete imaging polarimeter using birefringent wedge prisms,” Opt. Express 11(13), 1510–1519 (2003). [CrossRef]

15. N. Hagen, K. Oka, and E. L. Dereniak, “Snapshot mueller matrix spectropolarimeter,” Opt. Lett. 32(15), 2100–2102 (2007). [CrossRef]

16. F. Snik, T. Karalidi, and C. U. Keller, “Spectral modulation for full linear polarimetry,” Appl. Opt. 48(7), 1337–1346 (2009). [CrossRef]

17. T. Wakayama, Y. Otani, and T. Yoshizawa, “Axisymmetrical mueller matrix polarimeter,” in Polarization Science and Remote Sensing IV, vol. 7461 (International Society for Optics and Photonics, 2009), p. 74610M.

18. T. Wakayama, T. Higashiguchi, H. Oikawa, K. Sakaue, M. Washio, M. Yonemura, T. Yoshizawa, J. S. Tyo, and Y. Otani, “Determination of the polarization states of an arbitrary polarized terahertz beam: Vectorial vortex analysis,” Sci. Rep. 5(1), 9416 (2015). [CrossRef]

19. C. S. Chun, D. L. Fleming, and E. Torok, “Polarization-sensitive thermal imaging,” in Automatic Object Recognition IV, vol. 2234 (International Society for Optics and Photonics, 1994), pp. 275–287.

20. V. Gruev, J. V. der Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010). [CrossRef]

21. S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013). [CrossRef]

22. J. S. Tyo, C. F. LaCasse, and B. M. Ratliff, “Total elimination of sampling errors in polarization imagery obtained with integrated microgrid polarimeters,” Opt. Lett. 34(20), 3187–3189 (2009). [CrossRef]

23. C. F. LaCasse, R. A. Chipman, and J. S. Tyo, “Band limited data reconstruction in modulated polarimeters,” Opt. Express 19(16), 14976–14989 (2011). [CrossRef]

24. D. J. Diner, A. Davis, B. Hancock, G. Gutt, R. A. Chipman, and B. Cairns, “Dual-photoelastic-modulator-based polarimetric imaging concept for aerosol remote sensing,” Appl. Opt. 46(35), 8428–8445 (2007). [CrossRef]

25. D. J. Diner, A. Davis, B. Hancock, S. Geier, B. Rheingans, V. Jovanovic, M. Bull, D. M. Rider, R. A. Chipman, A.-B. Mahler, and S.C. McClain, “First results from a dual photoelastic-modulator-based polarimetric camera,” Appl. Opt. 49(15), 2929–2946 (2010). [CrossRef]

26. C. F. LaCasse, J. S. Tyo, and R. A. Chipman, “Role of the null space of the drm in the performance of modulated polarimeters,” Opt. Lett. 37(6), 1097–1099 (2012). [CrossRef]

27. C. F. LaCasse, J. S. Tyo, and R. A. Chipman, “The role of the null space in modulated polarimeters,” Opt. Lett. 37(6), 1097–1099 (2012). [CrossRef]

28. D. A. LeMaster and K. Hirakawa, “Improved microgrid arrangement for integrated imaging polarimeters,” Opt. Lett. 39(7), 1811–1814 (2014). [CrossRef]

29. A. S. Alenin, I. J. Vaughn, and J. S. Tyo, “Optimal bandwidth micropolarizer arrays,” Opt. Lett. 42(3), 458–461 (2017). [CrossRef]

30. A. S. Alenin, I. J. Vaughn, and J. S. Tyo, “Optimal bandwidth and systematic error of full-stokes micropolarizer arrays,” Appl. Opt. 57(9), 2327–2336 (2018). [CrossRef]

31. C. F. LaCasse, T. Ririe, R. A. Chipman, and J. S. Tyo, “Spatio-temporal modulated polarimetry,” in Polarization Science and Remote Sensing V, vol. 8160 (International Society for Optics and Photonics, 2011), p. 81600K.

32. I. J. Vaughn, A. S. Alenin, and J. S. Tyo, “Channeled spatio-temporal stokes polarimeters,” Opt. Lett. 43(12), 2768–2771 (2018). [CrossRef]

33. I. J. Vaughn, A. S. Alenin, and J. S. Tyo, “Focal plane filter array engineering i: rectangular lattices,” Opt. Express 25(10), 11954–11968 (2017). [CrossRef]

34. A. S. Alenin and J. S. Tyo, “Generalized channeled polarimetry,” J. Opt. Soc. Am. A 31(5), 1013–1022 (2014). [CrossRef]

35. I. J. Vaughn, A. S. Alenin, and J. S. Tyo, “Bounds on the microanalyzer array assumption,” in Polarization: Measurement, Analysis, and Remote Sensing XII, vol. 9853 (International Society for Optics and Photonics, 2016), p. 98530W.

36. J. Song, I. J. Vaughn, A. S. Alenin, M. E. Gehm, and J. S. Tyo, “Channel-first design of modulated polarimeters,” in Polarization: Measurement, Analysis, and Remote Sensing XIII, vol. 10655 (International Society for Optics and Photonics, 2018), p. 106550I.

37. I. J. Vaughn, A. S. Alenin, and J. S. Tyo, “Statistical scene generation for polarimetric imaging systems,” arXiv preprint arXiv:1707.02723, (2017).

38. C. F. LaCasse, O. G. Rodríguez-Herrera, R. A. Chipman, and J. S. Tyo, “Spectral density response functions for modulated polarimeters,” Appl. Opt. 54(32), 9490–9499 (2015). [CrossRef]

Dimensions		$ξ$	$η$	$ν$	$ξ$	$η$	$ν$	$ξ$	$η$	$ν$
System	Channels	$W$ / $ϵ$ (static)			$W$ / $ϵ$ (slow)			$W$ / $ϵ$ (fast)
Hybrid	$s_{0}$	0.39/0.25	0.39/0.25	0.10/0.23	0.38/0.27	0.38/0.27	0.23/0.29	0.38/0.26	0.38/0.26	0.50/0.26
Hybrid	$s_{1} / s_{2}$	0.38/0.25	0.38/0.25	0.10/0.12	0.38/0.26	0.38/0.26	0.13/0.14	0.39/0.26	0.39/0.26	0.33/0.21
DoFP	$s_{0}$	0.37/0.24	0.37/0.24		0.37/0.24	0.37/0.24		0.37/0.26	0.37/0.26
DoFP	$s_{1} / s_{2}$	0.26/0.20	0.26/0.20		0.26/0.20	0.26/0.20		0.26/0.18	0.26/0.18
DoT	$s_{0}$			0.20/0.25			0.23/0.24			0.28/0.35
DoT	$s_{1} / s_{2}$			0.18/0.29			0.17/0.27			0.13/0.15

Imaging dynamic scenes with a spatio-temporally channeled polarimeter

Abstract

1. Introduction

2. Channel structure

3. System design

3.1 Physical system

3.2 Channel filters

3.3 Data Collection

4. Results

4.1 Image reconstruction comparison

4.2 MTF distribution

5. Conclusion and future work

Funding

References

Supplementary Material (1)

Cited By

Figures (9)

Tables (1)

Equations (8)

Optics Express