Maximum entropy: A complement to Tikhonov regularization for determination of pair distance distributions by pulsed ESR

https://doi.org/10.1016/j.jmr.2005.07.021Get rights and content

Abstract

Tikhonov regularization (TIKR) has been demonstrated as a powerful and valuable method for the determination of distance distributions of spin-pairs in bi-labeled biomolecules directly from pulsed ESR signals. TIKR is a direct method, which requires no iteration, and, therefore, provides a rapid and unique solution. However, the distribution obtained tends to exhibit oscillatory excursions with negative portions in the presence of finite noise, especially in the peripheral regions of the distribution. The Shannon–Jaynes entropy of a probability distribution provides an intrinsic non-negativity constraint on the probability distribution and an unbiased way of obtaining information from incomplete data. We describe how the maximum entropy regularization method (MEM) may be applied to solve the ill-posed nature of the dipolar signal in pulsed ESR. We make use of it to suppress the negative excursions of the distance distribution and to increase the tolerance to noise in the dipolar signal. Model studies and experimental data are investigated, and they show that, with the initial or “seed” probability distribution that is required for MEM taken as the TIKR result, then MEM is able to provide a regularized solution, subject to the non-negativity constraint, and it is effective in dealing with noise that is problematic for TIKR. In addition we have incorporated into our MEM method the ability to extract the intermolecular dipolar component, which is embedded in the raw experimental data. We find that MEM minimization, which is implemented iteratively, is greatly accelerated using the TIKR result as the seed, and it converges more successfully. Thus we regard the MEM method as a complement to TIKR by securing a positive pair distance distribution and enhancing the accuracy of TIKR.

Introduction

Tikhonov regularization (TIKR) [1] has recently been introduced [2], [3], [4] to extract distance distributions between spin-pairs from intramolecular dipole–dipole interactions measured by pulsed electron spin resonance (ESR) techniques [5], in particular double quantum coherence (DQC) ESR and double electron–electron resonance (DEER). The ability to obtain such distributions, for distances as great as 7 nm [6], significantly enhances the application of ESR techniques in studying protein structure [7], [8], [9] in large biomolecules. A particularly significant application is in providing distance distributions of intermediate folding states of proteins [2], [10], [11].

The integral equation [2] that connects the intramolecular dipole–dipole interactions, which are measured in the ESR experiment, with the distance distributions between spin-pairs is in the form of a Fredholm equation of the first kind [1], that can be represented in discrete form as a linear equation K (r,t)P (r) = S0 (t), where K is a kernel matrix representing the shape of a Pake doublet dipolar signal in the time domain for a given radial separation, r, while P is the vector representing the pair distance distribution, and S0 is the vector representing the intramolecular dipolar signal from pulsed ESR experiments, which includes experimental noise [2]. This discrete form, in the presence of finite noise, is ill-posed, so it cannot usually be solved in a straightforward manner by singular value decomposition (SVD) or other least squares methods, and, therefore, requires regularization methods. The difficulties connected with solving this ill-posed problem to extract P (r) from pulsed ESR experiments have been previously discussed using SVD and regularization methods [2], [3], [4].

Among the regularization methods, TIKR has been generally accepted as an important one. It has been demonstrated in model and experimental studies that TIKR with a regularization parameter determined by the L-curve criterion provides a mathematically reliable estimate for the distance distribution for reasonable noise levels (i.e., SNR of 30–500 [2]). Another virtue of this TIKR method is that it is easy to compute numerically and it leads to the unique solution for a given regularization parameter, λ. That is, the formal minimization of the TIKR functional given byΦTIKR[P]KP-S02+λ2P2yields the simple matrix equation [12]Pλ=KTK+λ2-1KTS0.This equation for the Tikhonov functional minimizer is readily solved by SVD to yield the form given by [12]Pλ=(Σ2+λI)-1VΣUTS0=i=1Rank(K)fiuiTS0σivi,wherefiσi2σi2+λ2to obtain the unique P for the given value of λ. In Eq. (2b), the kernel matrix K (of dimension M × N) is decomposed by SVD into two orthonormal matrices, V (whose columns are composed of the N-dimensional vectors vi) and U (whose columns are composed of the M-dimensional vectors ui), and a diagonal matrix Σ whose elements are the singular values, σi. It can be rewritten (as shown by the second equality on the right-hand side) as the standard SVD solution, but where the term for the ith singular value is multiplied by a filter function fi, which acts to screen out undesired small singular values, σi for which σi  λ. The filter, shown in Eq. (2b), is specifically for the functional in Eq. (1). The difference between various regularization methods lies in the way that the filters are defined. The analysis given by Eqs. (2a), (2b) is easily repeated for many values of λ, from which the L-curve criterion is readily applied to yield the optimum λ.

However, a possible criticism of this approach is that the P (r) obtained tends to exhibit oscillatory excursions with negative portions in the presence of finite noise, especially in the peripheral regions of the regularized distance distributions. A physically meaningful P (r) must, of course be non-negative, but corrupting noise has no such requirement. In model tests it was shown that the oscillations do not significantly affect the main distribution. It nevertheless is desirable to constrain the P (r) to be positive, so the TIKR method of Eq. (1) has been adapted to include this constraint, and this has been applied to pulsed ESR experiments [3], [4]. The resulting computational algorithm, using a self-consistent (SC) approach to determine the optimum λ and subjecting to the non-negativity constraint, requires iterative methods [13] that may be initiated with a good starting guess of P (r) [14]. (These authors do supply a sophisticated deterministic annealing method when good initial values are not available [14].) Thus the introduction of the non-negativity constraint leads to a computationally more cumbersome algorithm.

In the present paper, we discuss the use of a different approach than Eq. (1) for the regularization in pulsed ESR. It is based on maximizing the entropy (ME) function associated with P (r). This implicitly restricts P (r) to be positive, but also at the expense of computational ease, requiring a numerical non-linear minimization procedure. In one study [15] the ME approach was found to be computationally significantly faster than constrained TIKR. Our main objective is to determine whether the inclusion of this information-theoretic ME principle would lead to improved estimates of P (r) under conditions of lower SNR typically encountered in real experiments. Furthermore, we use the proposed method to determine P (r) and to simultaneously extract information about the baseline in the experimental data. The ME minimization is implemented using a conjugate-gradient (CG) algorithm, which provides rapid local convergence. A good initial guess of P (r) to seed the iterative minimization is crucial to obtaining useful results by this method. Thus we also address this issue and we report an approach that we find to work quite well, that is based on using the result of Eqs. (2a), (2b) in conjunction with the L-curve criterion, as the seed to initiate the ME regularization approach.

Maximum entropy (ME) has been widely used as a general and powerful method for reconstructing results from noisy and incomplete data in various fields (e.g., image reconstruction [16] such as radio astronomy, medical tomography, and X-ray imaging; and fluorescence spectroscopy [17]). The idea is that by maximizing the Shannon–Jaynes entropyE=-α(τ)lnα(τ)dτ,where α (τ) is the function to be determined, the solution is determined/selected from the many sets of functions that can fit the data. This underlying idea has been well explained in information theory and statistical mechanics by taking entropy as a probabilistic concept [18], [19], i.e., the entropy of a probability distribution can be considered as a measure of the uncertainty of the experimental outcomes. The virtues of using the maximum entropy concept are that: (i) it provides an unbiased way of obtaining information from incomplete data; (ii) it implicitly possesses the non-negativity constraint to the probability distribution. Application of ME in the context of image reconstruction was originally introduced by Frieden [20] and later, was found useful in this field. Many efforts have been made using ME to reconstruct an image from incomplete/noisy data [16], [21], [22], and even to increase SNR in NMR spectroscopy [23], [24].

In this report, we replace the penalty term of the Tikhonov regularization functional, i.e., the second term on the right-hand side of Eq. (1) with the entropy Eq. (3) to solve the ill-posed inverse problem that we encounter in determining pair distance distributions from pulsed ESR experiments. In the past, the convergence behavior of the functional has been investigated and tested with numerical experiments. A transformed form of the entropy, which retains the advantages of using the Shannon–Jaynes entropy, is suggested in the literature and used in the analysis to provide better convergence for the functional minimization [25], [26]. Details about this transformation are given in Section 2 and discussed in later sections. The model distributions and experimental data we use are the same as those used previously [2]. We find that ME regularization is best utilized as a complement to TIKR for the determination of pair distance distributions. In fact, in the early stages of this work we found that ME regularization used independently of TIKR failed to recover satisfactory results (as compared to the TIKR results) in all tested cases. However, when the result of the TIKR was used to initiate the ME regularization it does become a viable method.

The dipolar time evolution signal obtained from pulse ESR experiments includes both intra- and intermolecular interactions, where the former is what we desire and the latter usually is unwanted. The intermolecular contribution manifests itself as a modification of a DEER signal envelope, thus producing a large decaying “baseline”; whereas in DQC it leads to a small offset [7], whose slope increases with concentration. It also leads to a damping of the intramolecular signal. Analysis of the dipolar signal often becomes difficult when the spin concentration is increased, which makes these concentration effects stronger. The removal (typically by subtraction) of the baseline from raw experimental data has been performed independently of the determination of distance distributions of spin-pairs [2], [7], [27], [28], [29]. Such a procedure is also commonly used in the spectral analysis of nuclear ESEEM.

One proposed method was that intramolecular interactions could be separated from intermolecular interactions by studying the concentration dependence of the pulsed ESR signals. More typically, the baseline function contributed from intermolecular interactions is generally obtained by fitting a low-degree polynomial to the raw experimental data. This method, in some cases of DEER experiments, usually utilizes weighted fitting in certain regions (e.g., weighted fitting of the last part of the raw data) where intermolecular interactions dominate the overall decay. In other words, the baseline subtraction requires some prior insight. In this report we propose a more unified way, which is based on the ME regularization, to simultaneously extract the intermolecular contribution in the raw experimental data and determine the spin-pair distance distribution from the intramolecular dipolar interactions. A virtue of such an approach is that all the data are utilized to fit the baseline as well as the P (r), thus, this approach is more likely to avoid bias from, e.g., artifacts in local regions of the signal.

Three models, bimodal, box-like, and broad trimodal distributions, are tested with the levels of SNR  500 and 30. The model study is first performed without considering any baseline in the time evolution data and then tested with a model baseline in the analysis. The dipolar time evolution signals of the proteins T4 lysozyme and cytochrome c, respectively measured by DQC-ESR and DEER techniques, are analyzed using the ME regularization method. The improvements include successful suppression of the peripheral noise-like oscillations of the Tikhonov results, the better tolerance to the noise levels of SNR < 30, and simultaneous determination of the intermolecular interaction and the spin-pair distance distribution. Based on these improvements, ME regularization is proposed as a complement to the Tikhonov regularization using the L-curve criterion, and we suggest that a similar strategy might be useful for the self-consistent (SC) method of TIKR.

Section snippets

MEM functional and its minimization

The maximum entropy regularization functional is given below in Eq. (4). One readily recognizes that the introduction of ME to the least squares problem is equivalent to the method of Lagrange multipliers. ME was first rigorously studied as a regularization method by Klaus and Smith [30]. It uses the entropy as the penalty term and is what we call MEM in this report. (Note that MEM is referred to maximizing the Shannon–Jaynes entropy instead of a regularization method in some literature

Results for the model data

Three model distributions

Results for the experimental data

The experimental data (i.e., the manually pre-processed baseline-subtracted data) that we previously analyzed using TIKR was used without any modifications in this report on the MEM analysis. Hereafter, we call the experimental signals before and after baseline subtraction as “raw experimental signals” and “intramolecular dipolar signals,” respectively. The latter is the “baseline-subtracted” signal obtained using Eqs. (6a), (6b), (6c) to manually remove the intermolecular component. Two

MEM used as a complement to Tikhonov regularization

The maximum entropy method of regularization has been rigorously examined for its stability as well as its convergence [25], [26], and was previously mainly applied to analyses of sums of exponentials [39]. The previous study which most relates to our present study is that of Amato and Hughes [15]. They demonstrated the convergence of the solution of the minimization problem (i.e., Eq. (5)) by showing the procedure leads to a valid regularization method. In addition, they compare the

Summary

The maximum entropy regularization method for the determination of distance distributions of spin-pairs directly from pulse ESR dipolar signals has been introduced as a complement to Tikhonov regularization in this study. From the model distributions investigated, it is clear that the Tikhonov regularization must first be used. The MEM regularization, which implicitly guarantees a non-negative solution, can then be used to refine the TIKR result that implicitly guarantees a non-negative

Acknowledgments

This work was supported by grants from NIH/NCRR, NIH/NIBIB, and NSF/Chemistry. It made use of the computer facility of the Cornell Center for Materials Research (CCMR).

References (44)

  • E.V. Pletneva et al.

    Many faces of the unfolded state: conformational heterogeneity in denatured yeast cytochrome c

    J. Mol. Biol.

    (2005)
  • C.W. Groetsch

    The theory of Tikhonov regularization for Fredholm equations of the first kind

    (1984)
  • G. Jeschke et al.

    Data analysis procedures for pulse ELDOR measurements of broad distance distributions

    Appl. Magn. Reson.

    (2004)
  • M.K. Bowman et al.

    Visualization of distance distribution from pulsed double electron–electron resonance data

    Appl. Magn. Reson.

    (2004)
  • L.J. Berliner et al.

    Biological magnetic resonance: distance measurements in biological systems by EPR

    (2000)
  • P.P. Borbat et al.

    Measurement of large distances in biomolecules using double-quantum filtered refocused electron spin-echoes

    J. Am. Chem. Soc.

    (2004)
  • P.P. Borbat et al.

    Protein structure determination using long-distance constraints from double-quantum coherence ESR: study of T4 lysozyme

    J. Am. Chem. Soc.

    (2002)
  • V.M. Grigoryants et al.

    Method of rapid mix EPR applied to the folding of bi-spin-labeled protein as a probe for the dynamic onset of interaction between sequentially distant side chains

    J. Phys. Chem. B.

    (2004)
  • K. DeWeerd et al.

    EPR-detected folding kinetics of externally located cysteine-directed spin-labeled mutants of iso-1-cytochrome c

    Biochemistry

    (2001)
  • P.C. Hansen

    Rank-deficient and discrete ill-posed problems : numerical aspects of linear inversion

    (1997)
  • U. Amato et al.

    Maximum-entropy regularization of fredholm integral-equations of the 1st kind

    Inverse Probl.

    (1991)
  • J. Skilling et al.

    Maximum-entropy image-reconstruction—general algorithm

    Monthly Notices R. Astron. Soc.

    (1984)
  • Cited by (138)

    • High-resolution approaches to ionospheric exploration

      2019, The Dynamical Ionosphere: A Systems Approach to Ionospheric Irregularity
    View all citing articles on Scopus
    View full text