Quality measures for speaker verification with short utterances

Highlights

• This work proposes new speech quality measures for quality-based score fusion in automatic speaker verification (ASV).

• The proposed quality measures are formulated from the zero-order Baum–Welch statistics.

• We fuse two popular ASV systems: Gaussian mixture model with universal background model (UBM) and i-vector.

• The experiments are conducted on NIST SRE 2008 and 2010 for various duration conditions.

• The proposed quality metric based ASV methods yield improved recognition performance.

Abstract

The performances of the automatic speaker verification (ASV) systems degrade due to the reduction in the amount of speech used for enrollment and verification. Combining multiple systems based on different features and classifiers considerably reduces speaker verification error rate with short utterances. This work attempts to incorporate supplementary information during the system combination process. We use quality of the estimated model parameters as supplementary information. We introduce a class of novel quality measures formulated using the zero-order sufficient statistics used during the i-vector extraction process. We have used the proposed quality measures as side information for combining ASV systems based on Gaussian mixture model–universal background model (GMM–UBM) and i-vector. The proposed methods demonstrate considerable improvement in speaker recognition performance on NIST SRE corpora, especially in short duration conditions. We have also observed improvement over existing systems based on different duration-based quality measures.

Introduction

The automatic speaker verification (ASV) technology uses the characteristics of human voice for the detection of individuals [1], [2]. The technology provides a low cost biometric solution suitable for real-world applications such as in banking [3], finance [4], and forensics [5]. Similar to other traditional pattern recognition applications, an ASV system includes three fundamental modules [1], [6]: an acoustic feature extraction unit that extracts relevant information from the speech signal in a compact manner, a modeling block to represent those features and a scoring and decision scheme to distinguish between genuine speakers and impostors. The state-of-the-art ASV system uses i-vector technology that represents a speech utterance with a single vector of fixed length either using Gaussian mixture model–universal background model (GMM–UBM) [7] or deep neural network (DNN) technology [8]. More recently, deep neural network (DNN) based embeddings are used for speaker recognition [9]. First, a DNN trained in a supervised manner to classify different speakers with known labels. Then, the trained DNN is employed to find a fixed-dimensional representation, known as x-vectors [9], corresponding to a variable length speech utterance.

Despite of these recent technological advancements, the mismatch issues are still a major concern for its real-world applications [10]. The performance of ASV system considerably degrades in presence of mismatch due to intra-speaker variability caused by the variations in speech duration [10], [11], background noise [12], vocal effort [13], spoken languages [14], emotion [15], channels [16], room reverberation [17], etc. In this paper, we focus on one of the most important mismatch factor, speech duration, the amount of speech data used in enrollment and verification.

State-of-the-art ASV systems exhibit satisfactory performance with adequately long (2 minutes) speech data. However, reduction in amount of speech drastically degrades the ASV performance [10], [12], [18], [19], [20]. The requirement of sufficiently long speech for training or testing, especially in presence of large intersession variability has limited the potential of widespread real-world implementations. An ASV system, in real world, is naturally constrained on the amount of speech data. Though this requirement can be fulfilled in training in some special cases, it is not always possible to maintain the same in verification for end-user convenience. In forensics applications, it is less likely to get sufficient data even for enrollment also [10], [19]. Therefore, getting reliable performance for short duration speech is one of the most important requirement in ASV application.

The performance of ASV systems are notably degraded with the reduction of amount of speech due to the lack of information provided in short utterance condition [19], [18], [21], [22]. In [7], it is reported that the i-vector based ASV systems are less sensitive to limited duration utterances than support vector machine (SVM) and JFA. The performance still deteriorates considerably with limited duration utterance as reported in [20], [18]. The duration variability problem is handled by extracting the duration pattern from the automatic speech recognition prior to modeling and scoring process in [23]. In [24], the short duration problem is approached, demonstrating the potential of fusion between GMM–UBM and SVM based systems using logistic regression. The work in [25] attempted to model the duration variability as noise and also by a synthetic process. The work in [26] has attempted to model variability caused by short duration segments in i-vector domain. In [27], [28], i-vector based ASV system is calibrated for short duration using duration based quality measures. The work in [29] attempted to improve short utterance speaker recognition by modeling speech unit classes.

The latest DNN-based speaker embedding approaches have shown promising results for speaker recognition with short utterances [9], [30]. Another recent work demonstrates that DNN-based i-vector mapping is useful for speaker recognition with short utterances [31]. Even though the DNN-based methods give good recognition accuracy, they require massive amount of training data, careful selection of network architecture and related tuning parameters. In this current work, we aim at improving the speaker recognition performance by efficiently combining two popular ASV systems based on GMM–UBM and i-vector representation which require lesser number of tuning parameters and amount of training data compared to the DNN-based methods. Moreover, the GMM–UBM and i-vector method are suitable with limited computational resources.

The research dealing with the effect of duration in speaker recognition have concentrated mostly on the consequences of classification performance, expressed in terms of equal error rate (EER) and minimum detection cost function (DCF) assuming the speaker model parameters are estimated satisfactorily. However, the speaker models are affected due to duration variability in short duration. The idea of quality metric was successfully applied in biometric authentication systems [32], [33]. The quality metrics were employed to improve the efficiency of the multi-modal biometric systems [34], [35], [36]. The work in [37] was motivated by a need to test claims that quality measures are predictive of matching performance. They also evaluated it by quantifying the association between estimated quality metric values and observed matching results.

The quality metrics are also successfully used in speech based bio-metric systems [38], [39]. The work in [39] studied a frame-level quality measure, obtaining encouraging results. However, the work in [38] showed a conventional user-independent multilevel SVM-based score fusion, adapted for the inclusion of quality information in the fusion process. The work in [40] focused on quality measure based system fusion, giving the emphasis on noisy and short duration test conditions using NIST 2012 database. The commonly used ASV systems, such as i-vector and GMM–UBM, do not include the information about the quality of estimated speaker models and information of duration variability. The work documented in [41], analyzed several quality measures for speaker verification from the point of view of their utility in an authentication task by selecting several quality measures derived from classic indicator like ITU P.563 estimator of subjective quality, signal to noise ratio and kurtosis of linear predictive coefficients. Moreover, the work [41] proposed a novel quality measure derived from what we have called universal background model likelihood (UBML). The work in [42], analyzed the factors that negatively impact the biometric quality and also depict a review of overall framework for the challenges of biometric quality.

The work in [28] used duration of speech segments to formulate the quality metrics and subsequently utilized the same for the calibration of recognition scores. However, the duration based quality metrics may not improve performance where the duration is fixed for either enrollment or verification or both. These durations based quality measures ignored the information of quality of speaker-model estimation. The quality of speaker-model parameters are not only dependent on duration, noise but also on phonetic distribution, intelligibility of speech etc. However, to develop a solution by targeting the basic building blocks of an ASV system, we attempted to incorporate the information of duration variability which degrades the quality of speaker-models. The concept of quality may be defined as degree of goodness of an element [39], [38], which, in our case, is the speaker-models. We treat BW statistics not only as the source of speaker information but also as a source of quality of estimated speaker models.

The Baum–Welch (BW) statistics, which represent the speech features in the intermediate step of i-vector extraction process, is affected by the duration variability. Consequently, the variability gets propagated in the subsequent representation, i.e., i-vector. We hypothesize that BW statistics can help to extract the quality of speaker-model parameters. We demonstrate through graphical analysis that the utterance duration is associated with the dissimilarity measures between intermediate statistics and background model parameters. We propose to use this measure as a quality measure. In this work, we propose to formulate this quality measure from the BW statistics and universal background model (UBM) parameters.

The proposed quality measures can be infused as additional information in the ASV technique to improve the system performance. The quality measures can be incorporated in potentially four possible stages of ASV system: feature extraction, speaker-model training, score computation and fusion of scores [38]. The use of quality measures in score fusion stage is most straightforward and has been successfully applied in speech, finger-print, face based multimodal person authentication systems [43], [44]. In this paper, we incorporate the proposed quality measures in score fusion stage to improve the performance of speaker recognition system in various duration conditions. In short duration, the linear score fusion strategy showed efficient performance with GMM and SVM based classification framework [24]. However, the i-vector based system (with GPLDA based channel compensation) was reported to perform more efficiently over JFA and GMM–SVM based framework in short utterance conditions [7]. Here, we show a comparative performance study of i-vector and GMM–UBM on NIST corpora (Fig. 1). We observe that though i-vector system performs better than GMM–UBM for long duration speech, the GMM–UBM system still shows comparable or even better performance for short duration conditions [45], [46]. This observation inspire to fuse i-vector and GMM–UBM to develop a more accurate and reliable solution for practical application of ASV systems. We have incorporated the estimated quality measures while blending the GMM–UBM and i-vector based ASV system. Incorporation of quality measures not only showed considerable improvement in performance but also consistency in various duration conditions. The proposed systems showed more relative improvement in short duration conditions which is more relevant for practical requirement. A preliminary version of this work was presented in [45]. In this work, we conduct extensive analysis and experiments.

The rest of the paper is organized as follows. The theoretical aspects of classical GMM–UBM and i-vector GPLDA system are discussed in Section 2. Analysis on intermediate subsystems under different duration variability condition is presented in Section 3. Section 4 describes the proposed quality measures and quality aided fusion based system. Details of experimental setup are provided in Section 5. The comparison of performance metrics of baseline GMM–UBM and i-vector GPLDA based ASV system and results on proposed quality aided fusion system are reported in Section 6. Finally, conclusion is drawn in Section 7.