Abstract

This paper proposes a dialogue strategy for clarifying and constraining queries to document retrieval systems with speech input interfaces. It is indispensable for spoken dialogue systems to interpret user’s intention robustly in the presence of speech recognition errors and extraneous expressions characteristic of spontaneous speech. In speech input, moreover, users’ queries tend to be vague, and they may need to be clarified through dialogue in order to extract sufficient information to get meaningful retrieval results. In conventional database query tasks, it is easy to cope with these problems by extracting and confirming keywords based on semantic slots. However, it is not straightforward to apply such a methodology to general document retrieval tasks.

In this paper, we first introduce two statistical measures for identifying critical portions to be confirmed. The relevance score (RS) represents the matching degree with the document set. The significance score (SS) detects portions that affect retrieval results. With these measures, the system can generate confirmations to handle speech recognition errors, prior to and after the retrieval, respectively. Then, we propose a dialogue strategy for generating clarifications to narrow down the retrieved items, especially when many documents are matched because of a vague input query. The optimal clarification question is dynamically selected based on information gain (IG) – the reduction in the number of matched items. A set of possible clarification questions is prepared using various knowledge sources. As a bottom-up knowledge source, we extract a list of words that can take a number of objects and potentially causes ambiguity, using a dependency structure analysis of the document texts. This is complemented by top-down knowledge sources of metadata and hand-crafted questions.

Our dialogue strategy is implemented and evaluated against a software support knowledge base of 40 K entries. We demonstrate that our strategy significantly improves the success rate of retrieval.

Keywords: Spoken dialogue system; Information retrieval; Document retrieval; Dialogue strategy

Article Outline

1. Introduction

2. Document retrieval system with speech interface

2.1. System overview

2.2. Task and back-end retrieval system

3. Confirmation strategy for robust retrieval against ASR errors and redundancies in spoken language expressions

3.1. Prior confirmation using relevance score (RS)

3.1.1. Definition of relevance score

3.1.2. Confirmation for critical words using relevance score

3.2. Posterior confirmation using significance score (SS)

3.2.1. Definition of significance score

3.2.2. Confirmation using significance score

3.3. Experimental evaluation of confirmation strategy

3.3.1. Evaluation of retrieval success rate

3.3.2. Evaluation of efficiency of confirmation

4. Dialogue strategy to clarify user’s vague queries

4.1. Dialogue strategy based on information gain (IG)

4.2. Question generation based on bottom-up and top-down knowledge sources

4.2.1. Questions based on dependency structure analysis (method 1)

4.2.2. Questions based on metadata included in the KB (method 2)

4.2.3. Questions based on human knowledge (method 3)

4.3. Update of retrieval query sentence

4.4. Experimental evaluation

5. Conclusion

Acknowledgements

References