Skip to main content
SpringerLink
Log in

Evaluation Methodologies

  • Chapter
  • First Online:
Book cover Handbook of Biometric Anti-Spoofing

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Anti-spoofing systems, regardless of the technique, biometric mode or degree of independence of external equipment, are most commonly treated as binary classification systems. The two classes that they differentiate are genuine accesses and spoofing attacks. From this perspective, their evaluation is equivalent to the established evaluation standards for the binary classification systems. However, the anti-spoofing systems are designed to operate in conjunction with recognition systems and as such can affect their performance. From the point of view of a recognition system, the spoofing attacks are a separate class that they need to detect and reject. As the problem of spoofing attacks detection grows to this pseudo-ternary status, the evaluation methodologies for the recognition systems need to be revised and updated. Consequentially, the database requirements for spoofing databases become more specific. The focus of this chapter is the task of biometric verification and its scope is threefold: first, it gives the definition of the spoofing detection problem from the two perspectives. Second, it states the database requirements for a fair and unbiased evaluation. Finally, it gives an overview of the existing evaluation techniques for anti-spoofing systems and verification systems under spoofing attacks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In this chapter, we shall treat as positive class or simply as positives, examples in a (discriminative) binary classification system one wishes to keep and, as negative class or negatives, examples that should be discarded.

References

  1. Rodrigues RN, Ling LL, Govindaraju V (2009) Robustness of multimodal biometric fusion methods against spoofing attacks. J Vis Lang Computing 20(3):169–179

    Article  Google Scholar 

  2. Johnson PA, Tan B, Schuckers S (2010) Multimodal fusion vulnerability to non-zero (spoof) imposters. In: IEEE international workshop information forensics and security

    Google Scholar 

  3. Akhtar Z, Fumera G, Marcialis GL, Roli F (2011) Robustness evaluation of biometric systems under spoof attacks. In: 16th International conference on image analysis and processing, pp 159–168

    Google Scholar 

  4. Akhtar Z, Fumera G, Marcialis GL, Roli F (2011) Robustness analysis of likelihood ration score fusion rule for multi-modal biometric systems under spoof attacks. In: 45th IEEE international carnahan conference on security technology, pp 237–244

    Google Scholar 

  5. Akhtar Z, Fumera G, Marcialis GL, Roli F (2012) Evaluation of serial and parallel multibiometric systems under spoofing attacks. In: 5th IEEE international conference on biometrics: theory, applications and systems

    Google Scholar 

  6. Villalba J, Lleida E (2011) Preventing replay attacks on speaker verification systems. In: 2011 IEEE international carnahan conference on security technology (ICCST), pp 1–8

    Google Scholar 

  7. Marasco E, Johnson P, Sansone C, Schuckers, S (2011) Increase the security of multibiometric systems by incorporating a spoofing detection algorithm in the fusion mechanism. In: Proceedings of the 10th international conference on multiple classifier systems, pp 309–318

    Google Scholar 

  8. Marasco E, Ding Y, Ross A (2012) Combining match scores with liveness values in a fingerprint verification system. In: 5th IEEE international conference on biometrics: theory, applications and systems

    Google Scholar 

  9. Chingovska I, Anjos A, Marcel S (2013) Anti-spoofing in action: joint operation with a verification system. In: Proceedings of IEEE conference on computer vision and pattern recognition, workshop on biometrics

    Google Scholar 

  10. Pan G, Sun L, Wu Z, Lao S (2007) Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: IEEE 11th international conference on computer vision, 2007. ICCV 2007, pp 1–8

    Google Scholar 

  11. Bao W, Li H, Li N, Jiang W (2009) A liveness detection method for face recognition based on optical flow field. In: International conference on Image analysis and signal processing, 2009. IASP 2009, pp 233–236

    Google Scholar 

  12. yan J, Zhang Z, Lei Z, Yi D, Li SZ (2012) Face liveness detection by exploring multiple scenic clues. In: 12th International conference on control, automation, robotics and vision (ICARCV 2012), China

    Google Scholar 

  13. Tan X, Li Y, Liu J, Jiang L (2010) Face liveness detection from a single image with sparse low rank bilinear discriminative model. In: Proceedings of European conference on computer vision (ECCV), LNCS, vol 6316. Springer, pp 504–517

    Google Scholar 

  14. Galbally J, Alonso-Fernandez F, Fierrez J, Ortega-Garcia J (2012) A high performance fingerprint liveness detection method based on quality related features. Future Gener Comput Syst 28(1):311–321

    Article  Google Scholar 

  15. Yambay D, Ghiani L, Denti P, Marcialis G, Roli F, Schuckers S (2012) LivDet 2011—fingerprint liveness detection competition 2011. In: 2012 5th IAPR international conference on biometrics (ICB), pp 208–215

    Google Scholar 

  16. Mansfield AJ, Wayman JL, Dr Rayner D, Wayman JL (2002) Best practices in testing and reporting performance. NPL Report CMSC

    Google Scholar 

  17. Galbally-Herrero J, Fierrez-Aguilar J, Rodriguez-Gonzalez JD, Alonso-Fernandez F, Ortega-Garcia J, Tapiador M (2006) On the vulnerability of fingerprint verification systems to fake fingerprints attacks. In: IEEE international carnahan conference on security technology, pp 169–179

    Google Scholar 

  18. Ruiz-Albacete V, Tome-Gonzalez P, Alonso-Fernandez F, Galbally J, Fierrez J, Ortega-Garcia J (2008) Direct attacks using fake images in iris verification. In: Proceedings of COST 2101 workshop on biometrics and identity management, BIOID, Springer, pp 181–190

    Google Scholar 

  19. Galbally J, Fierrez J, Alonso-Fernandez F, Martinez-Diaz M (2011) Evaluation of direct attacks to fingerprint verification systems. Telecommunication systems, special issue on biometrics 47(3):243–254

    Article  Google Scholar 

  20. Hastie T, Tibshirani R, Friedman JH (2001) The elements of statistical learning: data mining, inference, and prediction: with 200 full-color illustrations. Springer, New York

    Google Scholar 

  21. Lui YM, Bolme D, Phillips P, Beveridge J, Draper B (2012) Preliminary studies on the good, the bad, and the ugly face recognition challenge problem. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), pp 9–16

    Google Scholar 

  22. Marcialis GL, Lewicke A, Tan B, Coli P, Grimberg D, Congiu A, Tidu A, Roli F, Schuckers S (2009) First international fingerprint liveness detection competition - livdet 2009. In: Proceeding of IAPR international conference on image analysis and processing (ICIAP), LNCS, vol 5716. pp 12–23

    Google Scholar 

  23. Ghiani L, Yambay D, Mura V, Tocco S, Marcialis G, Roli F, Schuckers S (2013) Livdet 2013—fingerprint liveness detection competition. In: IEEE International conference on biometrics (ICB)

    Google Scholar 

  24. Zhiwei Z, Yan J, Liu S, Lei Z, Yi D, Li SZ (2012) A face antispoofing database with diverse attacks. In: Proceedings of IAPR international conference on biometrics (ICB), pp 26–31

    Google Scholar 

  25. Chingovska I, Anjos A, Marcel S (2012) On the effectiveness of local binary patterns in face anti-spoofing. In: Proceedings of IEEE international conference of the biometrics special interest group (BIOSIG), pp 1–7

    Google Scholar 

  26. Peixoto B, Michelassi C, Rocha A (2011) Face liveness detection under bad illumination conditions. In: 2011 18th IEEE International conference on image processing (ICIP), pp 3557–3560

    Google Scholar 

  27. Poh N, Bengio S (2006) Database, protocols and tools for evaluating score-level fusion algorithms in biometric authentication. Pattern Recognit J 39(2):223–233

    Article  Google Scholar 

  28. Martin A, Doddington G, Kamm T, Ordowski M (1997) The det curve in assessment of detection task performance. In: Proceedings of eurospeech, pp 1895–1898

    Google Scholar 

  29. Bengio S, Keller M, Mariéthoz J (2003) The expected performance curve. Technical report IDIAP-RR-85-2003, IDIAP

    Google Scholar 

  30. Wang L, Ding X, Fang C (2009) Face live detection method based on physiological motion analysis. Tsinghua Sci Technol 14(6):685–690

    Article  Google Scholar 

  31. Zhang Z, Yi D, Lei Z, Li S (2011) Face liveness detection by learning multispectral reflectance distributions. In: 2011 IEEE international conference on automatic face gesture recognition and workshops (FG 2011), pp 436–441

    Google Scholar 

  32. Johnson P, Lazarick R, Marasco E, Newton E, Ross A, Schuckers S (2012) Biometric liveness detection: framework and metrics. In: International biometric performance conference

    Google Scholar 

  33. Gao X, Tsong Ng T, Qiu B, Chang SF (2010) Single-view recaptured image detection based on physics-based features. In: IEEE international conference on multimedia and expo (ICME), Singapore

    Google Scholar 

  34. Tronci R, Muntoni D, Fadda G, Pili M, Sirena N, Murgia G, Ristori M, Ricerche S, Roli F (2011) Fusion of multiple clues for photo-attack detection in face recognition systems. In: Proceedings of the 2011 international joint conference on biometrics, IJCB ’11, IEEE Computer Society, pp 1–6

    Google Scholar 

  35. Jain AK, Flynn P, Ross AA (eds) (2008) Handbook of biometrics. Springer, New York

    Google Scholar 

  36. Adler A, Schuckers S (2009) Security and liveness, overview. In: Jain AK, Li SZ (eds) Encyclopedia of biometrics. Springer, New York

    Google Scholar 

  37. Galbally J, Cappelli R, Lumini A, de Rivera GG, Maltoni D, Fiérrez J, Ortega-Garcia J, Maio D (2010) An evaluation of direct attacks using fake fingers generated from iso templates. Pattern Recogn Lett 31(8):725–732

    Article  Google Scholar 

  38. Rodrigues R, Kamat N, Govindaraju V (2010) Evaluation of biometric spoofing in a multimodal system. In: 2010 Fourth IEEE international conference on biometrics: theory applications and systems (BTAS)

    Google Scholar 

  39. Matsumoto T, Matsumoto H, Yamada K, Hoshino S (2002) Impact of artifical “gummy” fingers on fingerprint systems. In: SPIE Proceedings: optical security and counterfeit deterrence techniques, vol. 4677

    Google Scholar 

  40. Patrick P, Aversano G, Blouet R, Charbit M, Chollet G (2005) Voice forgery using alisp: indexation in a client memory. In: Proceedings. (ICASSP ’05). IEEE international conference on acoustics, speech, and signal processing, 2005, vol 1, pp 17–20

    Google Scholar 

  41. Alegre F, Vipperla R, Evans N, Fauve B (2012) On the vulnerability of automatic speaker recognition to spoofing attacks with artificial signals. In: Signal processing conference (EUSIPCO), 2012 proceedings of the 20th European, pp 36–40

    Google Scholar 

  42. Bonastre JF, Matrouf D, Fredouille C (2007) Artificial impostor voice transformation effects on false acceptance rates. In: INTERSPEECH, pp 2053–2056

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank the projects BEAT (http://www.beat-eu.org) and TABULA RASA (http://www.tabularasa-euproject.org) both funded under the 7th Framework Programme of the European Union (EU) (grant agreement number 284989 and 257289) respectively.

Author information

Authors and Affiliations

  1. Idiap Research Institute, rue Marconi 19, 1920, Martigny, Switzerland

    Ivana Chingovska, André Anjos & Sébastien Marcel

Authors
  1. Ivana Chingovska
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. André Anjos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sébastien Marcel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ivana Chingovska .

Editor information

Editors and Affiliations

  1. Idiap Research Institute, Martigny, Switzerland

    Sébastien Marcel

  2. University of Southampton, United Kingdom

    Mark S. Nixon

  3. Institute of Automation Center for Biometrics and Security Resea, Chinese Academy of Sciences, Beijing, China

    Stan Z. Li

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag London

About this chapter

Cite this chapter

Chingovska, I., Anjos, A., Marcel, S. (2014). Evaluation Methodologies. In: Marcel, S., Nixon, M., Li, S. (eds) Handbook of Biometric Anti-Spoofing. Advances in Computer Vision and Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-4471-6524-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-6524-8_10

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-6523-1

  • Online ISBN: 978-1-4471-6524-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation