Skip to main content
SpringerLink
Log in

CUDA-aware MPI implementation of Gibbs sampling for an IRT model

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Item response theory (IRT) is a popular approach for addressing large-scale assessment problems in psychometrics and other areas of applied research. An emergent research direction that integrates it with machine learning techniques has made IRT applicable to a wide range of fields. The fully Bayesian approach for estimating IRT models is computationally expensive due to the large number of iterations, which require a large amount of memory to store massive amount of data. This limits the use of the procedure in many applications using traditional CPU architecture. In an effort to overcome such restrictions, previous studies focused on utilizing high performance computing using either distributed memory-based Message Passing Interface (MPI) or massive threads compute unified device architecture (CUDA) to achieve certain speedups with a simple IRT model. This study focuses on this model and aims at demonstrating the scalability of parallel algorithms integrating CUDA into MPI computing paradigm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  1. Bock, R.D., Aitkin, M.: Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika 46(4), 443–459 (1981)

    Article  MathSciNet  Google Scholar 

  2. Mislevy, R.J.: Estimation of latent group effects. J. Am. Stat. Assoc. 80(392), 993–997 (1985)

    Article  MathSciNet  Google Scholar 

  3. Patz, R.J., Junker, B.: A straightforward approach to Markov chain Monte Carlo methods for item response model. J. Educ. Behav. Stat. 24(2), 146–178 (1999)

    Article  Google Scholar 

  4. Tutakawa, R.K., Lin, H.Y.: Bayesian estimation of item response curves. Psychometrika 51(2), 251–267 (1986)

    Article  MathSciNet  Google Scholar 

  5. Bafumi, J., Gelman, A., Park, D.K., Kaplan, N.: Practical issues in implementing and understanding Bayesian ideal point estimation. Polit. Anal. 13(2), 171–187 (2005)

    Article  Google Scholar 

  6. Martin, C.S., Chung, T., Kirisci, L., Langenbucher, J.W.: Item response theory analysis of diagnostic criteria for alcohol and cannabis use disorders in adolescents: implications for DSM-V. J. Abnorm. Psychol. 115(4), 807–814 (2006)

    Article  Google Scholar 

  7. Feske, U., Kirisci, L., Tarter, R.E., Plkonis, P.A.: An application of item response theory to the DSM-III-R criteria for borderline personality disorder. J. Pers. Disord. 21(4), 418–433 (2007)

    Article  Google Scholar 

  8. Beseler, C.L., Taylor, L.A., Leeman, R.F.: An item-response theory analysis of DSM-IV alcohol-use disorder criteria and “binge’’ drinking in undergraduates. J. Stud. Alcohol Drugs 71(3), 418–423 (2010)

    Article  Google Scholar 

  9. Gilder, D.A., Gizer, I.R., Ehlers, C.L.: Item response theory analysis of binge drinking and its relationship to lifetime alcohol use disorder symptom severity in an American Indian community sample. Alcoholism 35(5), 984–995 (2011)

    Article  Google Scholar 

  10. Panter, A.T., Reeve, B.B.: Assessing tobacco beliefs among youth using item response theory models. Drug Alcohol Depend. 68(1), 21–39 (2002)

    Article  Google Scholar 

  11. Courvoisier, D., Etter, J.F.: Using item response theory to study the convergent and discriminant validity of three questionnaires measuring cigarette dependence. Psychol. Addict. Behav. 22(3), 391–401 (2008)

    Article  Google Scholar 

  12. Rose, J.S., Dierker, L.C.: An item response theory analysis of nicotine dependence symptoms in recent onset adolescent smokers. Drug Alcohol Depend. 110(12), 70–79 (2010)

    Article  Google Scholar 

  13. Fienberg, S.E., Johnson, M.S., Junker, B.W.: Classical multilevel and Bayesian approaches to population size estimation using multiple lists. J. R. Stat. Soc. Ser. A 162(3), 383–392 (1999)

    Article  Google Scholar 

  14. Reiser, M.: An application of the item-response model to psychiatric epidemiology. Sociol. Methods Res. 18(1), 66–103 (1989)

    Article  MathSciNet  Google Scholar 

  15. Orlando, M., Sherbourne, C.D., Thissen, D.: Summed-score linking using item response theory: application to depression measurement. Psychol. Assess. 12(3), 354–359 (2000)

    Article  Google Scholar 

  16. Tsutsumi, A., Iwata, N., Watanabe, N., de Jonge, J., Pikhart, H., Fernndez-Lpez, J.A., Xu, L., Peter, R., Knutsson, A., Niedhammer, I., Kawakami, N., Siegrist, J.: Application of item response theory to achieve cross-cultural comparability of occupational stress measurement. Int. J. Methods Psychiatr. Res. 18(1), 58–67 (2009)

    Article  Google Scholar 

  17. Chen, Z., Ahn, H.: Item response theory based ensemble in machine learning. Int. J. Autom. Comput. 17(5), 621–636 (2020)

    Article  Google Scholar 

  18. Martínez-Plumed, F., Pruden̂cio, R. B. C., Martínez-Usó, A., Hernańdez-Orallo, J.: Making sense of item response theory in machine learning. In: ECAI’16: Proceedings of the Twenty-second European Conference on Artificial Intelligence, pp. 1140–1148 (2016)

  19. Bergner, Y., Dröschler, S., Kortemeyer, G., Rayyan, S., Seaton, D., Pritchard, D. E.: Model-based collaborative filtering analysis of student response data: machine-learning item response theory. Paper presented at the International Conference on Educational Data Mining (EDM), Chania, Greece, June (2012)

  20. Birnbaum, A.: Statistical theory for logistic mental test models with a prior distribution of ability. J. Math. Psychol. 6(2), 258–276 (1969)

    Article  Google Scholar 

  21. Baker, F.B., Kim, S.H.: Item Response Theory: Parameter Estimation Techniques, 2nd edn. Dekker, New York (2004)

    Book  Google Scholar 

  22. Molenaar, I.W.: Estimation of item parameters. In: Fischer, G.H., Molenaar, I.W. (eds.) Rasch Models: Foundations, Recent Developments, and Applications, pp. 39–51. Springer, New York (1995)

    Chapter  Google Scholar 

  23. Smith, A.F.M., Roberts, G.O.: Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods (with discussion). J. R. Stat. Soc. B 55(1), 3–23 (1993)

    Google Scholar 

  24. Tierney, L.: Markov chains for exploring posterior distributions. Ann. Stat. 22(4), 1701–1728 (1994)

    MathSciNet  Google Scholar 

  25. Albert, J.H.: Bayesian estimation of normal ogive item response curves using Gibbs sampling. J. Educ. Stat. 17(3), 251–269 (1992)

    Article  Google Scholar 

  26. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984)

    Article  Google Scholar 

  27. Lord, F.M., Novick, M.R.: Statistical Theories of Mental Test Scores. Addison-Wesley, Boston (1968)

    Google Scholar 

  28. Sheng, Y., Headrick, T.C.: An algorithm for implementing Gibbs sampling for 2PNO IRT models. J. Mod. Appl. Stat. Methods 6(1), 341–349 (2007)

    Article  Google Scholar 

  29. Harwell, M., Stone, C.A., Hsu, H., Kirisci, L.: Monte Carlo studies in item response theory. Appl. Psychol. Meas. 20(2), 101–126 (1996)

    Article  Google Scholar 

  30. Foster, I.: Designing and Building Parallel Programs: Concepts and Tools for Parallel Software Engineering. Addison-Wesley, Boston (1995)

    Google Scholar 

  31. Pastias, K., Rahimi, M., Sheng, Y., Rahimi, S.: Parallel computing with a Bayesian item response model. Am. J. Comput. Math. 2(2), 65–71 (2012)

    Article  Google Scholar 

  32. Sheng, Y., Rahimi, M.: High performance Gibbs sampling for IRT models using row-wise decomposition. ISRN Comput. Math. 2012(264040), 1–9 (2012)

    Article  Google Scholar 

  33. Sheng, Y., Welling, W.S., Zhu, M.M.: A GPU-based Gibbs sampler for a unidimensional IRT model. ISRN Comput. Math. 2014(368149), 1–11 (2014)

    Google Scholar 

  34. Sheng, Y., Welling, W.S., Zhu, M.M.: GPU-accelerated computing with Gibbs sampler for the 2PNO IRT model. In: van der Ark, L.A., et al. (eds.) Quantitative Psychology Research, pp. 59–73. Springer, New York (2015)

    Chapter  Google Scholar 

  35. NVIDIA, Vingelmann, P., Fitzek, F.H.P.: CUDA, Release: 10.2.89 (2020) https://developer.nvidia.com/cuda-toolkit

  36. Kraus, J.: An introduction to CUDA-aware MPI. NVidia Developer (blog) (2013) https://developer.nvidia.com/blog/introduction-cuda-aware-mpi/

  37. Adinets, A.: CUDA dynamic parallelism API and principles. NVidia Developer (blog) (2018) https://developer.nvidia.com/blog/using-cuda-warp-level-primitives/

  38. CUDA-CUBLAS Library 11.7. NVIDIA Corp. (2022) https://docs.nvidia.com/cuda/cublas/

  39. Reckase, M.D.: The past and future of multidimensional item response theory. Appl. Psychol. Meas. 21(1), 25–36 (1997)

    Article  Google Scholar 

  40. Farahani, H. S., Fatehi, A., Sh, M. A.: Between-domain instance transition via the process of Gibbs sampling in RBM. arXiv, (2020). https://doi.org/10.48550/arxiv.2006.14538

  41. Papamarkou, T., Hinkle, J., Young, M. T., Womble, D.: Challenges in Markov chain Monte Carlo for Bayesian neural networks. arXiv, (2019). https://doi.org/10.48550/arxiv.1910.06539

  42. Canillas, R., Laurent, R., Faix, M., Vaufreydaz, D., Mazer, E.: Autonomous robot controller using bitwise Gibbs sampling. In: 2016 IEEE 15th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC), pp. 72–76 (2016). https://doi.org/10.1109/ICCI-CC.2016.7862096

Download references

Acknowledgements

We thank the University of Chicago’s Research Computing Center for their support of this work.

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Texas A &M University, College Station, TX, 77843, USA

    William S. Welling

  2. Committee on Quantitative Methods, University of Chicago, 1155 E. 60th Street, Chicago, IL, 60637, USA

    Yanyan Sheng

  3. Computer Science Department, Montclair State University, 1 Normal Ave., Montclair, NJ, 07043, USA

    Michelle M. Zhu

Authors
  1. William S. Welling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanyan Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michelle M. Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

WW and YS designed the study. WW developed and optimized the parallel programs and carried out the analyses. YS drafted the main manuscript text and prepared all figures. MZ reviewed the initial design and the developed parallel algorithms. All authors reviewed the manuscript.

Corresponding author

Correspondence to Yanyan Sheng.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1

Appendix 1

Figure 9 displays the execution time of CUDA-aware MPI with varying number of nodes and tasks to implement Gibbs sampling to tests with k (\(k=20, 50, 100, 200\)) items and 500 to 1,000,000 persons with different sizes, whereas Fig. 10 displays the speedup of CUDA-aware MPI over MPI under the same sample size, test length and node/task conditions. It is noted that MPI implementations with 1 node and 2 tasks failed to provide a result when sample size reaches 1,000,000, and hence the speedup values could not be obtained and hence plotted in Fig.  10.

Fig. 9
figure 9

Execution time for implementing the CUDA-aware MPI program of Gibbs sampling for tests and samples with different sizes by varying the number of nodes and tasks

Full size image
Fig. 10
figure 10

Speedup of CUDA-aware MPI relative to MPI for tests and samples with different sizes by varying the number of nodes and tasks

Full size image

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Welling, W.S., Sheng, Y. & Zhu, M.M. CUDA-aware MPI implementation of Gibbs sampling for an IRT model. Cluster Comput 27, 1821–1830 (2024). https://doi.org/10.1007/s10586-023-04049-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-023-04049-z

Keywords

Search

Navigation