Sanjit A. Seshia
Skip Abstract Section
Abstract
Making AI more trustworthy with a formal methods-based approach to AI system verification and validation.
- Censi, A., Slutsky, K., Wongpiromsarn, T., Yershov, D., Pendleton, S., Fu, J., and Frazzoli, E. Liability, ethics, and culture-aware behavior specification using rulebooks. In 2019 Intern. Conf. on Robotics and Automation (ICRA), IEEE, 8536--8542.Google Scholar
- Cohen, J.M., Rosenfeld, E., and Kolter, J.Z. Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th Intern. Conf. on Machine Learning 97 (2019), 1310--1320.Google Scholar
- Desai, A., Ghosh, S., Seshia, S.A., Shankar, N., and Tiwari, A. SOTER: A runtime assurance framework for programming safe robotics systems. In IEEE/IFIP Intern. Conf. on Dependable Systems and Networks (2019), 138--150.Google ScholarCross Ref
- Dietterich, T.G. and Horvitz, E.J. Rise of concerns about AI: Reflections and directions. Communications of the ACM 58, 10 (2015), 38--40.Google ScholarDigital Library
- Dreossi, T., Donze, A., and Seshia, S.A. Compositional falsification of cyber-physical systems with machine learning components. In Proceedings of the NASA Formal Methods Conf. (Lecture Notes in Computer Science) 10227 (2017), 357--372.Google Scholar
- Dreossi, T., Fremont, D.J., Ghosh, S., Kim, E., Ravanbakhsh, H., Vazquez-Chanlatte, M., and Seshia, S.A. VerifAI: A toolkit for the formal design and analysis of artificial intelligence-based systems. In 31st Intern. Conf. on Computer Aided Verification (Lecture Notes in Computer Science) 11561 (2017), 432--442.Google Scholar
- Dreossi, T., Jha, S., and Seshia, S.A. Semantic adversarial deep learning. In 30th Intern. Conf. on Computer Aided Verification (Lecture Notes in Computer Science) 10981 (2018), 3--26.Google Scholar
- Fisac, J.F. et al. A general safety framework for learning-based control in uncertain robotic systems. IEEE Transactions on Automatic Control 64, 7 (2018), 2737--2752.Google Scholar
- Fremont, D.J., Donzé, A., Seshia, S.A., and Wessel, D. Control improvisation. In 35th IARCS Annual Conf. on Foundations of Software Technology and Theoretical Computer Science (2015), 463--474.Google Scholar
- Fremont, D.J., Dreossi, T., Ghosh, S., Yue, X., Sangiovanni-Vincentelli, A.L., and Seshia, S.A. Scenic: A language for scenario specification and scene generation. In Proceedings of the 40th annual ACM SIGPLAN Conf. on Programming Language Design and Implementation (2019).Google ScholarDigital Library
- Garcia, J. and Fernández, F. A comprehensive survey on safe reinforcement learning. J. of Machine Learning Research 16, 1 (2015), 1437--1480.Google Scholar
- Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., and Vechev, M. AI2: Safety and robustness certification of neural networks with abstract interpretation. In IEEE Symposium on Security and Privacy (2018), 3--18.Google ScholarCross Ref
- Goodfellow, I., McDaniel, P., and Papernot, N. Making machine learning robust against adversarial inputs. Communications of the ACM 61, 7 (2018), 56--66.Google ScholarDigital Library
- Huang, X., Kwiatkowska, M., Wang, S., and Wu, M. Safety verification of deep neural networks. In Intern. Conf. on Computer Aided Verification, Springer, (2017), 3--29.Google ScholarCross Ref
- Icarte, R.T., Klassen, T., Valenzano, R., and McIlraith, S. Using reward machines for high-level task specification and decomposition in reinforcement learning. Intern. Conf. on Machine Learning (2018), 2107--2116.Google Scholar
- Jha, S., Sahai, T., Raman, V., Pinto, A., and Francis, M. Explaining AI decisions using efficient methods for learning sparse Boolean formulae. J. of Automated Reasoning 63, 4 (2019), 1055--1075.Google ScholarDigital Library
- Li, W., Sadigh, D., Sastry, S.S., and Seshia, S.A. Synthesis for human-in-the-loop control systems. In Proceedings of the 20th Intern. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (2014), 470--484.Google ScholarCross Ref
- Liu, C. et al. Algorithms for verifying deep neural networks. Foundations and Trends in Optimization 4, 3--4 (2021), 244--404.Google ScholarDigital Library
- Meel, K.S. et al. Constrained sampling and counting: Universal hashing meets SAT solving. In Beyond NP, Papers from the 2016 AAAI Workshop, (Feb. 12, 2016).Google Scholar
- Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D.L., and Kolobov, A. BLOG: Probabilistic models with unknown objects. Statistical Relational Learning (2007), 373.Google Scholar
- Mitchell, T.M., Keller, R.M., and Kedar-Cabelli, S.T. Explanation-based generalization: A unifying view. Machine Learning 1, 1 (1986), 47--80.Google ScholarCross Ref
- Nghiem, T. et al. Monte-Carlo techniques for falsification of temporal properties of non-linear hybrid systems. In Proceedings of the 13th ACM Intern. Conf. on Hybrid Systems: Computation and Control (2010), 211--220.Google ScholarDigital Library
- Nuzzo, P., Li, J., Sangiovanni-Vincentelli, A.L., Xi, Y., and Li, D. Stochastic assume-guarantee contracts for cyber-physical system design. ACM Transactions on Embedded Computing Systems 18, 1, (Jan. 2019).Google ScholarDigital Library
- Pearl, J. The seven tools of causal inference, with reflections on machine learning. Communications of the ACM 62, 3 (2019), 54--60.Google ScholarDigital Library
- Puggelli, A., Li, W., Sangiovanni-Vincentelli, A.L., and Seshia, S.A. Polynomial-time verification of PCTL properties of MDPs with convex uncertainties. In Proceedings of the 25th Intern. Conf. on Computer-Aided Verification (2013).Google ScholarCross Ref
- Russell, S.J. and Norvig, P. Artificial Intelligence: A Modern Approach. Prentice Hall (2010).Google ScholarDigital Library
- Sadigh, D. et al. Data-driven probabilistic modeling and verification of human driver behavior. In Formal Verification and Modeling in Human-Machine Systems, AAAI Spring Symposium (2014).Google Scholar
- Sadigh, D., Sastry, S., Seshia, S.A., and Dragan, A.D. Information gathering actions over human internal state. In Proceedings of the IEEE/RSJ Intern. Conf. on Intelligent Robots and Systems (2016).Google ScholarDigital Library
- Selsam, D., Liang, P., and Dill, D.L. Developing bug-free machine learning systems with formal mathematics. In Proceedings of the 34th Intern. Conf. on Machine Learning 70 (2017), 3047--3056.Google ScholarDigital Library
- Seshia, S.A. Combining induction, deduction, and structure for verification and synthesis. In Proceedings of the IEEE 103, 11 (2015), 2036--2051.Google ScholarCross Ref
- Seshia, S.A. Introspective environment modeling. 19th Intern. Conf. on Runtime Verification (2019), 15--26.Google ScholarDigital Library
- Seshia, S.A. et al. Formal specification for deep neural networks. In Proceedings of the Intern. Symp. on Automated Technology for Verification and Analysis (2018), 20--34.Google ScholarCross Ref
- Sha, L. Using simplicity to control complexity. IEEE Software 18, 4 (2001), 20--28.Google Scholar
- Shoukry, Y., et al. SMC: Satisfiability modulo convex optimization. In Proceedings of the 10th Intern. Conf. on Hybrid Systems: Computation and Control (2017).Google Scholar
- Tomlin, C., Mitchell, I., Bayen, A.M., and Oishi, M. Computational techniques for the verification of hybrid systems. In Proceedings of the IEEE 91, 7 (2003), 986--1001.Google ScholarCross Ref
- Tran, H-D., Xiang, W., and Johnson, T.T. Verification approaches for learning-enabled autonomous cyber-physical systems. IEEE Design & Test (Aug. 2020). Google ScholarCross Ref
- Vazquez-Chanlatte, M., Jha, S., Tiwari, A., Ho, M.K., and Seshia, S.A. Learning task Specifications from demonstrations. In Advances in Neural Information Processing Systems 31 (2018), 5372--5382.Google Scholar
- Wing, J.M. A specifier's introduction to formal methods. IEEE Computer 23, 9 (Sept. 1990), 8--24.Google ScholarDigital Library
- Wing, J.M. Trustworthy AI. Communications of the ACM 64, 10 (2021), 64--71.Google ScholarDigital Library
- Xu, J., Zhang, Z., Friedman, T., Liang, Y., and Van den Broeck, G. A semantic loss function for deep learning with symbolic knowledge. In Proceedings of the 35th Intern. Conf. on Machine Learning (2018), 5498--5507.Google Scholar
Index Terms
- Toward verified artificial intelligence
Recommendations
Artificial intelligence
AbstractArtificial intelligence (AI) is the Science and Engineering domain concerned with the theory and practice of developing systems that exhibit the characteristics we associate with intelligence in human behavior. Starting with a brief history of ...
Comments