Abstract
The controller area network (CAN) protocol, used in many modern vehicles for real-time inter-device communications, is known to have cybersecurity vulnerabilities, putting passengers at risk for data exfiltration and control system sabotage. To address this issue, researchers have proposed to utilize security measures based on cryptography and message authentication; unfortunately, such approaches are often too computationally expensive to be deployed in real time on CAN devices. Additionally, they have developed machine learning (ML) techniques to detect anomalies in CAN traffic and thereby prevent attacks. The main disadvantage of existing ML-based techniques is that they either depend on additional computational hardware or they heuristically assume that all communication anomalies are malicious.
In this article, we show that tree-based learning ensembles outperform anomaly-based techniques like AutoRegressive Integrated Moving Average (ARIMA) and Z-Score when used to detect attacks that result in increased bus utilization. We evaluated the detection capacity of three tree-based ensembles, Adaboost, gradient boosting, and random forests, and collectively refer to these as DT-DS. We conclude that the decision tree ensemble with Adaboost performs best with an area under curve (AUC) score of 0.999, closely followed by gradient boosting and random forests with 0.997 and 0.991 AUC scores, respectively, when trained using message profiles. We observe that with an increase in the observation window, the DT-DS models present an average AUC score of 0.999, and offer a nearly perfect detection of attacks, at the cost of increased latency in detection of attacked messages. We evaluate the performance of the IDS for Aeronautical Radio, Incorporated– (ARINC) encoded CAN communication traffic in avionic systems, generated using an aerospace testbench, ARINC-825TBv2. The IDS has been evaluated against the active attacks of a state-of-the-art predictive attacker model. Additionally, we observed that the performance of IDS approaches such as ARIMA and Z-Score degrade considerably with a decrease in the size of the observation time window. In contrast, the performance of DT-DS models is consistent, with only an average drop of 0.005 in the AUC score.
- [1] ARINC Specification 825.
Standard . ARINC.Google Scholar - [2] FlightGear Flight Simulator. Retrieved from https://www.flightgear.org/about/.Google Scholar
- [3] 2018. White paper using CAN bus serial communications in space flight applications.Google Scholar
- [4] . 2011. Performance evaluation study of intrusion detection systems. Proc. Comput. Sci. 5 (2011), 173–180.
DOI: Google ScholarCross Ref - [5] . 2015. IntelliCAN: Attack-resilient controller area network (CAN) for secure automobiles. In Proceedings of the IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFTS’15). 233–236.
DOI: Google ScholarCross Ref - [6] . 2017. Linking received packet to the transmitter through physical-fingerprinting of controller area network. In Proceedings of the IEEE Workshop on Information Forensics and Security (WIFS’17). 1–6.
DOI: Google ScholarCross Ref - [7] . 2016. A simple intrusion detection method for controller area network. In Proceedings of the IEEE International Conference on Communications (ICC’16). 1–7.
DOI: Google ScholarCross Ref - [8] . 2020. Evaluation of CAN bus security challenges. Sensors 20 (
04 2020), 16–17.DOI: Google ScholarCross Ref - [9] . 2017. Security issues in controller area networks in automobiles. In Proceedings of the 18th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA). 93–98.
DOI: Google ScholarCross Ref - [10] . 2020. Cybersecurity of onboard charging systems for electric vehicles—Review, challenges and countermeasures. IEEE Access (
12 2020), 1–1.DOI: Google ScholarCross Ref - [11] . 2011. Comprehensive experimental analyses of automotive attack surfaces. In Proceedings of the 20th USENIX Conference on Security (SEC’11). USENIX Association, Berkeley, CA, 6.Google ScholarDigital Library
- [12] . 2017. A novel unsupervised anomaly detection approach for intrusion detection system. In Proceedings of the IEEE 3rd International Conference on Big Data Security on Cloud (Bigdatasecurity’17), IEEE International Conference on High Performance and Smart Computing (HPSC’17), and IEEE International Conference on Intelligent Data and Security (IDS’17). 69–73.
DOI: Google ScholarCross Ref - [13] . 2019. Basic Ensemble Learning (Random Forest, AdaBoost, Gradient Boosting): Step by Step Explained. Retrieved from https://towardsdatascience.com/basic-ensemble-learning-random-forest-adaboost-gradient-boosting-step-by-step-explained-95d49d1e2725.Google Scholar
- [14] . 2016. Error handling of in-vehicle networks makes them vulnerable. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’16). Association for Computing Machinery, New York, NY, 1044–1055.
DOI: Google ScholarDigital Library - [15] . 2018. VoltageIDS: Low-level communication characteristics for automotive intrusion detection system. IEEE Trans. Inf. Forens. Secur. 13, 8 (2018), 2114–2129.
DOI: Google ScholarCross Ref - [16] . 2016. Automobile driver fingerprinting. Proc. Priv. Enhanc. Technol. 2016 (
01 2016).DOI: Google ScholarCross Ref - [17] . 2015. FBI: Researcher Admitted to Hacking Plane In-flight, Causing it to “Climb.” Retrieved from https://arstechnica.com/information-technology/2015/05/fbi-researcher-admitted-to-hacking-plane-in-flight-causing-it-to-climb/.Google Scholar
- [18] . 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 1 (
August 1997), 119–139.DOI: Google ScholarDigital Library - [19] . 2016. An intrusion detection method for securing in-vehicle CAN bus. In Proceedings of the 17th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA’16). 176–180.
DOI: Google ScholarCross Ref - [20] . 2019. Efficient intrusion detection with bloom filtering in controller area networks. IEEE Trans. Inf. Forens. Secur. 14, 4 (2019), 1037–1051.
DOI: Google ScholarDigital Library - [21] . 2017. LiBrA-CAN: Lightweight broadcast authentication for controller area networks. ACM Trans. Embed. Comput. Syst. 16, 3, Article
90 (April 2017), 28 pages.DOI: Google ScholarDigital Library - [22] . 2018. A framework for aviation cybersecurity. In Proceedings of the IEEE National Aerospace and Electronics Conference (NAESON’18). 132–136.
DOI: Google ScholarCross Ref - [23] . 1999. CANopen application layer and communication profile.Google Scholar
- [24] CAN Bus Error Handling. Retrieved from https://www.kvaser.com/about-can/the-can-protocol/can-error-handling/.Google Scholar
- [25] An Overview of ARINC. Retrieved from https://www.kvaser.com/arinc/.Google Scholar
- [26] . Controller Area Network (CAN) Overview.
Technical Report .Google Scholar - [27] . 2017. An attack-resilient source authentication protocol in controller area network. In Proceedings of the Symposium on Architectures for Networking and Communications Systems (ANCS’17). IEEE Press, 109–118.
DOI: Google ScholarDigital Library - [28] . 2016. A novel intrusion detection method using deep neural network for in-vehicle network security. In Proceedings of the IEEE 83rd Vehicular Technology Conference (VTC Spring’16). 1–5.
DOI: Google ScholarCross Ref - [29] . 2019. Investigating CAN Bus Network Integrity in Avionics Systems. Retrieved from https://www.rapid7.com/research/report/investigating-can-bus-network-integrity-in-avionics-systems/.Google Scholar
- [30] . 2012. Standardization of CAN networks for airborne use through ARINC 825.Google Scholar
- [31] . 2010. Experimental security analysis of a modern automobile. In Proceedings of the IEEE Symposium on Security and Privacy (SP’10). IEEE, 447–462.Google ScholarDigital Library
- [32] . 2020. INDRA: Intrusion detection using recurrent autoencoders in automotive embedded systems.
arxiv:cs.CR/2007.08795 . Retrieved from https://arxiv.org/abs/2007.08795.Google Scholar - [33] . 2014. Evaluation metrics for intrusion detection systems—A study. Int. J. Comput. Sci. Mobile Appl. (2014).Google Scholar
- [34] . 2018. Supervised and unsupervised intrusion detection based on CAN message frequencies for in-vehicle network. J. Inf. Process. 26 (2018), 306–313.Google ScholarCross Ref
- [35] . 2017. A study on big data thinking of the internet of things-based smart-connected car in conjunction with controller area network bus and 4G-long term evolution. Symmetry 9, 8 (2017).
DOI: Google ScholarCross Ref - [36] . 2008. An approach to specification-based attack detection for in-vehicle networks. In Proceedings of the IEEE Intelligent Vehicles Symposium. 220–225.
DOI: Google ScholarCross Ref - [37] . 2012. Cyber-security for the controller area network (CAN) communication protocol. In Proceedings of the International Conference on Cyber Security. 1–7.
DOI: Google ScholarDigital Library - [38] . 2012/11. An algorithm for detection of malicious messages on CAN buses. In Proceedings of the National Conference on Information Technology and Computer Science. Atlantis Press, 627–630.
DOI: Google ScholarCross Ref - [39] . 2020. CANnolo: An anomaly detection system based on LSTM autoencoders for controller area network. IEEE Trans. Netw. Serv. Manage. (2020), 1–1.
DOI: Google ScholarCross Ref - [40] . 2016. Intrusion-damage assessment and mitigation in cyber-physical systems for control applications. In Proceedings of the 24th International Conference on Real-Time Networks and Systems (RTNS’16). 141–150.Google ScholarDigital Library
- [41] . 2019. Butterfly attack: Adversarial manipulation of temporal properties of cyber-physical systems. In Proceedings of the IEEE Real-Time Systems Symposium (RTSS’19).Google ScholarCross Ref
- [42] . 2017. Anomaly detection of CAN bus messages through analysis of ID sequences. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV’17). 1577–1583.
DOI: Google ScholarDigital Library - [43] . 2000. Testing intrusion detection systems: A critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory. ACM Trans. Inf. Syst. Secur. 3, 4 (
November 2000), 262–294.DOI: Google ScholarDigital Library - [44] . A Survey of Remote Automotive Attack Surfaces.
Technical Report .Google Scholar - [45] . 1998. Classification and regression trees software and new developments. In Advances in Data Science and Classification, , , and (Eds.). Springer, Berlin, 311–318.Google ScholarCross Ref
- [46] . 2017. Modeling inter-signal arrival times for accurate detection of CAN bus signal injection attacks: A data-driven approach to in-vehicle intrusion detection. In Proceedings of the 12th Annual Conference on Cyber and Information Security Research (CISRC’17). Association for Computing Machinery, New York, NY, Article
11 , 4 pages.DOI: Google ScholarDigital Library - [47] . 2016. Practical DoS attacks on embedded networks in commercial vehicles. In Information Systems Security, , , , , and (Eds.). Springer International Publishing, Cham, 23–42.Google Scholar
- [48] . 2011. Entropy-based anomaly detection for in-vehicle networks. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV’11). 1110–1115.
DOI: Google ScholarCross Ref - [49] . 2014. Effective approach toward intrusion detection system using data mining techniques. Egypt. Inf. J. 15, 1 (2014), 37–50.
DOI: Google ScholarCross Ref - [50] . 2008. Efficient in-vehicle delayed data authentication based on compound message authentication codes. In Proceedings of the IEEE 68th Vehicular Technology Conference. 1–5.
DOI: Google ScholarCross Ref - [51] . 2011. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12 (2011), 2825–2830.Google ScholarDigital Library
- [52] . 2001. SPINS: Security protocols for sensor networks. In Proceedings of the 7th Annual International Conference on Mobile Computing and Networking (MobiCom’01). Association for Computing Machinery, New York, NY, 189–199.
DOI: Google ScholarDigital Library - [53] . 2008. Evaluation: From precision, recall and F-factor to ROC, informedness, markedness & correlation. Mach. Learn. Technol. 2 (
01 2008).Google Scholar - [54] . 2014. Schedule integration framework for time-triggered automotive architectures. In Proceedings of the 51st ACM/EDAC/IEEE Design Automation Conference (DAC’14).Google ScholarDigital Library
- [55] . 2020. Machine learning applications in misuse and anomaly detection. In Security and Privacy From a Legal, Ethical, and Technical Perspective. IntechOpen.
DOI: Google ScholarCross Ref - [56] . 2013. Non-invasive spoofing attacks for anti-lock braking systems. In Cryptographic Hardware and Embedded Systems.
Lecture Notes in Computer Science , Vol. 8086. Springer, Berlin.Google Scholar - [57] . 2017–. pmdarima: ARIMA estimators for Python. Retrieved from http://www.alkaline-ml.com/pmdarima.Google Scholar
- [58] . 2016. Intrusion detection system based on the analysis of time intervals of CAN messages for in-vehicle network. In Proceedings of the International Conference on Information Networking (ICOIN’16). 63–68.
DOI: Google ScholarDigital Library - [59] . 2013. How to Hack Your Mini Cooper: Reverse Engineering Controller Area Network CAN Messages on Passenger Automobiles. Retrieved May 13, 2021 from https://doi.org/10.5446/38934Google Scholar
- [60] . 2013. Survey on security threats and protection mechanisms in embedded automotive networks. In Proceedings of the 43rd Annual IEEE/IFIP Conference on Dependable Systems and Networks Workshop (DSN-W’13). 1–12.
DOI: Google ScholarCross Ref - [61] . CANaerospace. Retrieved from https://www.stockflightsystems.com/canaerospace.html.Google Scholar
- [62] . 2015. Frequency-based anomaly detection for the automotive CAN bus. In Proceedings of the World Congress on Industrial Control Systems Security (WCICSS’15). 45–49.
DOI: Google ScholarCross Ref - [63] 2016. Introduction to the Controller Area Network(CAN). Retrieved from https://www.ti.com/lit/an/sloa101b/sloa101b.pdf.Google Scholar
- [64] . 2018. Detection of automotive CAN cyber-attacks by identifying packet timing anomalies in time windows. In Proceedings of the 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W’18). 231–238.
DOI: Google ScholarCross Ref - [65] . 2019. Automotive intrusion detection based on constant CAN message frequencies across vehicle driving modes. In Proceedings of the ACM Workshop on Automotive Cybersecurity (AutoSec’19). Association for Computing Machinery, New York, NY, 9–14.
DOI: Google ScholarDigital Library - [66] . 2019. Survey of automotive controller area network intrusion detection systems. IEEE Des. Test 36, 6 (2019), 48–55.
DOI: Google ScholarCross Ref - [67] . 2019. ARINC-825TBv2: A hardware-in-the-ioop simulation platform for aerospace security research. In Proceedings of the 30th International Workshop on Rapid System Prototyping (RSP’19). Association for Computing Machinery, New York, NY, 29–35.
DOI: Google ScholarDigital Library - [68] . 2006. Anomaly based network intrusion detection with unsupervised outlier detection. In Proceedings of the IEEE International Conference on Communications, Vol. 5.
DOI: Google ScholarCross Ref
Index Terms
- DT-DS: CAN Intrusion Detection with Decision Tree Ensembles
Recommendations
An Adaptive Rule-Based Intrusion Alert Correlation Detection Method
ICNDC '10: Proceedings of the 2010 First International Conference on Networking and Distributed ComputingIntrusion detection system (IDS) is a security layer that is used to discover ongoing intrusive attacks and anomaly activities in information systems and is usually working in a dynamically changing environment. Although increasing IDSs are developed in ...
Optimal thresholds for intrusion detection systems
HotSos '16: Proceedings of the Symposium and Bootcamp on the Science of SecurityIn recent years, we have seen a number of successful attacks against high-profile targets, some of which have even caused severe physical damage. These examples have shown us that resourceful and determined attackers can penetrate virtually any system, ...
Enhancing Intrusion Detection System with proximity information
Intrusion Detection Systems (IDSes) proposed to identify or prevent the wide spread of worms can be largely classified as signature-based or anomaly-based. Modern worms are often sufficiently intelligent to hide their activities and evade anomaly ...
Comments