Ruoming Pang
Vern Paxson
ABSTRACT
A key step in the semantic analysis of network traffic is to parse the traffic stream according to the high-level protocols it contains. This process transforms raw bytes into structured, typed, and semantically meaningful data fields that provide a high-level representation of the traffic. However, constructing protocol parsers by hand is a tedious and error-prone affair due to the complexity and sheer number of application protocols.This paper presents binpac, a declarative language and compiler designed to simplify the task of constructing robust and efficient semantic analyzers for complex network protocols. We discuss the design of the binpac language and a range of issues in generating efficient parsers from high-level specifications. We have used binpac to build several protocol parsers for the "Bro" network intrusion detection system, replacing some of its existing analyzers (handcrafted in C++), and supplementing its operation with analyzers for new protocols. We can then use Bro's powerful scripting language to express application-level analysis of network traffic in high-level terms that are both concise and expressive. binpac is now part of the open-source Bro distribution.
- M. B. Abbott and L. L. Peterson. A language-based approach to protocol implementation. IEEEACM Transactions on Networking, 1(1):4--19, 1993.]] Google Scholar
Digital Library
- M. Arlitt, B. Krishnamurthy, and J. C. Mogul. Predicting short-transfer latency from TCP arcana: A trace-based validation. In Proceedings of the Internet Measurement Conference (IMC), October 2005.]] Google ScholarDigital Library
- Abstract Syntax Notation One (ASN.1). ISOIEC 8824-1:2002.]]Google Scholar
- G. Back. Datascript - a specification and scripting language for binary data. In GPCE '02: The ACM SIGPLAN/SIGSOFT Conference on Generative Programming and Component Engineering, pages 66--77, London, UK, 2002. Springer-Verlag.]] Google ScholarDigital Library
- E. Biagioni, R. Harper, and P. Lee. A network protocol stack in Standard ML. Higher-Order and Symbolic Computation, 14(4):309--356, 2001.]] Google ScholarDigital Library
- T. P. Blumer and J. C. Burruss. Generating a service specification of a connection management protocol. In PSTV, pages 161--170, 1982.]] Google ScholarDigital Library
- N. Borisov, D. J. Brumley, H. J. Wang, J. Dunagan, P. Joshi, and C. Guo. Generic application-level protocol analyzer and its language. Under submission.]]Google Scholar
- Common Internet File System. http://www.snia.org/tech_activities/CIFS/CIFS-TR-1p00_FINAL.pdf.]]Google Scholar
- D. Crocker. RFC 2234: Augmented BNF for Syntax Specifications: ABNF.]] Google ScholarDigital Library
- DCE 1.1: Remote procedure call. http://www.opengroup.org/onlinepubs/9629399/toc.htm.]]Google Scholar
- DSniff. www.monkey.org/dugsong/dsniff.]]Google Scholar
- The Ethereal Network Analyzer. http://www.ethereal.com/.]]Google Scholar
- A. Feldmann, N. Kammenhuber, O. Maennel, B. Maggs, R. D. Prisco, and R. Sundaram. A Methodology for Estimating Interdomain Web Traffic Demand. In Proceedings of the Internet Measurement Conference (IMC), October 2004.]] Google ScholarDigital Library
- R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee. RFC 2616: Hypertext Transfer Protocol - HTTP1.1, June 1999.]] Google ScholarDigital Library
- K. Fisher and R. Gruber. PADS: A domain-specific language for processing ad hoc data. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 295--304, New York, NY, USA, 2005. ACM Press.]] Google ScholarDigital Library
- K. Fisher, Y. Mandelbaum, and D. Walker. The next 700 data description languages. In Proceedings of the 24th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL), pages 2--15, New York, NY, USA, 2006. ACM Press.]] Google ScholarDigital Library
- M. Handley, C. Kreibich, and V. Paxson. Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics. In Proceedings of USENIX Security Symposium, 2001.]] Google ScholarDigital Library
- G. J. Holzmann. The model checker SPIN. IEEE Trans. Softw. Eng., 23(5):279--295, 1997.]] Google ScholarDigital Library
- V. Jacobson, C. Leres, and S. McCanne. TCPDUMP. ftp://ftp.ee.lbl.gov/libpcap.tar.Z.]]Google Scholar
- S. C. Johnson. YACC - Yet Another Compiler-Compiler. Computer Science Technical Report No. 32, Bell Laboratories, Murray Hill, New Jersey, July 1975.]]Google Scholar
- J. Jung and E. Sit. An Empirical Study of Spam Traffic and the Use of DNS Black Lists. In Proceedings of the Internet Measurement Conference (IMC), Taormina, Italy, October 2004.]] Google ScholarDigital Library
- E. Kohler, M. F. Kaashoek, and D. R. Montgomery. A readable TCP in the Prolac protocol language. In Proceedings of the ACM SIGCOMM Conference, pages 3--13, Cambridge, MA, August 1999.]] Google ScholarDigital Library
- C. Kreibich. NetDude (NETwork DUmp data Displayer and Editor). http://netdude.sourceforge.net/.]]Google Scholar
- C. Kreibich. Design and implementation of netdude, a framework for packet trace manipulation. June 2004.]]Google Scholar
- A. Kumar, V. Paxson, and N. Weaver. Exploiting Underlying Structure for Detailed Reconstruction of an Internet-scale Event. In Proceedings of the Internet Measurement Conference (IMC), October 2005.]] Google ScholarDigital Library
- P. J. McCann and S. Chandra. Packet Types: Abstract specifications of network protocol messages. In Proceedings of the ACM SIGCOMM Conference, pages 321--333, 2000.]] Google ScholarDigital Library
- P. Mockapetris. DOMAIN NAMES - IMPLEMENTATION AND SPECIFICATION, Section 4.1.4, November 1987. RFC 1035.]] Google ScholarDigital Library
- NFR Security. http://www.nfr.com.]]Google Scholar
- R. Pang, V. Yegneswaran, P. Barford, V. Paxson, and L. Peterson. Characteristics of Internet Background Radiation. In Proceedings of the Internet Measurement Conference (IMC), October 2004.]] Google ScholarDigital Library
- T. Parr and R. Quong. ANTLR: A predicated-ll (k) parser generator. Software, Practice and Experience, 25, July 1995.]] Google ScholarDigital Library
- V. Paxson. BRO: A system for detecting network intruders in real time. In Proceedings of USENIX Security Symposium, San Antonio, TX, January 1998.]] Google ScholarDigital Library
- V. Paxson, K. Asanovic, S. Dharmapurikar, J. Lockwood, R. Pang, R. Sommer, and N. Weaver. Rethinking hardware support for network analysis and intrusion prevention. In Proceedings of Workshop on Hot Topics in Security (HotSec), Vancouver, B.C., Canada, July 2006.]] Google ScholarDigital Library
- NetWare Core Protocol. http://forge.novell.com/modules/xfmod/project?ncp.]]Google Scholar
- T. H. Ptacek and T. N. Newsham. Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection. Technical report, Secure Networks, Inc., January 1998.]]Google Scholar
- M. Roesch. SNORT: Lightweight intrusion detection for networks. In Proceedings of USENIX LISA, 1999.]] Google ScholarDigital Library
- The SNORT network intrusion detection system. http://www.snort.org.]]Google Scholar
- S. Saroiu, K. P. Gummadi, R. J. Dunn, S. D. Gribble, and H. M. Levy. An analysis of internet content delivery systems. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation (OSDI), December 2002.]] Google ScholarDigital Library
- C. Shannon and D. Moore. The Spread of the Witty Worm. http://www.caida.org/analysis/security/witty, 2004.]]Google Scholar
- R. Srinivasan. RFC 1831: RPC: Remote Procedure Call Protocol Specification.]] Google ScholarDigital Library
- R. Srinivasan. RFC 1832: XDR: External Data Representation Standard.]] Google ScholarDigital Library
- Ethereal OSPF Protocol Dissector Buffer Overflow Vulnerability. http://www.idefense.com/intelligence/vulnerabilities/display.php?id=349.]]Google Scholar
- Snort TCP Stream Reassembly Integer Overflow Exploit. http://www.securiteam.com/exploits/5BP0O209PS.html.]]Google Scholar
- Symantec Multiple Firewall NBNS Response Processing Stack Overflow. http://www.eeye.com/html/research/advisories/AD20040512A.html.]]Google Scholar
- tcpdump ISAKMP packet delete payload buffer overflow. http://xforce.iss.net/xforce/xfdb/15680.]]Google Scholar
- Separation of concerns. http://en.wikipedia.org/wiki/Separation_of_concerns.]]Google Scholar
- C. Wong, S. Bielski, J. M. McCune, and C. Wang. A study of mass-mailing worms. In Proceedings of the 2005 ACM Workshop on Rapid Malcode (WORM), pages 1--10, New York, NY, USA, 2004. ACM Press.]] Google ScholarDigital Library
Index Terms
- binpac: a yacc for writing application protocol parsers
Recommendations
A translational BNF grammar notation (TBNF)
BNF grammar notation came into existence about 1960 for the specification of programming languages. It was first used for the automatic generation of parsers about 1972. BNF was later replaced with EBNF offering regular expression notation in the right ...
Domain-specific language integration with compile-time parser generator library
GPCE '10: Proceedings of the ninth international conference on Generative programming and component engineeringSmooth integration of domain-specific languages into a general purpose host language requires absorbing of domain code written in arbitrary syntax. The integration should cause minimal syntactical and semantic overhead and introduce minimal dependency ...
Packrat parsers can handle practical grammars in mostly constant space
PASTE '10: Proceedings of the 9th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineeringPackrat parsing is a powerful parsing algorithm presented by Ford in 2002. Packrat parsers can handle complicated grammars and recursive structures in lexical elements more easily than the traditional LL(k) or LR(1) parsing algorithms. However, packrat ...
Comments