The History of Complex Event Processing

Academic Roots

Modern event processing grew its roots in the mid 1990’s, when academic research began at Cal Tech (Mani Chandy), Cambridge University (John Bates), and Stanford University (David Luckham).  The focus of this research was to develop a new approach to process streaming event data by identifying complex sequences of events (event A followed by B, then C), with temporal constraints (within 20 minutes), spatial constraints (within 5 miles of each other), and to control complex actions as a result of these patterns (begin warning an operator at code yellow until another event, D, is detected within 5 seconds).  An example of a CEP logic, written in a high-level representation of Apama's CEP programming language is shown at left.

Commercial Software Vendors

Commercial software companies were created based on this research.  Chandy’s work spawned the iSpheres (acquired by Avaya last year), and Bates co-founded Apama with Giles Nelson in 1999.  In 2002, Luckham published a book called: "The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems" that helped popularize the term “complex event processing,” or CEP.  In 2003, two more vendors were founded - Coral8 (Stanford) and StreamBase (MIT / Brown / Brandeis; Michael Stonebraker / Stan Zdonick / Mitch Cherniack).  TIBCO BusinessEvents entered the market in 2004 as well.  Other CEP firms include Kaskad, Aptsoft, and Aleri Labs.

Application Use Cases

Recently CEP has experienced rapid growth and adoption, and has publically recognized in a variety of industries with documented use cases.  Capital markets was the first market to heavily utilize CEP it forms the basis of algorithmic trading (read this tutorial on algorithmic trading) applications in top firms like Deutsche Bank, ABN Amro, and JP Morgan (read this tutorial on CEP and algorithmic trading in Doctor Dobb's).  Aleri Labs has deployed algorithmic FX pricing engines at Commerzbank.  Apama's CEP engine has been customized as an algorithmic trading platform through the addition of market connectivity, algorithms, and pre-packaged user interfaces that implement user interfaces such as algorithmic order management, market aggregation, and trading surveillance (details here).  Other uses have published from other applications in the capital markets, such as exchange surveillance (e.g., Kaskad's at the Boston Stock Exchange), and in smart order routing (SRO), market making, MiFID compliance (Microsoft and Apama and Aleri Labs), bond pricing, and more. Outside of capital markets, CEP has been applied to a wide range of other applications.  Another significant area of adoption for CEP in in the travel industry, where TIBCO's BusinessEvents was selected for Airlines Operations Monitoring by Air France / KLM, and many other airlines have announced projects to automate operations.  TIBCO has announced the use of their CEP product in telecommunications, used by Telecom Italia for their entertainment-on-demand portal.  Coral8 is being used for patient care in the health care industry,  StreamBase is being used in the gaming industry, and Apama is used to manage real-time supply chain and retail operations in a deployed item-level RFID tagging systems like BGN in Holland.

The Event Processing Market

As a result, the event processing market is now considered an independent software market, and includes commercial software for complex event processing (CEP), business activity monitoring, and event data management (aka data stream management). Gartner announced its inaugural event processing industry event which will cover event processing and BAM, firms like AITE group and Bloor have published reports on the event processing industry and software vendors and applications. 
 

So CEP is now being labeled as a new paradigm of computing as being compared to business intelligence in terms of potential impact on the market as a way to monitor, analyze, and act on events of any type, as a way to add intelligence and action to events in any kind of application. 

Analyst Coverage on Event Processing

Gartner, Forrester, Bloor, and IDC all cover the event processing market in depth.  In the capital markets, the leading industry analyst covering event processing is the AITE group, who issued their first report on event processing in 2006.

Blogs and Community Site

The most heavily trafficked blogs on event processing include The Event Processing Blog and complexevents.com (David Luckham's site and blog)

Hundreds of technologies, analysts, and customers of CEP software vendors have joined the CEP-interest eGroup.  This group is open to anyone for enrollment and discussion of any event processing issues.