Real-time Streaming Analytics
Fast Data Bridges Operational Work and the Data Pipelin
While the big data portion of the enterprise data architecture is well designed for storing and analyzing massive amounts of historical data at rest, the architecture of the fast data portion is equally critical to the data pipeline.
There is good evidence, much of it evident in the EMC/IDC report’s analysis of the growth in mobile, sensors, and IoT, that all serious data growth in the future will come from fast in-memory database data. Fast data comes into data systems in streams; they are fire hoses. These streams look like observations, log records, interactions, sensor readings, clicks, game play, and so forth: things happening hundreds to millions of times a second.
Clarity is growing that at the core of the big data side of the architecture is a fully distributed file system (HDFS or another FS) that will provide a central, commoditized repository for data at rest within the enterprise.
This market is taking shape today, with relevant vendors taking their places within this architecture. Fast data is going through a more fundamental and immediate shift. Understanding the opportunity and potential disruption to the status quo is beginning in earnest. Fast data is where many of the truly revolutionary advances will be made.
