The Future of Data Warehouses in the Age of Big Data Many companies are saddled with data warehouses that weren’t designed to handle big data, but they can evolve their data warehouses into “analytics warehouses” capable of processing structured and unstructured data. The disappointing ROI largely stems from an inherent inadequacy in data warehouses: They were designed to handle the kind of structured data stored in ERP systems, not the unstructured data from social media, mobile devices, Web traffic, and other sources now streaming into enterprises. Introducing the Analytics Warehouse Fundamentally, the analytics warehouse functions as a central repository for an enterprise’s structured and unstructured data. In a traditional data warehousing architecture, structured data from ERP systems, CRM systems, file shares, and line of business applications is batch processed into the enterprise data warehouse using ETL (extract, transform, load) database processes. Software for running ad hoc queries and business intelligence systems take data from the warehouse environment, which may include operational data stores and data marts, to generate reports for users. The architecture for the analytics warehouse builds on the traditional data warehouse architecture in three primary ways: 1. A distributed file system (like Hadoop) sits between source data systems and the data warehouse. It collects, aggregates, and processes huge volumes of unstructured data, and stages it for loading into the data warehouse. 2. Structured and unstructured data from back end systems can be brought into the data warehouse in real- and near-real time. 3. Engines that use statistical and predictive modeling techniques to perform data discovery, visualization, inductive and deductive reasoning, and real-time decision-making reside between the data warehouse and end users. These engines identify patterns in big data. They can also complement and feed traditional ad hoc querying tools and business intelligence applications. Companies interested in building out their traditional data warehouse infrastructures may consider starting with reporting, if they don’t already have reporting capabilities in place, Then, they can begin integrating analytics technologies to their reporting framework. When companies start bringing this data together and federating it inside a data warehouse, the total cost of ownership for the data warehouse may begin to go down while the ROI goes up, The ability to integrate big data technologies, analytics technologies, back office systems, and traditional data warehouses has the potential to fundamentally change the economics of data warehousing for the better.
Posted on: Wed, 13 Nov 2013 16:18:52 +0000
Trending Topics
Recently Viewed Topics
© 2015