There has been quite a bit of discussion lately around inclusion of Big Data — inclusion of more and more data into analysis driving business decision making to improve business productivity. For example, a McKinsey & Company study concluded that creative and effective use of big data could create $300BN in value annually for US Healthcare. A few examples are outlined in a recent Gigaom report, my favorite of which is the Zero Intelligence Project for analyzing progress in the Afghanistan conflict.
While the idea of Big Data is not exactly new, the technology is certainly evolving to a new frontier. With lowering costs of computing, (e.g., cloud based computing and storage) companies are saving and storing more information. With mash ups and improvements in search, it is easier to have several data sources working with one another vs. one source.
Two areas to bring Big Data forward are enterprise search (e.g., Microsoft’s Project Barcelona, Apache Solr) and cloud based data sources for information (e.g., Infochimps, Microsoft Azure Datamarketplace)