Wednesday, September 4, 2013

The Future of Big Data is Cognitive Big Data Apps

Volume, Velocity, Variety and Veracity of your data, the 4V challenge, has become untamable.  Wait, yet another big data blog?  No, not really.  In this blog, I would like to propose a cognitive app approach that can transform your big-data problems into big opportunities at a fraction of the cost.

Everyone is talking about big data problems but not many are helping us in understanding big data opportunities.  Let's define a big data opportunity in the context of customers because growing customer base, customer satisfaction and customer loyalty is everyone’s business:

  • you have a large, diverse and growing customer base
  • your customers are more mobile and social than ever before
  • you have engaged with your customers where ever they are: web, mobile, social, local
  • you believe that "more data beats better algorithms" and that big data is all data
  • you wish to collect all data - call center records, web logs, social media, customer transactions and more so that
  • you can understand your customers better and how they speak of and rank you in their social networks
  • you can group (segment) your customers to understand their likes and dislikes
  • you can offer (recommend) them the right products at the right time and at the right price
  • you can preempt customer backlash and prevent them for leaving (churn) to competitors and taking their social network with them (negative network effects)
  • all this effort will allow you to forecast sales accurately, run targeted marketing campaigns and cut cost to improve revenues and profitability
  • you wish to do all of this without hiring an army of data analysts, consultants and data scientists
  • and without buying half-dozen or more tools, getting access to several public / social data sets and integrating it all in your architecture
  • and above all, you wish to do it fast and drive changes in real time
  • And most importantly, you wish to rinse and repeat this approach for the foreseeable future
There are hardly any enterprise solutions in the market that can address the challenges listed above.  You have no other choice but to build a custom solution by hiring several consultants and striking separate licenses agreements with public and social data vendors to get a combined lens on public and private data.  This approach will be cost prohibitive for most enterprise customers and as "90% of the IT projects go" will be mired with delays, cost overruns and truck load of heartache. 

The advances in technologies like in-memory databases and graph structures as well as democratization of data science concepts can help in addressing the challenges listed above in a meaningful and cost-effective way.  Intelligent big data apps are the need of the hour.  These apps need to be designed and built from scratch keeping the challenges and technologies such as cognitive computing[1] in mind.  These apps will leave the technology paradigms of 1990s like "data needs to be gathered and modeled (caged) before an app is built" in the dumpster and will achieve the flexibility required from all modern apps to adapt as the underlying data structures and data sources change.  These apps can be deployed right off the shelf with minimum customization and consulting because the app logic will not be anchored to the underlying data-schema and will evolve with changing data and behavior.

The enterprise customers will soon be asking for a suite of such cognitive big data apps for all domain functions so that they can put the big data opportunities to work to run their businesses better than their competitors.  Without dynamic cognitive approach in apps, addressing the 4V challenge will be a nightmare and big data will fail to deliver its promise.

Stay tuned for future blogs on this topic including discussions on a pioneering technology approach.

[1] Cognitive computing is the ability to analyze oceans of data in context with related information and expertise.  Cognitive systems learn from how they’re used and adjust their rules and results dynamically.  Google search engine and knowledge graph technology is predicated upon this approach.  

 This blog has benefited from the infinite wisdom and hard work of my former colleagues Ryan Leask and Harish Butani and that of my current colleagues Sethu M., Jens Doerpmund and Vijay Vijayasankar.

Image courtesy of  MemeGenerator


  1. Thanks a lot for sharing this with all folks you really recognise what you are talking about! In this complex environment business need to present there company data in meaningful way.Sqiar ( which is in UK,provide services like Tableau and Data Warehousing etc .In these services sqiar experts convert company data into meaningful way.

  2. Wonderful blog & good post.Its really helpful for me, awaiting for more new post. Keep Blogging!

    Hadoop Course in Chennai

  3. IPL T20 news are updated regularly in

  4. 4 Ways Financial Institutions Can Bank On Big Data In 2015. Big data is now taking the guesswork out of discerning which individuals are the best targets for a particular product. TO know more, visit big data training in chennai

  5. In coming years, cloud computing is going to rule the world. The cloud based CRM tool provider like Salesforce have massive demand in the market. Thus talking salesforce training in Chennai from reputed Salesforce training institutes in Chennai will ensure bright career prospects for aspiring professionals.


  6. Thanks for sharing, I will bookmark and be back again

    Hadoop Certification in Chennai

  7. I just want to say I’m new to blogs and seriously loved this web site. Probably I’m likely to bookmark your blog post . You really have great stories. Regards for revealing your web site.

    Linux Training in Chennai Velachery


  8. This blog is really awesome thus i need more number of information and it is very useful and very clear to understand.

    Java-training institute in Chennai


  9. This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.

    Online Reputation Management

  10. I have been following you for a couple of months now but this is my first time commenting on a blog post. Thank you for sharing your knowledge and experience with us. Keep up the good work. Already bookmarked for future reference.
    Hadoop Training in Chennai
    Hadoop Training
    Best Hadoop Training in Chennai
    Best Hadoop Training Institute in Chennai

  11. qtp training in chennai
    Wow! That's really great information guys.I know lot of new things here. Really great contribution.Thank you ...

  12. This post is really nice and informative. The explanation given is really comprehensive and informative.. android Training in Chennai

  13. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

    AWS Training in Chennai

  14. Great post! I am see the programming coding and step by step execute the outputs.I am gather this coding more information. It's helpful for me my friend. Also great blog here with all of the valuable information you have.
    Seo Company in India

  15. Tremendous post building article. I read this article Big data programming concepts and I'll execute the coding step by steps for the outputs.I learn for much more new information for big data.I suggested to my friend also.
    We have to learn for Seo Related to click Seo Training in Chennai
    Seo Training Institute


  16. This blog explains the details about changing the ways of doing that business. That is understand well and doing some different process. Provides he best output of others. Thanks for this blog.

    iOS App Development Company

  17. Your thinking toward the respective issue is awesome also the idea behind the blog is very interesting which would bring a new evolution in respective field. Keep update more information.
    Interior Designers in Chennai


  18. Wonderful blog.. Thanks for sharing informative Post. Its very useful to me.

    Installment loans
    Payday loans
    Title loans

  19. Nice it seems to be good post... It will get readers engagement on the article since readers engagement plays an vital role in every blog.i am expecting more updated posts from your hands.
    Android App Development Company

  20. These ways are very simple and very much useful, as a beginner level these helped me a lot thanks fore sharing these kinds of useful and knowledgeable information.
    iOS App Development Company
    iOS App Development Company

  21. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing..
    Fitness SMS
    Fitness Text
    Salon SMS
    Salon Text
    Investor Relation SMS
    Investor Relation Text
    Mobile Marketing Services
    mobile marketing companies
    Sms API

  22. This is excellent information. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Mobile Marketing Service
    Mobile Marketing Companies
    Sms API
    Texting API

  23. This article is very much helpful and i hope this will be an useful information for the needed one. Keep on updating these kinds of informative things...
    Texting API
    Text message marketing
    Digital Mobile Marketing
    Mobile Marketing Services
    Mobile marketing companies
    Fitness SMS

  24. Everybody is talking about microservices. Industry veterans may remember monolithic or SOA-based solutions being the way of doing things. Times have changed. New tools have allowed developers to focus on specific problems without adding excessive complexity to deployment or other administrative tasks that are usually associated with isolated services.
    Microservices training in chennai

  25. The Spring Framework is a lightweight framework for developing Java enterprise applications. It provides high performing, easily testable and reusable code. Spring handles the infrastructure as the underlying framework so that you can focus on your application.Spring is modular in design, thereby making creation, handling and linking of individual components so much easier. Spring implements Model View Container(MVC) design pattern.
    spring mvc validation example

  26. Content is very clear and easily explained it is very interesting aswell. I hope you postagain soon.

    Python Training in Chennai

  27. It has been just unfathomably liberal with you to give straightforwardly what precisely numerous people would've promoted for an eBook to wind up making some money for their end, basically given that you could have attempted it in the occasion you needed.

    Selenium Training in Chennai

  28. is an online platform that allows people to find offers and the greatest deals in and around OMR. It is the directory of the business in and around omr. It helps internet users to surf the required deals and offers.

    Offers in Chennai

  29. Thanks for appreciating. Really means and inspires a lot to hear from you guys.I have bookmarked it and I am looking forward to reading new articles. Keep up the good work..Believe me, This is very helpful for me.
    Fleet Management Software

  30. Its really an Excellent post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog. Thanks for sharing....

    Restaurant in OMR
    Apartments in OMR
    Villas in OMR
    Resorts in OMR

  31. I do agree with all the ideas you have presented in your post. They’re really convincing and will certainly work. Still, the posts are very short for newbies. Could you please extend them a little from next time?..Keep this great work..
    Journal Of Computer Science
    Journal Of Management
    Plagiarism service
    Engineering Journal
    Journal Of Chemistry