{"id":9360,"date":"2015-07-21T10:37:53","date_gmt":"2015-07-21T15:37:53","guid":{"rendered":"http:\/\/www.mrc-productivity.com\/blog\/?p=9360"},"modified":"2022-11-22T16:03:07","modified_gmt":"2022-11-22T22:03:07","slug":"5-hurdles-to-hadoop-adoption-and-how-to-fix-them","status":"publish","type":"post","link":"https:\/\/www.mrc-productivity.com\/blog\/2015\/07\/5-hurdles-to-hadoop-adoption-and-how-to-fix-them\/","title":{"rendered":"5 hurdles to Hadoop adoption (and how to fix them)"},"content":{"rendered":"

\"Education\"Summary: Hadoop has emerged as somewhat of a “poster child” for the Big Data movement. Its ability to store and process massive amounts of data on commodity hardware has caught the eye of many businesses. But, while Hadoop holds massive potential for your business, it’s not without challenges. If you plan on adopting Hadoop in the near future, here are some hurdles you must address. <\/em>
<\/span>
\n<\/a>
\n\"hadoop-logo\"As data volumes rise, we\u2019re seeing more businesses gravitate towards Hadoop. An open-source software framework, Hadoop helps businesses store and process massive amounts of data without purchasing expensive hardware. <\/p>\n

How are businesses using Hadoop? In all sorts of ways. I\u2019ve seen examples of businesses using Hadoop to find ideal prospects, prevent hardware failure, identify warning signs of security breaches, and so much more.<\/p>\n

The fact is, this data explosion offers a huge opportunity. But, businesses can only use it as a competitive advantage if they can somehow capture and store this data. Since traditional databases aren\u2019t built for \u201cBig Data\u201d, Hadoop provides the best means of accomplishing this goal.<\/p>\n

But, while Hadoop offers numerous advantages, it comes with its fair share of challenges and hurdles. If your business plans on adopting Hadoop, you must first understand these challenges, and how to address each one. What are they? Here are the 5 biggest hurdles to Hadoop adoption:<\/p>\n

1. Undefined value proposition<\/h3>\n

One of the biggest hurdles to Hadoop adoption has nothing to do with Hadoop from a technical standpoint. Business leaders aren\u2019t clear on the value. Why should they devote time and resources to a project, if they don\u2019t understand the payback? <\/p>\n

A recent Gartner survey<\/span><\/a> highlights this fact. Nearly half of the respondents claimed they weren\u2019t adopting Hadoop because they weren\u2019t sure how it would provide them with value.<\/p>\n

What can you do about this? What value does Hadoop offer, and how can you communicate this value to business leaders?<\/p>\n

One of the biggest reasons for this boils down to a simple fact: Many businesses don\u2019t believe they have that much data. Yet in reality, they have access to more data than they realize–a fact we explored in a recent article<\/span><\/a>. The first step to capitalizing on this data is capturing and storing it in Hadoop.<\/p>\n

But, even if they do have the data volumes to justify Hadoop, many don\u2019t act due to uncertainty. Business leaders aren\u2019t sure how to capitalize on this data. <\/p>\n

If you\u2019re asking yourself that question, let\u2019s answer it with another question: How are other companies capitalizing on Hadoop? While the list could go on, here\u2019s a past article that explains just 7 real-life use cases of Hadoop<\/span><\/a>. Hopefully that gives you some ideas as to the possibilities.<\/p>\n

2. Finding good talent<\/h3>\n

What is the biggest hurdle to Hadoop adoption? According to the survey mentioned above, it\u2019s the lack of Hadoop skills. Why are businesses having so much trouble finding qualified Big Data talent? As explained below, picking up Hadoop skills is more difficult than learning other technical skills.<\/p>\n

\n\u201cThere is a big barrier to learning big data technology,\u201d says Jeffrey Ricker, CEO of Ricker Lyman Robotic<\/span><\/a>. \u201cWith most software, a developer just downloads the software to his laptop and starts hacking. You can\u2019t do that with Hadoop. It requires a minimum of 4 servers to work. Most developers do not have four servers lying around that they can play with to learn a new technology. Cloud is an option, but it is not cheap. For most people, it is not a place to experiment. The barrier to learning is preventing the supply of developers from meeting the exploding demand for big data expertise.\u201d\n<\/p><\/blockquote>\n

\"photo
photo credit: jarmoluk<\/a> via pixabay<\/a> cc<\/a><\/figcaption><\/figure>\n

So, how can you bridge this skills gap? Besides the obvious answer of bringing in new talent, you have a couple of options: <\/p>\n

1. Create your own skills:<\/strong> Training from within your business is cost-effective, and offers another valuable benefit: The trainees already know your business. This approach results in employees who know your business and Hadoop. To help you get started along this path, here\u2019s a great list of free Hadoop training courses<\/span>.<\/p>\n

2. Find the right software: <\/strong>Big Data and Hadoop are still growing fields, but we\u2019re starting to see products emerge that bridge the skills gap for you. For instance, a product like Splice Machine<\/span><\/a> merges the traditional RDBMS with Hadoop–removing the skills gap entirely. Expect to see more offerings crop up that aim to ease the transition.<\/p>\n

3. Hadoop distribution confusion<\/h3>\n

While Hadoop is free and open source software, some vendors have developed their own distributions. They do this to add new capabilities, improve the code base, and offer support. The problem: With a growing number of distributions, differentiating between all of them presents a challenge. How do you know which one to pick?<\/p>\n

\n\u201cThere are many different Hadoop distributions, starting from freely available Hortonworks, Cloudera, MapR and ending with large commercial distributions like IBM InfoSphere BigInsights and Oracle Big Data Appliance,\u201d says Sergey Tryuber, of Grid Dynamics<\/span><\/a>. \u201cSelecting the right distribution is not an easy task (even for experienced staff), since each of them embed different Hadoop components (like Cloudera Impala in CDH), configuration managers (Ambari, Cloudera Manager, etc.), and an overall vision of a Hadoop mission.\u201d\n<\/p><\/blockquote>\n

So, how do you know which option works best for your business? Rather than get into all of the details in this article, here are a couple of articles that compare different distributions in detail.
\n1. Comparing the Top Hadoop Distributions<\/span><\/a><\/p>\n

2. How the 9 Leading Hadoop Distributions Stack Up<\/span><\/p>\n

4. Data accessibility<\/h3>\n
\"photo
photo credit: TiBine<\/a> via pixabay<\/a> cc<\/a><\/figcaption><\/figure>\n

Hadoop provides the framework to store and process data, but that data provides little value for the average business analyst (or business user) unless they can easily transform it into meaningful management information. The problem is, Hadoop was designed as a batch-processing tool. On its own, it offers little in the way of analytics for end users.<\/p>\n

\n“Hadoop is getting increasingly adopted by enterprises because it provides a cost effective, scalable and flexible platform for bringing in all kinds of data sources and building a data repository or “data lake”,\u201d says Ajay Anand, Vice President of products at Kyvos Insights<\/span><\/a>. \u201cHowever, Hadoop is not very accessible for the business user – it’s hard to use, and not designed to be interactive.\u201d\n<\/p><\/blockquote>\n

What can you do about this? Fortunately, we’re seeing advancements in this area. Hadoop analytics is a growing area. Traditional BI vendors are adding Hadoop support to their offerings, and new Hadoop analytic vendors are cropping up. Expect this trend to increase in the coming years.<\/p>\n

\n“It can be difficult to see how Hadoop will deliver business value if the perception is that Hadoop is a large and complex system only accessible by an elite group of IT staff,” says Tyler Wassell, Software Development Manager at mrc<\/span><\/a>. “But if you look at the efforts of traditional BI vendors over the past couple years, you will see that they are quickly bringing Hadoop data analytics to the end user. Business users can now access Hadoop data just like they have accessed any other traditional data. They can answer complex questions, and gain new insights using data that has been captured, processed, and transformed in Hadoop.”\n<\/p><\/blockquote>\n

5. Hadoop integration and management<\/h3>\n

Will Hadoop replace your existing database? While some products offer this option, Hadoop is most often used in tandem with existing systems. <\/p>\n

What does this mean? It means that you must integrate Hadoop with your existing systems–a challenge that is more difficult and time consuming for large Hadoop clusters. It also means you must devote more resources into managing your Hadoop infrastructure.<\/p>\n

\n\u201cA large cluster faces more unique problems specific to the organization’s workflow and data volumes,\u201d says Mark Kerzner, Chief Product Architect at LexInnova<\/span><\/a>. \u201cOne may have to optimize for performance, integrate with existing systems, correctly distribute the load between current and Hadoop infrastructure, and so on.\u201d\n<\/p><\/blockquote>\n

What can you do about this? The answer depends on your database and systems you have in place. Fortunately, most database vendors do have tools and instructions for Hadoop integration. For those looking to manage their Hadoop infrastructure, this article lists some great Hadoop-related tools (and more)<\/span><\/a> that might come in handy.<\/p>\n

Summary<\/h3>\n

Now, these are just a few of the most common Hadoop hurdles. If you would like to add anything to this list, I\u2019d love to hear it. Feel free to share in the comments.<\/p>\n","protected":false},"excerpt":{"rendered":"

Summary: Hadoop has emerged as somewhat of a “poster child” for the Big Data movement. Its ability to store and process massive amounts of data on commodity hardware has caught the eye of many businesses. But, while Hadoop holds massive potential for your business, it’s not without challenges. If you plan on adopting Hadoop in …<\/p>\n

5 hurdles to Hadoop adoption (and how to fix them)<\/span> Read More »<\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"default","ast-global-header-display":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","slim_seo":{"title":"5 hurdles to Hadoop adoption (and how to fix them) - mrc's Cup of Joe Blog","description":"Summary: Hadoop has emerged as somewhat of a \"poster child\" for the Big Data movement. Its ability to store and process massive amounts of data on commodity har"},"footnotes":""},"categories":[8],"tags":[79,100],"class_list":["post-9360","post","type-post","status-publish","format-standard","hentry","category-education","tag-big-data","tag-hadoop"],"_links":{"self":[{"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/posts\/9360","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/comments?post=9360"}],"version-history":[{"count":14,"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/posts\/9360\/revisions"}],"predecessor-version":[{"id":14129,"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/posts\/9360\/revisions\/14129"}],"wp:attachment":[{"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/media?parent=9360"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/categories?post=9360"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/blog\/wp-json\/wp\/v2\/tags?post=9360"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}