The Mac Smack

Its the Smack World of Hell Mac

How to Install Hadoop on Mac OS [Easy Method]

by Leave a Comment

Hello Friends and Fans! Welcome to another article by Sadiq Apple Solutions. Today we are going to talk about Installing Hadoop on Mac. The very 1st thing that you guys should know is. What is Hadoop and what are the functions/uses of it. Because to Install Hadoop on mac demands a slight knowledge of Mac. If you are new to Mac then be prepared, you guys will have to pass through a series of the zig-zag path. Ok, I have decided that 1st I’ll discuss a lit bit about Hadoop like I always share some value-added information about the topic. Then I’ll tell you guys How to install Hadoop on Mac OS.

Guys do you know?

What is Hadoop?

I will start with something interesting for you guys. Hadoop is a part of Apache project sponsored by Apache Software Foundation. Hadoop is an open source. It functions to store data and helps in running applications in clusters of commodity hardware. Another interesting thing about Hadoop is that it is a Java Based programming framework.

Function of Hadoop

Hadoop is extremely useful as,

  • It provides huge data storage. Data may include, text, picture, video.
  • Hadoop provides an enormous processing power. This makes it extremely useful in complicated processing processes.
  • The most amazing thing about Hadoop is the ability to handle virtually limitless concurrent tasks or jobs.

Which People Require Hadoop

IT Professionals, Youtubers, Video Business people, Photographers. Professional Gamers, Engineers, Graphic designers. I have listed a very few people here. All those persons who have the link to the  Mac and their work involve Huge data storage, fast processing that also include complicated processing and multitasking require Hadoop. Do you know?  Hadoop was discovered/designed/created by computer scientists Doug Cutting and Mike Cafarella in 2006 to support distribution for the Nutch search engine.

Useful for Steve Fans: You people will find it very interesting that it was named Hadoop by the inventor Mr. Cutting  after the Name Hadoop of a yellow baby elephant toy that owned by his son.

How to Install Hadoop on Mac

Now let’s move further to the procedure of installation of Hadoop on Mac OS X. Installing Hadoop on Mac is not as simple as typing a single line command in Terminal to perform an action. It requires a mix of knowledge, concentration, and patience. However, you don’t need to worry about not knowing everything. Just follow the steps that I tell you and you will succeed without a delay.

Step by Step Guide 

How to Install Hadoop on Mac OS

Now we have come to the point which you were searching desperately. How to install Hadoop on Mac. Here you go. This step by step guide will help you to do that with ease and in no time. You will need to know some command. I am sharing with you the commands. You just need to follow these on your mac. I fully hope that after reading this article you will be able to easily install Hadoop on your Mac OS.

One major thing that I would like to mention here that is you should 1st have to check that either Java is already installed on your system or not. Before taking the start to run this command. That will check Java Version on your Mac. For your ease and comfort, I have split the installation process into 3 Sections and named

  • SECTION 1 
  • 1st STEP – INSTALL HOMEBREW
  • 2nd STEP – INSTALL HADOOP
  • SECTION 2
  • 3rd STEP – How to Configure Hadoop
  • 4th STEP – FORMAT HDFS 
  • 5th STEP – VERIFY EXISTENCE OF SSH LOCALHOST KEYS
  • 6th STEP – Enable Remote Login
  • SECTION 3
  • 8th STEP – LOGIN

Each Section is further sub-divided into steps so that you feel it like a piece of cake.

Best of luck for the installation process of Hadoop on Mac.

Here you go,

SECTION 1 

1st STEP

In the 1st step, you need to install HomeBrew. You can also install Hadoop by simply pasting the following command in Terminal:

$ ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”

2nd STEP – INSTALL HADOOP

After that you have installed homebrew, you have to install Hadoop. You can do so by pasting the following script in the Terminal:

$ brew install Hadoop

SECTION 2

3rd STEP – How to Configure Hadoop

After the installation, it is very important to configure Hadoop.

It is also mandatory to mention here that, Hadoop is installed in the following directory

/usr/local/Cellar/Hadoop).

Do the following to configure Hadoop:

FOLLOW THESE STEPS TO CONFITURE HADOOP ON YOUR MAC

In the 1st step, search for [ Hadoop-env.sh ] 

you will easily find Hadoop-env.sh 

here,  

/usr/local/Cellar/hadoop/2.6.0/libexec/etc/Hadoop/Hadoop-env.sh ( 2.6.0 is the version and it could be different in your case).

 

After you have searched that, the 2nd Step is to find [ Core-site.xml ] 

Core-site.xml is available at  

/usr/local/Cellar/hadoop/2.6.0/libexec/etc/hadoop/core-site.xml and edit it in the same manner:

 

Hope you have edited that till now, the 3rd step is to look for mapred-site.xml 

at

/usr/local/Cellar/hadoop/2.6.0/libexec/etc/hadoop/mapred-site.xml and edit in the same way:

 

And locate [ hdfs-site.xml ] 

at

/usr/local/Cellar/hadoop/2.6.0/libexec/etc/hadoop/hdfs-site.xml  also edit this as you have done before,

 

Edit  ~/.profile using the editor you like such a vim, etc, and add the following two commands ( ~/.profile may not exist by default):

$ source ~/.profile

Now you have to execute the following in Terminal to update:

$ source ~/.profile

4th STEP – FORMAT HDFS 

Before using Hadoop, you must format HDFS. You can do so by using this:

$ hdfs namenode -format

5th STEP – VERIFY EXISTENCE OF SSH LOCALHOST KEYS

Check for the existence of ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub files to verify the existence of ssh localhost keys. If these exist to move forward, if they don’t, execute this in Terminal:

$ ssh-keygen -t rsa

6th STEP – Enable Remote Login

by navigating the following path: “System Preferences” -> “Sharing”. Check “Remote Login”. You can do so by using this:

$ ssh-keygen -t rsa

7th STEP AUTHORIZE SSH KEYS 

Now you have to Authorize SSH Keys to make the system aware of the keys that will be used so that it accepts the login. You can do this by using this:

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

SECTION 3

8th STEP – LOGIN

Finally, you can try to login by following this:

$ ssh localhost

Last login: Fri Mar  6 20:30:53 2015

$ exit

and start Hadoop using this:

$ hstart

and then, stop Hadoop using:

$ hstop

In this way, you have successfully installed working Hadoop on your Mac. Now you can use it anytime you desire and anywhere you need.

Why Use Hadoop?

The possibilities are endless. I personally believe that anyone can use Hadoop according to his/her needs but you must 1st learn when did you need it.

  • Hadoop has the ability to store and process massive amounts of any kind of data in a quick manner.
  • It has great flexibility as you don’t have to preprocess data before storing it, unlike traditional databases.
  • You can easily grow your system to handle more data simply by adding nodes.
  • Hadoop is completely free to use and uses commodity hardware to store large amounts of data.
  • Data and application processing are protected against hardware failure so you don’t have to worry about them.
  • Hadoop’s distributed computing model processes big data very fast.

Components of Hadoop

Hadoop has the following 4 Components

  • First one is Hadoop Distributed File System (HDFS)
  • Second is Hadoop Yet Another Resource Negotiator (YARN)
  • Third is Hadoop Common (HC)
  • Fourth is Hadoop MapReduce (HMR)

H uses Hadoop Common as a kernel to provide the framework’s essential libraries. Hadoop Distributed File System (HDFS) is capable of storing data across thousands of commodity servers to achieve high bandwidth between nodes. The Hadoop MapReduce provides the programming model used to tackle large distributed data processing, i.e. mapping data and then reducing it to a result. Hadoop Yet Another Resource Negotiator (YARN) provides resource management and scheduling for user applications. These were the major components of Hadoop. Along with these, there are several other projects that can improvise and extend Hadoop’s basic capabilities Of the Apache-like,

  • The Flume
  • HBase
  • Cloudera Impala
  • Oozie
  • Phoenix
  • Pig
  • Apache Sqoop
  •  Spark
  • Storm
  • Apache ZooKeeper.

Concluding Thoughts

Hope you have liked this article related to install Hadoop on Mac. I have discussed with you about the Hadoop. Functions of Hadoop. Who can use Hadoop? I have also shared with you the step by step guide to install Hadoop on Mac. I have divided the installation process into 3 sections and 8 steps. These are as follows.

  • SECTION 1 
  • 1st STEP – INSTALL HOMEBREW
  • 2nd STEP – INSTALL HADOOP
  • SECTION 2
  • 3rd STEP – How to Configure Hadoop
  • 4th STEP – FORMAT HDFS 
  • 5th STEP – VERIFY EXISTENCE OF SSH LOCALHOST KEYS
  • 6th STEP – Enable Remote Login
  • SECTION 3
  • 8th STEP – LOGIN

If you follow the step by step guide and carefully apply the procedure discussed in this article, I fully hope that you will get the task done easily.

Most Pivotal from Steve – Safety Always 1st

Like I always repeat. Before performing any such procedure, I strongly recommend taking the backup of your device and data so that If any issue arises, your data will be safe. Also, I have a very Important tip for you. Take the backup of your device on monthly basis. This will always be helpful to you.

Hope you have liked today’s article. Do let me know in the comments section. If you like to know more, do let me know. What do you want to know in the next article, do share that with me? This is Steve Sadiq signing off for now.

Stay Blessed.

Peace 🙂