Hadoop for beginners

 

What is Hadoop?

Hadoop is a system for storing data, but is different and more scaleable than a traditional database, such as an SQL database

1.
A normal SQL database requires “schema on write”

If database B is expecting data from database A – database B must know the datatype etc that is being transferred

In Hadoop, there is no “preconfiguring”.
The data from A to B is just sent. Then the rules are applied to the code that reads the data.
This is called “schema on read”

 

2.
In SQL database, the data is stored in a logical form; with related tables and columns.
In Hadoop the data is a compressed file, distributed and duplicated among different nodes/servers.

 3.
To search 100 servers, for all data containing the word “David”
The query is sent via a java-program, that will search across all the replications of the data
Instead of each copy conducting the same search, the search does a portion each
The search-answers are then sent separately to a ‘reducer’
If one server broke, Hadoop will still send an answer to the user.
If one server broke across a SQL network, SQL would not send any data. It adheres to “Two phase commit” which wouldn’t allow this.
SQL is better suited to transactions, that need 100% accuracy all of the time
Hadoop is great for searching massive data, that maybe be less structured

4.
It is more complicated to retrieve data from Hadoop, as it is retrieved via a java program.  There are however interfaces now that will use SQL

 

Hadoop can incorporate a number of machines at once, to retrieve data more quickly.
It harnesses the power of more machines, working together
Hadoop comes from Google white-papers

It has a master component – the NameNode and the JobTracker
and a slave component – the DataNode and the TaskTracker

 

For more detailed information, check out this Hadoop playlist on youtube:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s