In this video we are discussing basics of MapReduce Algorithm
So, how this Map and the Reduce two tasks with the help of the mapper class and
the Reducer class, how the things are taking place we shall be discussing in this video
with one sample example to clear your all doubts. So, let us start with this basics of MapReduce algorithm
So here we are having the mapper class, also we are having the Reducer class
output of mapper class will be the input to the reducer class the mapper class is
taking the input and ultimately from the reducer class we're having the output
formed and these inputs and outputs will be in the form of key value pairs this
mapper class will do the tokenizing input mapping and shuffle and shot and
here this reducer will do the searching and reducing operations so these are the
main prime operations in mapper class and these are the two operations to be done in the reducer class. So, different tasks in map reduce algorithms
So, let us discuss more into details. The map reduce programs have two tasks, the map task
and the reduced task, as it has been shown here. The map task is done by the mapper class
and the reduced task will be done by the reducer class. So, this is a map task done by the
mapper class Reduce task will be done by the reducer class Mapper class takes an input tokenizes it and maps and shorts it You can find that map class or the mapping class is taking one input tokenizing it
and doing the suffling and short. So the mapper class takes the input, tokenizes it, and maps and shorts it
The output of the mapper class is used at the input to the reducer class, as I have mentioned
earlier, which in turn such as the matching pairs and reduces them, according to the business
logic, whatever is required, the respective outputs will be obtained and respective customized
function will be working at the reducer class for the reducing operation
MapReduce task example. So, one of the most fundamental map reduce example is word count problem
This work on problem is something like our hello world. problem in MapReduce. So it takes the string with different words as input and counts the number
of words in of each type and let's see how the mapper class and reducer class work for this
task. So a string will be taken as input and then it will count the number of distinct words
in that particular string and that is known as a word count problem. The mapper class tokenize
the strings and makes a shorted list of the words and it makes the words. It makes the words
as the key and the number as the value. So, what will become the key and the number
That means the frequency of occurrence that is a number that is a count will be the value in that case The reducer class takes the list and count the number of entries of each word in the input
which is the output from the mapper. So, the reducer class takes the list and counts the number
of entries of each word. It will check that how many times one word has occurred
Finally, creates another list with key and their respective values, that is the count values
is number of times for which the word has occurred in the sentence
So, here is one example for your better understanding. So, here you see, there is one sentence is there
One input is here. That is our a duck is a bird
We have taken a very simple example. So, mapper class, here we're having take line from the input and tokenize it
And for each word in the line, emit word, comma, one. So there is a, it will create such tokens here
For each and every word in the sentence, it will go for emit word comma 1
So, in this way, it will produce output like A, comma 1, duck, 1, is comma 1, A comma 1 and
B, B, comma 1. So that is the mapper output. Next, it is going for the shorted output
So now it has got shorted in the alphabetical order. So, A, A, B, D, I
So in the alphabetical order, on the keys, the corresponding shorted output has been obtained
Now, remove duplicate keys. So here you can find that A has occurred for twice So that why the key A has got removed here and rest of them are unique So no elimination was taking place So now this this output will be going as
the input to this reducer class. So, reducer class is having one algorithm, something like this
So, reducer key, comma, values. So sum is equal to zero. For each value in key, sum is
equal to sum plus value and emit key comma sum. So, in this way, where it is taking the key
values as input and emitting the key sums as output, and that is also a key value pair
So, A will become, we'll have the sum to, bird one, duck, comma one, and we're having
is a final output which will be obtained. So from this example, there is a word count example
it is getting clear what is the purpose, what are the operations are being done in the
mapper class and also in the reducer class. MapReduce task example, let the string is
a duck is a bird, it has five words and the input is given as input to the input of the
mapper class. So, at first this input will be taken by the mapper class. And mapper will generate
the key value pairs. After that, reducer takes it as input and find final word count. And
the output. In this way, the final word count will be obtained as the output. So, in this video
we have discussed what is the basics of MapReduce algorithm with details discussion and
with some sample examples. Thanks for watching this video