in this video we are discussing example
of MapReduce program and here you have
considered the program of word count
that is a HelloWorld program we can
consider in about MapReduce so let us
discuss this implementation or this
particular program execution step by
step and this video is associated with
one practical demonstration for your
easy understanding so let us start with
the discussion that is example of
MapReduce program the practical
demonstration of what com problem to run
a MapReduce program we need to start the
Hadoop daemons fast so please initialize
the Hadoop daemons so that it can you
can run this MapReduce program under it
create a copy a file into the HDFS and
put the file inside a separate directory
in this case we are considering slash
input directory so here you are having
one file on which this MapReduce program
will be working the MapReduce will
create another directory in this case it
will be slash output we have mentioned
this one you can change it and in that
directory there will be a part file and
the file will hold the final MapReduce
program output so in the output
directory the file will be there which
is a part file containing the output of
the MapReduce program and in the input
folder will be having our respective
file on which the map map to this
program will be working on that and for
that purpose you are supposed to use the
put command to put the file from the
from the UNIX system to the HDFS the
Watkyn program is one of the default
programs and that comes with hadoop
itself and we can use the jar file to
execute the MapReduce tasks accordingly
the syntax of using the jar file to
start the MapReduce task will be like
this so Hadoop jar jar path class name
and the respective parameters in this
case the parameters are the input and
output directories are to be mentioned
in this particular command so let us go
for one practical demonstration for the
easy understanding
of this concept whenever we install
Hadoop always you know that hadoo comes
with the default one MapReduce program
one on the word count
so this Watkyn program we are going to
execute right now we can show you that
how this particular Watkyn program can
get executed it comes with the Hadoop by
default automatically so in that case
we're opening one terminal here so
contour all dirty there's a terminal
were having we're going for eco dollar
home to show you the corresponding value
of this variable system variable that is
a home so there is a slash home slash
big data so echo dollar Hadoop install
so this particular variable is having
the value their path is slash home slash
big data slash Hadoop so remember this
one so now what we are trying to do is
that we shall execute our work on
program on a on a certain file so now
here I'm going to show you the file here
see I'm just going to show you the file
so I'm just opening this one so I'm at
the home home in sin which path that is
a slash home slash big data so that is a
home
so under this home we are having Hadoop
my files and under this heart of my
files were having this student
underscore info dot C say CSV and also
sample underscore file dot txt
so we shall execute our MapReduce Watkyn
program on this sample underscore file
dot txt okay
so we'll be executing on it so let me
see the content of it so we shall go for
eco we shall go for cat rather so cat we
can go for that is our home you can
write this home in this way putting a
dollar in front we can go for home and
then slash there is a Hadoop my files at
the name of the file is sample file dot
txt ok so that is the way
content of the file you say the same
comment can also be written in this way
the same comment instead of writing this
home we can also put teal symbol here so
that will also do the same others we can
write into details that is a slash home
big data so that is another way to get
the same file content so these are
content on which my what count MapReduce
program will be working now let me tell
you where this particular MapReduce
program is is residing here so in that
case I shall come to home then I shall
come to Hadoop remember this path home
Hadoop then I shall go for share home
Hadoop sheer Hadoop again MapReduce so
home Hadoop sheared Hadoop MapReduce so
here you can see at the end we are
having one file one jar file so there is
a Hadoop - MapReduce - examples - two
point four point one turd jar so this
particular chart is actually containing
the word count class so let me show you
that one also so if I if I explore this
one we are going to come into our G then
Apache then Hadoop then examples so here
you can go for this what count so
there's a wat count or class so this
class we are going to execute okay so
that is our purpose now let me do one
thing so that is my path here so I'm
minimizing this one okay we shall create
one folder onto this HDFS we shall
create one folder on this HDFS so let it
be say my output okay so you see the
folder is not there the folder is not
there let it be my output and in that
indebt my output will be putting that
sample file dot txt so in the HDFS so
how to do that one so at first let me
create one folder at first so how to
create the folder so is the FS DF s -
mkdir slash in the root of the FDA HDFS
we are going to create say my input a
creating one folders of my input
entered let me check whether the folder
has got created or not
yes the folder my input has got created
but the folder is empty but the folder
is empty so let me copy let me copy that
sample there is a sample file dot txt
there so I shall go for HDFS DFS - put
DFS - put slash home slash big data
instead of that I can easily write that
respective field symbol or dollar
capital letters home so that can also be
written so Hadoop my files so how to my
files and then we shall go for say
sample file dot txt so this file we have
copied and now we shall put it into this
my my input I think it is my input yes
so so from this local file system the
file that is a sample underscore file
dot txt is going to get copied onto the
Hadoop file system so I'm just pressing
enter okay let me check yes the sample
underscore file dot txt
initially it was avoid now if this
particular file has got copied there
okay so I have done it now we shall
execute the jar I think you can remember
the path also so now we shall execute
the jar now okay so how to execute the
jar so I think the we have put the file
now we shall go for the jar so we are
going to run the MapReduce jar file and
the jar file is located as add say and
dollar Hadoop installed share hadoop
mapreduce
and the filename is Hadoop - MapReduce -
examples - two point four point one
torch are okay so let me go for that so
I am going for Hadoop Hadoop jar
I'm just executing this code now Hadoop
jar so dollar Hadoop underscore install
you know that instead of typing this
dollar Hadoop underscore install we can
also type slash home slash
big data slash Hadoop so that is a there
is a folder so now we are going for
dollar this and then we shall go for
share then we should go for say after
sure we're having this Hadoop then
MapReduce then we are having that jar
file name so Hadoop - MapReduce so in
that case we shall go for examples then
two point four point one dot jar so
there is a jar path we have given now
not only this we are also supposed to
mention the what count there is a class
name so what count and from where you
should read so there is a my input from
the HDFS it is my input so from there
the respective file will be read and I'm
going to make one output folder it will
be created automatically say my output
so my input and then my output okay so
now we are going for this enter not a
valid jar it is telling that it is not a
valid jar okay let me tell let me check
whatever this is a problem so let me go
for this Hadoop jawed slash had to be
installed shear then hadoop mapreduce
hadoop - MapReduce examples I think is
it is examples there is a two point four
point two point four point one I think
it should be one yes so there is a word
count and my input and my output okay so
now let me go for enter
okay so let me check whether the my
input is all is there it is already
there now let us go for my amp output Li
as my output has got created and this my
output is having a file there is a part
- R - five zeroes okay
so this file is actually containing the
the output so let me let me execute that
one okay I'm going for the clear so now
I want to have this so let me see the
part also so how to see how to print
that one because it is in the hadoo file
system so I should go for HDFS DFS and
then I shall go for the cat the cat is a
command and the path is my output the
path is my output and slash part minus R
minus 5 times 0 so otherwise I can also
use wildcard characters need not to type
everything in full but having this
output so HDFS DFS - cat - file output
slash part - R - 5 zeroes so these are
desperately of output I am getting here
so instead of typing full also I can go
for say wall cut character not
mentioning everything into details so
that can also work here so in this way
in this demonstration we have shown you
that how to run the MapReduce Watkyn
program from the jar calling the
respective class giving the input file
in a certain folder and mentioning the
output folder where the output will be
dumped and what are the sequence of
steps were supposed to do we have
demonstrated everything into details I
think it will clear it has cleared your
conception thanks for watching this
video