Map Reduce on YARN Framework:
YARN is Yet Another Resource Negotiator.
- Clients submit MapReduce job by interacting with Job objects, clients runs in its own JVM.
- Job’s code interact with Resource Manager to acquire application metadata such as application id.
- Job’s code move all job related resources to HDFS to make them available for rest of the job.
- Job’s code submits the application to Resource Manager.
- Resource Manager choses a Node manager with available resources and requests a container for MRAppMaster.
- Node manager allocates container for MRAppMaster; MRAppMaster will execute and co-ordinate MapReduce Job
- MRAppMaster grabs required resources from HDFS, such as input splits; these resources were copied in step 3.
- MRAppMaster negotiates with Resource Manager for available resources; Resource Manager will select Node Manager that has the most resources.
- MRAppMaster tells selected NodeManager to start Map and Reduce tasks.
- Node Manager creates YARN Child containers that will coordinate and run tasks.
- YARN Child acquires job resources from HDFS that will be required to execute Map and Reduce tasks.
- YARN Child executes Map and Reduce Tasks.