Book review: Apache Hadoop YARN

I've been looking for a comprehensive book on Apache Hadoop 2 and Yarn architecture, there are a few MEAPs available. This book in particular was finally released a few months back with all complete chapters. As all Hortonworks documentation, this book is well written and very easy to read. The choice to choose this book over others was simple. On top of that, it's written by the Hadoop committers so it's basically from the "horse's mouth". The current edition of the book has 12 chapters with additional material. The first chapter goes into history of how Hadoop came about and challenges the team at Yahoo had faced early in Hadoop history. This chapter opened up my eyes on how grandiose the project architecture was in the past and what it's become. It is very easy to take things for granted and this chapter does a great job explaining the choices the team had made. Chapter 2 gives a quick intro on how to deploy a single node cluster and start playing with Hadoop. Chapter 3 goes into the meat of the architecture. Reader will need to dedicate some time reading chapters 3 and 4. Chapters five and six are for system administrators. Chapter five goes into detail how to deploy a Hadoop 2 cluster with and without Apache Ambari. I always wondered what people do when they don't have tools like Ambari. You start appreciating these tools as numbers of nodes increase. Chapter six describes system administration for Hadoop 2, it gives a good understanding for system administrators that are just starting to work with Hadoop. It also goes over metrics and tools like Ganglia and Nagios. Finally, it ends with Ambari overview for monitoring. Again, it shows the complexity and why tools like Ambari are a must have. In chapters 7 and 8, reader again needs to spend some extra time, these chapters are rich with technical info and it is very easy to get lost unless you're really focused. Chapter 8 is dedicated to capacity scheduler and if you need to tune your Hadoop cluster for multi-tenancy, this is the go to chapter for you. In chapter 9, you get an overview of running your Mapreduce code on YARN. Chapter 10 shows you how to write a YARN application, get ready because this chapter is full of examples, well one example but it's pretty long. It goes over a JBoss application running on top of YARN. This is one of those chapters I'll be referring to a lot. Chapter 11 goes over a "Hello World" application for Hadoop 2. This is an overview of a sample app that ships with Hadoop called "Distributed Shell". To really understand how to write apps that run on Yarn, reader needs to understand how Distributed Shell works. Basically, it reminded me of tools like Ansible and Salt Stack. You can have your Hadoop 2 environment run console commands on multiple servers in distributed fashion. This is obviously overkill for the same purposes as Ansible and Salt Stack are purpose-built and only serves as an example of what can be done with Yarn. Finally, chapter 12 goes into brief description of kinds of apps available to run on Yarn, like HBase, Storm and Spark.

Final thoughts about the book: I really enjoyed the book and read it cover to cover. The only gripes I have is that yarn-book.com that was suppose to contain supplemental material has not been updated and none of the material has been published at the time of this post. Even though examples are described fully in the chapters, having full source code is essential. Hopefully it will be available soon. Other than that, I highly recommend this book for system administrators and perhaps developers getting started with Hadoop.