+91-9686502645

Big Data / Data Science

Bigdata Hadoop Administration


Hadoop Training and Administrating big data ecosystem in Linux environment.

Apache Hadoop is a collection of open-source software utilities that facilitate using a network of many computers to solve problems involving massive amounts of data and computation. It provides a software framework for distributed storage and processing of big data using the MapReduce programming model. Originally designed for computer clusters built from commodity hardware—still the common use—it has also found use on clusters of higher-end hardware.  All the modules in Bigdata Hadoop are designed with a fundamental assumption that hardware failures are common occurrences and should be automatically handled by the framework.

 

Target Audience

  • Network/system admin
  • Network/system engineer
  • Freshers

Hadoop Course Content:

OPERATING SYSTEM

Administering Open Source System

  • Administering Open Source System (Unix Systems),
  • The Role of an administrator
  • Open Source Licensing
  • Acquiring your Linux Distribution

 

Installation Process

  • Installation Process of Linux Red Hat System
  • Structuring the Filesystem
  • Selecting the software Packages
  • Performing Installation

Booting Linux

  • Managing Boot Process
  • Following Boot Scripts Sequence
  •  Assigning services with chk config
  • The /etc directory configuration Hierarchy

 

Rescuing an Unbootable System

  • Booting into Rescue Mode
  • Reinstalling the Boot Loader
  • Booting into Single-User Mode

 

User Features

  • PAM-Pluggable Authentication Module,
  • What does Home Directory of file users, mean?
  • What is Syntax for Change?
  • How to Change User Features of A User?

 

Linux PS Commands

  • The Linux group mod Command
  • The Linux g password Command
  • Linux PS Command
  • Procs
  • Memory
  • Swap
  • Proc Command
  • Pkill Linux Command
  • Syslog
  • View Newlog Entries

 

Package Manager

  • Manipulating portable tar archives
  • Installing software with REDHAT packet manager(RPM),
  • What is RPM-REDHAT package manager

 

IP Configuration

  • Rebuild a source RPM(SRPM) package
  • Static IP configuration
  • View network settings of an Ethernet Adapter
  • Assigning IP Address to an Interface
  • Configuring and testing IPV6 connectivity
  • Standalone server
  • Running services through TINTED

 

Creating Partitions

  • Creating Linux partition
  • Mounting a file system
  • Create users
  • Add a user into a group in Linux

 

Mounting

  • Mounting File System
  • Mount Specific file system

 

SAMBA Server and UDEV

  • Configure SAMBA Server
  • Examine the Steps in Reporting PCI Devices Bug
  • UDEV,
  • Add or Remove a Linux Kernel Modules/Drivers

 

Installing LVS

  • Define LVS
  • Installing LVS
  • Understanding Linux Director
  • Testing and Debugging
  • Real servers and Ip fail.

    Apache Hadoop

Hadoop Installation & setup

  • Hadoop 2.x Cluster Architecture
  • Federation and High Availability
  • A Typical Production Cluster setup
  • Hadoop Cluster Modes
  • Common Hadoop Shell Commands
  • Hadoop 2.x Configuration Files
  • Cloudera Single node cluster
  • Hive
  • Pig
  • Sqoop
  • Flume
  • Scala
  • Apache Spark.

 

Introduction to Big Data Hadoop: Understanding HDFS & Map Reduce

  • Introduction Big Data & Hadoop,
  • What is Big Data and where does Hadoop fit in
  • Two important Hadoop ecosystem components namely Map Reduce and HDFSI
  • In-depth Hadoop Distributed File System – Replications, Block Size, Secondary Name node, High Availability
  • In-depth YARN – Resource Manager, Node Manager.
  • Hands-on Exercise –
    • Working with HDFS
    • Replicating the data
    • Determining block size
    • Familiarizing with Namenode and Datanode.

 

Deep Dive in MapReduce

  • Detailed understanding of the working of MapReduce
  • The mapping and reducing process,
  • The working of Driver, Combiners, Partitioners, Input Formats, Output Formats, Shuffle and Sort
  • Hands-on Exercise
  • The detailed methodology for writing the Word Count Program in MapReduce
  • Writing custom partitioner
  • MapReduce with Combiner
  • Local Job Runner Mode
  • Unit Test
  • ToolRunner
  • Map Side Join
  • Reduce Side Join
  • Using Counters
  • Joining two datasets using Map-Side Join Vs Reduce-Side Join

 

Hadoop Administration – Multi-Node Cluster Setup using Amazon EC2

  • Create a four-node Hadoop cluster setup
  • Running the MapReduce Jobs on the Hadoop cluster
  • Successfully running the MapReduce code
  • Working with the Cloudera Manager setup.
  • Hands-on Exercise
  • The method to build a multi-node Hadoop cluster using an Amazon EC2 instance
  • Working with the Cloudera Manager.

 

Hadoop Administration – Cluster Configuration

  • Overview of Hadoop configuration
  • Importance of Hadoop configuration file
  • The various parameters and values of configuration
  • HDFS parameters and MapReduce parameters
  • Setting up the Hadoop environment
  • Include and Exclude configuration files,
  • Administration and maintenance of Name node, Data node directory structures, and files
  • File system image and Edit log
  • Hands-on Exercise
  • The method to do performance tuning of MapReduce program.

 

Bigdata Hadoop Administration – Maintenance, Monitoring and Troubleshooting

  • Introduction to the Checkpoint Procedure
  • Namenode failure and how to ensure the recovery procedure
  • Safe Mode
  • Metadata and Data backup
  • The various potential problems and solutions
  • What to look for, how to add and remove nodes.
  • Hands-on Exercise
  • How to go about ensuring the MapReduce Filesystem
  • Recovery for various different scenarios
  • JMX monitoring of the Hadoop cluster
  • How to use the logs and stack traces for monitoring and troubleshooting
  • Using the Job Scheduler for scheduling jobs in the same cluster
  • Getting the MapReduce job submission flow
  • FIFO schedule
  • Getting to know the Fair Scheduler and its configuration.
  •  ETL Connectivity with Hadoop Ecosystem
  •  How ETL tools work in Big data Industry
  • Introduction to ETL and Data warehousing
  • Working with prominent use cases of Big data in ETL industry
  • End to End ETL PoC showing big data integration with ETL tool.
  • Hands-on Exercise
    • Connecting to HDFS from ETL tool
    • Moving data from Local system to HDFS
    • Moving Data from DBMS to HDFS
    • Working with Hive with ETL Tool
    • Creating a Map-Reduce job in ETL tool

 

“Hadoop Training” by Corporate Trainers who are having real-time hands-on experience, Training with 100% Placement Assistance

Type Lesson Title Time

Scroll to Top