Hortonworks.com
  • Explore
    • All Tags
    • All Questions
    • All Repos
    • All SKB
    • All Articles
    • All Ideas
    • All Repos
    • All SKB
    • All Articles
    • All Ideas
    • All Users
    • All Badges
    • Leaderboard
  • Create
    • Ask a question
    • Add Repo
    • Create Article
    • Post Idea
    • Add Repo
    • Create Article
    • Post Idea
  • Tracks
    • All Tracks
    • Community Help
    • Cloud & Operations
    • CyberSecurity
    • Data Ingestion & Streaming
    • Data Processing
    • Data Science & Advanced Analytics
    • Design & Architecture
    • Governance & Lifecycle
    • Hadoop Core
    • Sandbox & Learning
    • Security
    • Solutions
  • Login
HCC Hortonworks Community Connection
  • Home /
  • Design & Architecture /
avatar image

What are the background process to Monitor the HDFS and Namenode Health check ?

Question by zkfs Dec 13, 2017 at 12:32 PM namenodehadoop-corehdfs-maintenanceprocess

What are the background process to monitor the HDFS and Namenode health check

Comment

People who voted for this

0 Show 0
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

1 Reply

· Add your reply
  • Sort: 
  • Votes
  • Created
  • Oldest
avatar image

Answer by Mridul Mishra · Dec 13, 2017 at 08:48 PM

you may check the process id of namenode through jps command on the master node and also datanode process on slave nodes and check for the process state using ps command.

Kindly use hdfs user superuser (hdfs) to run jps command. Also you may need to install jps binary if it's not already installed.

If you want to monitor you may schedule cron scripts to monitor the process so that it can give you alerts via email/sms.

Comment

People who voted for this

0 Show 2 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image zkfs · Dec 17, 2017 at 08:25 AM 0
Share

by JPS command we can check which are hadoop service runing on cluster, i am looking what background/Daemon process in responsible for monitoring the Health check of HDFS

avatar image zkfs zkfs · Dec 22, 2017 at 05:27 AM 0
Share

Following Demons : 0n hadoop cluster.

3 Daemons run on Master nodes:
NameNode – This daemon stores and maintains the metadata for HDFS.
Secondary NameNode – Performs housekeeping functions for the NameNode.
JobTracker/RM/YARN – Manages MapReduce jobs, distributes individual tasks to machines running the Task Tracker.

2 Daemons run on Slave nodes:
DataNode – Stores actual HDFS data blocks.
TaskTracker/Application Master – Responsible for instantiating and monitoring individual Map and Reduce tasks.

Your answer

Hint: You can notify a user about this post by typing @username

Up to 5 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

43
Followers
follow question

Answers Answer & comments

HCC Guidelines | HCC FAQs | HCC Privacy Policy

Hortonworks - Develops, Distributes and Supports Open Enterprise Hadoop.

© 2011-2017 Hortonworks Inc. All Rights Reserved.
Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.
Privacy Policy | Terms of Service

HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service

© 2011-2018 Hortonworks Inc. All Rights Reserved.

Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Login
  • Create
  • Ask a question
  • Add Repo
  • Create SupportKB
  • Create Article
  • Post Idea
  • Add Repo
  • Create SupportKB
  • Create Article
  • Post Idea
  • Tracks
  • Community Help
  • Cloud & Operations
  • CyberSecurity
  • Data Ingestion & Streaming
  • Data Processing
  • Data Science & Advanced Analytics
  • Design & Architecture
  • Governance & Lifecycle
  • Hadoop Core
  • Sandbox & Learning
  • Security
  • Solutions
  • Explore
  • All Tags
  • All Questions
  • All Repos
  • All SKB
  • All Articles
  • All Ideas
  • All Repos
  • All SKB
  • All Articles
  • All Ideas
  • All Users
  • Leaderboard
  • All Badges