Hortonworks.com
  • Explore
    • All Tags
    • All Questions
    • All Articles
    • All Ideas
    • All Repos
    • All SKB
    • All Users
    • All Badges
    • Leaderboard
  • Create
    • Ask a question
    • Create Article
    • Post Idea
    • Add Repo
  • Tracks
    • All Tracks
    • Community Help
    • Cloud & Operations
    • CyberSecurity
    • Data Ingestion & Streaming
    • Data Processing
    • Data Science & Advanced Analytics
    • Design & Architecture
    • Governance & Lifecycle
    • Hadoop Core
    • Sandbox & Learning
    • Security
    • Solutions
  • Login
HCC Hortonworks Community Connection
  • Home /
  • Security /
  • Home /
  • Security /
avatar image

Connecting to HBase in a Kerberos Enabled Cluster

  • Export to PDF
Article by JJ Meyer · Aug 02, 2016 at 07:43 PM · edited · Aug 02, 2016 at 03:14 PM
2

Short Description:

This is a walk through on how to connect to HBase via Java or Scala in a Kerberos enabled cluster.

Article

This article will show a step by step guide on how to connect to HBase in an a Kerberos enable cluster. If you need to setup a cluster that have Kerberos enabled, this is a good guid to follow.

Clone Example Project

This article will be based on this project. Please run the following to clone the project.

$ git clone https://github.com/jjmeyer0/hdp-test-examples

Creating Keytab

Before getting into the code, it is important to generate necessary files. If a key tab is not available follow the steps below to create one. In the example below, a key tab for the user jj and realm EXAMPLE.COM is created. The below commands should be run on one of the nodes in the cluster.

$ kadmin.local
$ addprinc jj@EXAMPLE.COM
$ <CTRL-D>
$ ktutil
$ addent -password -p jj -k 1 -e RC4-HMAC
$ wkt jj.keytab
$ q

Preparing User in HBase

The user that was used above must be given correct permissions in HBase. To do so do the following:

$ hbase shell
hbase(main):001:0> grant 'jj', 'RW'

Obtaining Necessary Files

This example also expects the files listed below. Below is a walkthrough on how to copy the necessary files from the cluster to local.

  • hbase-site.xml
  • <username>.keytab
  • krb5.conf
$ scp -i <insecure_private_key> vagrant@c6401:/etc/krb5.conf .
$ scp -i <insecure_private_key> vagrant@c6401:/etc/hbase/conf/hbase-site.xml .
$ scp -i <insecure_private_key> root@c6401:~/jj.keytab .

Once the files have been obtained, please move them to the following directory.

src/main/resources/

For testing, it is recommended to change 'hbase.client.retries.number' property in hbase-site.xml. By default it is 35. This is quite high when running some tests.

Code Walkthrough

The First thing that needs to be done is to create and load the HBase configuration.

// Setting up the HBase configuration
Configuration configuration = new Configuration();
configuration.addResource("src/main/resources/hbase-site.xml");

Next point to the krb5.conf file and setup the kerberos principal and keytab.

// Point to the krb5.conf file.
System.setProperty("java.security.krb5.conf", "src/main/resources/krb5.conf");
System.setProperty("sun.security.krb5.debug", "true");

// Override these values by setting -DkerberosPrincipal and/or -DkerberosKeytab
String principal = System.getProperty("kerberosPrincipal", "jj@EXAMPLE.COM");
String keytabLocation = System.getProperty("kerberosKeytab", "src/main/resources/jj.keytab");

Now login with the principal and keytab defined above.

UserGroupInformation.setConfiguration(configuration);
UserGroupInformation.loginUserFromKeytab(principal, keytabLocation)

Please see this file for full example. A Scala version can be found here.

Resources

  1. Ambari Quickstart Guide
  2. Full Code (This article covers HBase examples)
thub.nodes.view.add-new-comment
How-To/TutorialHbasejavakerberosscala
Add comment · Show 3
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 5 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

avatar image Niranjan Subramanian · May 16, 2017 at 10:13 AM 0
Share

@JJ Meyer

I've a HBase client application which exactly does the same as your code but for some unknown reason Kerberos authentication fails with this exception "GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]". I get this exception only when Kerberos ticket cache is empty but if I do a "Kinit" for my principal before starting the application it runs fine. This indicates Krb5LoginModule's option useTicketCache is set to true somewhere but using "UserGroupInformation.loginUserFromKeytab(principal, keytabLocation)" internally creates a JAAS config which would have useTicketCache as false. What would be the possible reason for this behaviour? From where does my client is picking up useTicketCache as true? I do have a JAAS config file with useTicketCache as true in HBASE_HOME/conf but I'm not using this anywhere in my client application, would my HBase client dependencies pick this configuration? Please let me know what could be the possible reason for this. Regards, Niranjan

avatar image Ravindrdanath Akila · Jun 27, 2018 at 05:48 AM 0
Share

FYR: https://bryanbende.com/development/2016/03/20/learning-kerberos-and-ugi

avatar image Ravindrdanath Akila · Jul 03, 2018 at 05:05 AM 0
Share

https://stackoverflow.com/questions/31233830/apache-spark-setting-spark-eventlog-enabled-and-spark-eventlog-dir-at-submit-or

http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/Spark-job-fails-in-cluster-mode/td-p/58772

if you want to use the keytab from inside a spark (cluster mode), copy the files using --files option and ALIAS. Then use relative path ./your.keytab

Article

Contributors

avatar image

avatar image avatar image avatar image avatar image avatar image
avatar image

Navigation

Connecting to HBase in a Kerberos Enabled Cluster

Related Articles

Choosing Kerberos approach for Hadoop cluster in an enterprise environment

Access Kerberos cluster from JAVA using cached ticket

Setup cross realm trust between two MIT KDC

Auth-to-local Rules Syntax

Ambari 2.4 Kerberos with FreeIPA

One Way Trust - MIT KDC to Active Directory

Converting Spaces to Underscores for user names in HDFS

Connect Hadoop client on Mac OS X to Kerberized HDP cluster

Updating the User-sepecified Kerberos Descriptor

A Secure HDFS Client Example

This website uses cookies for analytics, personalisation and advertising. To learn more or change your cookie settings, please read our Cookie Policy. By continuing to browse, you agree to our use of cookies.

HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service

© 2011-2019 Hortonworks Inc. All Rights Reserved.

Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Login
  • Create
  • Ask a question
  • Create Article
  • Post Idea
  • Add Repo
  • Create SupportKB
  • Tracks
  • Community Help
  • Cloud & Operations
  • CyberSecurity
  • Data Ingestion & Streaming
  • Data Processing
  • Data Science & Advanced Analytics
  • Design & Architecture
  • Governance & Lifecycle
  • Hadoop Core
  • Sandbox & Learning
  • Security
  • Solutions
  • Explore
  • All Tags
  • All Questions
  • All Articles
  • All Ideas
  • All Repos
  • All SKB
  • All Users
  • Leaderboard
  • All Badges