Hortonworks.com
  • Explore
    • All Tags
    • All Questions
    • All Repos
    • All SKB
    • All Articles
    • All Ideas
    • All Users
    • All Badges
    • Leaderboard
  • Create
    • Ask a question
    • Add Repo
    • Create Article
    • Post Idea
  • Tracks
    • All Tracks
    • Community Help
    • Cloud & Operations
    • CyberSecurity
    • Data Ingestion & Streaming
    • Data Processing
    • Data Science & Advanced Analytics
    • Design & Architecture
    • Governance & Lifecycle
    • Hadoop Core
    • Sandbox & Learning
    • Security
    • Solutions
  • Login
HCC Hortonworks Community Connection
  • Home /
  • Hadoop Core /
  • Home /
  • Hadoop Core /
avatar image

Hive LLAP (in HDP 2.6) is not supported with OneFS

  • Export to PDF
Vivek Somani created · Sep 29, 2017 at 12:38 AM · edited · Sep 29, 2017 at 12:59 AM
4

Article

Hortonworks had released HDP 2.6 in Q2 2017. Isilon engineering team worked closely with Hortonworks engineering team to validate HDP 2.6 with OneFS.

One of the features of HDP 2.6 was LLAP (Long Live And Process). LLAP enables as fast as sub-second SQL analytics on Hadoop by intelligently caching data in memory with persistent servers that instantly process SQL queries.

However, please note that LLAP is officially NOT SUPPORTED with OneFS. Customers are advised NOT to use LLAP (even though it may seem to work after config changes) because it could potentially result in data inconsistency issue

Additional Details

  • By default, LLAP uses “inode paths” to access data from file system. OneFS currently does not support referencing files by inode. So queries coming through LLAP will fail on OneFS.
  • However, it is possible to configure Hive LLAP to NOT use inode path by setting hive.llap.io.use.fileid.path = false

With this configuration change, the queries will be executed and this will make LLAP technically “usable” with Isilon but will also cause potential data inconsistency issue. This is because with this capability turned off, LLAP will try to access data by filename (and not by inode path) in the filesystem, which means that the LLAP cache can potentially has an outdated file with the same filename but not knowing that there is a newer one to fetch.

Recommendation:

LLAP support with Isilon is currently planned for future release of OneFS. For more details on timing of release, contact your Isilon/Hortonworks representative

thub.nodes.view.add-new-comment
faqfaqfaqhdp-2.6isilonllaponefs
Add comment
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Article

Contributors

avatar image

Follow

Follow

avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image avatar image avatar image avatar image
avatar image avatar image

Navigation

Hive LLAP (in HDP 2.6) is not supported with OneFS

Related Articles

HDFS Recovery Time from Single DataNode Failure

hive Insert to Dynamic Partition query Generating too many small files

HBase Replication - FAQ

Compression in HBase

My-SQL Hive User creation and grants on it

HDP 2.5.6.0 is now released and includes about 60 Apache JIRA fixes

Why container-executor.cfg is overwritten when restarting services

Apache Kafka New Features Journey from HDP 2.2.0 to HDP 2.6.1

HDFS Balancer: Balancing Data Between Disks on a DataNode

What is HDFS Ozone?

HCC Guidelines | HCC FAQs | HCC Privacy Policy

Hortonworks - Develops, Distributes and Supports Open Enterprise Hadoop.

© 2011-2017 Hortonworks Inc. All Rights Reserved.
Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.
Privacy Policy | Terms of Service

HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service

© 2011-2018 Hortonworks Inc. All Rights Reserved.

Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Login
  • Create
  • Ask a question
  • Add Repo
  • Create SupportKB
  • Create Article
  • Post Idea
  • Tracks
  • Community Help
  • Cloud & Operations
  • CyberSecurity
  • Data Ingestion & Streaming
  • Data Processing
  • Data Science & Advanced Analytics
  • Design & Architecture
  • Governance & Lifecycle
  • Hadoop Core
  • Sandbox & Learning
  • Security
  • Solutions
  • Explore
  • All Tags
  • All Questions
  • All Repos
  • All SKB
  • All Articles
  • All Ideas
  • All Users
  • Leaderboard
  • All Badges