Hortonworks.com
  • Explore
    • All Tags
    • All Questions
    • All Repos
    • All SKB
    • All Articles
    • All Ideas
    • All Users
    • All Badges
    • Leaderboard
  • Create
    • Ask a question
    • Add Repo
    • Create Article
    • Post Idea
  • Tracks
    • All Tracks
    • Community Help
    • Cloud & Operations
    • CyberSecurity
    • Data Ingestion & Streaming
    • Data Processing
    • Data Science & Advanced Analytics
    • Design & Architecture
    • Governance & Lifecycle
    • Hadoop Core
    • Sandbox & Learning
    • Security
    • Solutions
  • Login
HCC Hortonworks Community Connection
  • Home /
  • Data Science & Advanced Analytics /
avatar image

Apache Zeppelin Tech Preview Live

Question by vshukla Oct 23, 2015 at 10:12 PM Sparkzeppelinspark-sqldata-science

http://hortonworks.com/hadoop-tutorial/apache-zeppelin/

The Zeppelin TP is built against Spark 1.4.1 in HDP. We are also about to publish Spark 1.5.1 TP very soon and once that is out Zeppelin TP will also be revised to carry instructions for Spark 1.5.1.

Please play with it and post here if you run into any issues.

Comment
Neeraj Sabharwal

People who voted for this

1 Show 0
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

3 Replies

· Add your reply
  • Sort: 
  • Votes
  • Created
  • Oldest
avatar image
Best Answer

Answer by vshukla · Oct 24, 2015 at 06:13 PM

Added a couple of sentences of clarification to get folks to Spark 1.4.1.

If you have an HDP 2.3.0 cluster, it came with Spark 1.3.1, you can either upgrade the entire cluster withAmbari to 2.3.2 to get Spark 1.4.1 or only manually upgrade Spark to 1.4.1

Comment

People who voted for this

0 Show 1 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Simon Elliston Ball ♦ · Oct 24, 2015 at 11:03 PM 0
Share

Tried this on a 2.3.2 cluster (brand new build) with 1.4.1, and had the same problem with Zeppelin and Magellan. Seems like Zeppelin is doing something to the context.

avatar image

Answer by Neeraj Sabharwal · Oct 24, 2015 at 10:42 AM

Looks good. Installation was simple.

Blog needs minor editing.

Add the following properties and settings:

spark.driver.extraJavaOptions -Dhdp.version=2.3.2.0-2950 spark.yarn.am.extraJavaOptions -Dhdp.version=2.3.2.0-2950

this should be

spark.driver.extraJavaOptions -Dhdp.version=2.3.2.0-2950 

spark.yarn.am.extraJavaOptions -Dhdp.version=2.3.2.0-2950

For newbies, we may want to share this in the blog as it works like charm


screen-shot-2015-10-23-at-102230-pm.png (91.4 kB)
Comment

People who voted for this

0 Show 1 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image vshukla · Oct 24, 2015 at 06:17 PM 0
Share

Check samples notebook section for a link to a few notebooks.

avatar image

Answer by Ali Bajwa · Oct 24, 2015 at 05:54 PM

Installed this through the Ambari service for testing and basic Spark, SparkSQL, PySpark seem ok

Couple of issues:

1. tried out the Magellan blog notebook (after modifying it to include the %dep from the blog) and the UberRecord cell errors out:

From the log:

5/10/24 10:48:06 INFO SchedulerFactory: Job remoteInterpretJob_1445708886505 started by scheduler org.apache.zeppelin.spark.SparkInterpreter313266037
15/10/24 10:48:06 ERROR Job: Job failed
scala.reflect.internal.Types$TypeError: bad symbolic reference. A signature in Shape.class refers to term geometry
in value com.core which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling Shape.class.
	at scala.reflect.internal.pickling.UnPickler$Scan.toTypeError(UnPickler.scala:847)
	at scala.reflect.internal.pickling.UnPickler$Scan$LazyTypeRef.complete(UnPickler.scala:854)
	at scala.reflect.internal.pickling.UnPickler$Scan$LazyTypeRef.load(UnPickler.scala:863)
	at scala.reflect.internal.Symbols$Symbol.typeParams(Symbols.scala:1489)
	at scala.tools.nsc.transform.SpecializeTypes$$anonfun$scala$tools$nsc$transform$SpecializeTypes$$normalizeMember$1.apply(SpecializeTypes.scala:798)
	at scala.tools.nsc.transform.SpecializeTypes$$anonfun$scala$tools$nsc$transform$SpecializeTypes$$normalizeMember$1.apply(SpecializeTypes.scala:798)
	at scala.reflect.internal.SymbolTable.atPhase(SymbolTable.scala:207)
	at scala.reflect.internal.SymbolTable.beforePhase(SymbolTable.scala:215)
	at scala.tools.nsc.transform.SpecializeTypes.scala$tools$nsc$transform$SpecializeTypes$$norma

(side note: this notebook doesn't seem to have much documentation on what its doing like the other...would be good to add)

2. The blog currently says the below

This technical preview can be installed on any HDP 2.3.x cluster

...however 2.3.0 comes with Spark 1.3.1 which will not work unless they manually install Spark 1.4.1 TP so either:

a) we may want to include steps for those users too (esp since the current version of the sandbox comes with 1.3.1)

b) explicitly ask users to try the Zeppelin TP with 2.3.2


screen-shot-2015-10-24-at-105147-am.png (287.9 kB)
Comment

People who voted for this

0 Show 1 · Share
10 |6000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Ali Bajwa ♦ · Oct 25, 2015 at 10:15 AM 0
Share

Zeppelin Ambari service has been updated to install the updated TP Zeppelin bits for Spark 1.4.1 and 1.3.1. The update will be made for 1.5.1 this week after the TP is out

Also the Magellan notebook has also been updated with documentation and to enable it to run standalone on 1.4.1

Your answer

Hint: You can notify a user about this post by typing @username

Up to 5 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

21
Followers
follow question

Answers Answer & comments

This website uses cookies for analytics, personalisation and advertising. To learn more or change your cookie settings, please read our Cookie Policy. By continuing to browse, you agree to our use of cookies.

HCC Guidelines | HCC FAQs | HCC Privacy Policy | Privacy Policy | Terms of Service

© 2011-2019 Hortonworks Inc. All Rights Reserved.

Hadoop, Falcon, Atlas, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie and the Hadoop elephant logo are trademarks of the Apache Software Foundation.

  • Anonymous
  • Login
  • Create
  • Ask a question
  • Add Repo
  • Create SupportKB
  • Create Article
  • Post Idea
  • Tracks
  • Community Help
  • Cloud & Operations
  • CyberSecurity
  • Data Ingestion & Streaming
  • Data Processing
  • Data Science & Advanced Analytics
  • Design & Architecture
  • Governance & Lifecycle
  • Hadoop Core
  • Sandbox & Learning
  • Security
  • Solutions
  • Explore
  • All Tags
  • All Questions
  • All Repos
  • All SKB
  • All Articles
  • All Ideas
  • All Users
  • Leaderboard
  • All Badges