Can we bind spark2 history server to private IP?
I have multi-homing network, I want to bind Spark2 history server to private interface instead of 0.0.0.0
I tried changing spark.history.bind.host to have private IP and also tried adding SPARK_LOCAL_IP to spark env by referring https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-history-server.html however in vain.
e.g.
Instead of 0.0.0.0 I want private IP to be there.
Any pointers would be appreciated!
I'm not sure if there is a regression in HDP 2.6. I tried setting SPARK_LOCAL_IP in one of my HDP 2.5.5 version and this seems to work. I have exported SPARK_LOCAL_IP in under Advanced spark2-env and started spark2 History server. Below is the output
18/03/17 04:06:17 INFO log: Logging initialized @4468ms
18/03/17 04:06:18 INFO Server: jetty-9.2.z-SNAPSHOT
18/03/17 04:06:18 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@71154f21{/,null,AVAILABLE}
18/03/17 04:06:18 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@15f193b8{/json,null,AVAILABLE}
18/03/17 04:06:18 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@2516fc68{/api,null,AVAILABLE}
18/03/17 04:06:18 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@304a9d7b{/static,null,AVAILABLE}
18/03/17 04:06:18 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@6bfdb014{/history,null,AVAILABLE}
18/03/17 04:06:18 INFO ServerConnector: Started ServerConnector@21ae6e73{HTTP/1.1}{10.10.2.21:18081}
18/03/17 04:06:18 INFO Server: Started @4722ms
18/03/17 04:06:18 INFO Utils: Successfully started service on port 18081.
18/03/17 04:06:18 INFO HistoryServer: Bound HistoryServer to 10.10.2.21, and started at http://10.10.2.21:18081
^C
[root@vnode-21 ~]# netstat -tupln | grep 18081
tcp6 0 0 10.10.2.21:18081 :::* LISTEN 37481/java
Tried the same on HDP 2.6.3 and this is the output. It says started on 0.0.0.0:18081 and also Bound HistoryServer to 172.31.252.173
18/03/17 03:54:30 INFO AbstractConnector: Started ServerConnector@1d269ed7{HTTP/1.1,[http/1.1]}{0.0.0.0:18081}
18/03/17 03:54:30 INFO Utils: Successfully started service on port 18081.
18/03/17 03:54:30 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@3163987e{/,null,AVAILABLE,@Spark}
18/03/17 03:54:30 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@4e6d7365{/json,null,AVAILABLE,@Spark}
18/03/17 03:54:30 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@d4602a{/api,null,AVAILABLE,@Spark}
18/03/17 03:54:30 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@11a82d0f{/static,null,AVAILABLE,@Spark}
18/03/17 03:54:30 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@60e949e1{/history,null,AVAILABLE,@Spark}
18/03/17 03:54:30 INFO HistoryServer: Bound HistoryServer to 172.31.252.173, and started at http://172.31.252.173:18081
Guessing that there may be some issue with Jetty
.
-Aditya
@Kuldeep Kulkarni, As @Aditya Sirna mentioned setting SPARK_LOCAL_IP gets SHS to bind to the mentioned ip, However if you are setting this value in spark-env through ambari it will affect all your spark jobs. As SPARK_LOCAL_IP is a driver ip all drivers of your jobs using this spark-env will try to bind to mentioned ip.
Instead set export SPARK_LOCAL_IP=<ip> in /usr/hdp/current/spark-historyserver/sbin/start-history-server.sh so that only SHS will pick the ip.
@Sandeep Nemuri / @Aditya Sirna - Thanks for your help. I forgot to mention that I had tried adding it in the script as well and I could see that variable is being picked up in /proc/<pid-of-S2HS> but it still shows 0.0.0.0 in netstat.
Let me go through the additional notes given by @Aditya Sirna
Thanks for your help guys.
