[Erorr:]vieiwng Web UI on EMR cluster

[Divya Gehlot]

Hi,
I am on EMR 4.7 with Spark 1.6.1 and Hadoop 2.7.2
When I am trying to view Any of the web UI of the cluster either hadoop or
Spark ,I am getting below error
"
This site can’t be reached

"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.

Would really appreciate the help.

Thanks,
Divya

[Mohammad Tariq]

Hi Divya,

Do you you have inbounds enabled on port 50070 of your NN machine. Also,
it's a good idea to have the public DNS in your /etc/hosts for proper name
resolution.


[image: --]

Tariq, Mohammad
[image: https://]about.me/mti
<https://about.me/mti?promo=email_sig&utm_source=email_sig&utm_medium=external_link&utm_campaign=chrome_ext>




[image: http://]

Tariq, Mohammad
about.me/mti
[image: http://]
<http://about.me/mti>

Post by Divya Gehlot
Hi,
I am on EMR 4.7 with Spark 1.6.1 and Hadoop 2.7.2
When I am trying to view Any of the web UI of the cluster either hadoop or
Spark ,I am getting below error
"
This site can’t be reached
"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.
Would really appreciate the help.
Thanks,
Divya

[Jonathan Kelly]

I would not recommend opening port 50070 on your cluster, as that would
give the entire world access to your data on HDFS. Instead, you should
follow the instructions found here to create a secure tunnel to the
cluster, through which you can proxy requests to the UIs using a browser
plugin like FoxyProxy:
https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html

~ Jonathan

Post by Mohammad Tariq
Hi Divya,
Do you you have inbounds enabled on port 50070 of your NN machine. Also,
it's a good idea to have the public DNS in your /etc/hosts for proper name
resolution.
[image: --]
Tariq, Mohammad
[image: https://]about.me/mti
<https://about.me/mti?promo=email_sig&utm_source=email_sig&utm_medium=external_link&utm_campaign=chrome_ext>
[image: http://] <http://about.me/mti>
Tariq, Mohammad
about.me/mti
[image: http://]
<http://about.me/mti>

Post by Divya Gehlot
Hi,
I am on EMR 4.7 with Spark 1.6.1 and Hadoop 2.7.2
When I am trying to view Any of the web UI of the cluster either hadoop
or Spark ,I am getting below error
"
This site can’t be reached
"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.
Would really appreciate the help.
Thanks,
Divya

[Divya Gehlot]

Hi,
Thanks all for your prompt response.
I followed the instruction in the docs EMR SSH tunnel
<https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html>
shared by Jonathan.
I am on MAC and set up foxy proxy in my chrome browser

Divyas-MacBook-Pro:.ssh divyag$ ssh -N -D 8157
***@ec2-xx-xxx-xxx-xx.ap-southeast-1.compute.amazonaws.com

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 5: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 8: open failed: administratively prohibited: open failed


What am I missing now ?


Thanks,

Divya

Post by Jonathan Kelly
I would not recommend opening port 50070 on your cluster, as that would
give the entire world access to your data on HDFS. Instead, you should
follow the instructions found here to create a secure tunnel to the
cluster, through which you can proxy requests to the UIs using a browser
plugin like FoxyProxy: https://docs.aws.amazon.com/ElasticMapReduce/
latest/ManagementGuide/emr-ssh-tunnel.html
~ Jonathan

Post by Mohammad Tariq
Hi Divya,
Do you you have inbounds enabled on port 50070 of your NN machine. Also,
it's a good idea to have the public DNS in your /etc/hosts for proper name
resolution.
[image: --]
Tariq, Mohammad
[image: https://]about.me/mti
<https://about.me/mti?promo=email_sig&utm_source=email_sig&utm_medium=external_link&utm_campaign=chrome_ext>
[image: http://] <http://about.me/mti>
Tariq, Mohammad
about.me/mti
[image: http://]
<http://about.me/mti>

Post by Divya Gehlot
Hi,
I am on EMR 4.7 with Spark 1.6.1 and Hadoop 2.7.2
When I am trying to view Any of the web UI of the cluster either hadoop
or Spark ,I am getting below error
"
This site can’t be reached
"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.
Would really appreciate the help.
Thanks,
Divya

[Divya Gehlot]

Hi ,
Thank you all..
Hurray ...I am able to view the hadoop web UI now @ 8088 . even Spark
Hisroty server Web UI @ 18080
But unable to figure out the Spark UI web port ...
Tried with 4044,4040.. ..
getting below error
This site can’t be reached
How can I find out the Spark port ?

Would really appreciate the help.

Thanks,
Divya

Post by Divya Gehlot
Hi,
Thanks all for your prompt response.
I followed the instruction in the docs EMR SSH tunnel
<https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html>
shared by Jonathan.
I am on MAC and set up foxy proxy in my chrome browser
Divyas-MacBook-Pro:.ssh divyag$ ssh -N -D 8157
channel 3: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused
channel 5: open failed: connect failed: Connection refused
channel 22: open failed: connect failed: Connection refused
channel 23: open failed: connect failed: Connection refused
channel 22: open failed: connect failed: Connection refused
channel 23: open failed: connect failed: Connection refused
channel 22: open failed: connect failed: Connection refused
channel 8: open failed: administratively prohibited: open failed
What am I missing now ?
Thanks,
Divya

Post by Jonathan Kelly
I would not recommend opening port 50070 on your cluster, as that would
give the entire world access to your data on HDFS. Instead, you should
follow the instructions found here to create a secure tunnel to the
cluster, through which you can proxy requests to the UIs using a browser
plugin like FoxyProxy: https://docs.aws.amazon.com/ElasticMapReduce/late
st/ManagementGuide/emr-ssh-tunnel.html
~ Jonathan

Post by Mohammad Tariq
Hi Divya,
Do you you have inbounds enabled on port 50070 of your NN machine. Also,
it's a good idea to have the public DNS in your /etc/hosts for proper name
resolution.
[image: --]
Tariq, Mohammad
[image: https://]about.me/mti
<https://about.me/mti?promo=email_sig&utm_source=email_sig&utm_medium=external_link&utm_campaign=chrome_ext>
[image: http://] <http://about.me/mti>
Tariq, Mohammad
about.me/mti
[image: http://]
<http://about.me/mti>

Post by Divya Gehlot
Hi,
I am on EMR 4.7 with Spark 1.6.1 and Hadoop 2.7.2
When I am trying to view Any of the web UI of the cluster either hadoop
or Spark ,I am getting below error
"
This site can’t be reached
"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.
Would really appreciate the help.
Thanks,
Divya

[Jonathan Kelly]

Yes, Spark on EMR runs on YARN, so there is only a Spark UI when a Spark
app is running. To expand on what Natu says, the best way to view the Spark
UI for both running and completed Spark apps is to start from the YARN
ResourceManager UI (port 8088) and to click the "Application Master" link
(for running apps) or "History" link (for completed apps).

~ Jonathan

Hi,
I think the spark UI will be accessible whenever you launch a spark app in
the cluster it should be the Application Tracker link.
Regards,
Natu

Post by Divya Gehlot
Hi ,
Thank you all..
But unable to figure out the Spark UI web port ...
Tried with 4044,4040.. ..
getting below error
This site can’t be reached
How can I find out the Spark port ?
Would really appreciate the help.
Thanks,
Divya

Post by Divya Gehlot
Hi,
Thanks all for your prompt response.
I followed the instruction in the docs EMR SSH tunnel
<https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html>
shared by Jonathan.
I am on MAC and set up foxy proxy in my chrome browser
Divyas-MacBook-Pro:.ssh divyag$ ssh -N -D 8157
channel 3: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 3: open failed: connect failed: Connection refused
channel 4: open failed: connect failed: Connection refused
channel 5: open failed: connect failed: Connection refused
channel 22: open failed: connect failed: Connection refused
channel 23: open failed: connect failed: Connection refused
channel 22: open failed: connect failed: Connection refused
channel 23: open failed: connect failed: Connection refused
channel 22: open failed: connect failed: Connection refused
channel 8: open failed: administratively prohibited: open failed
What am I missing now ?
Thanks,
Divya

Post by Jonathan Kelly
I would not recommend opening port 50070 on your cluster, as that would
give the entire world access to your data on HDFS. Instead, you should
follow the instructions found here to create a secure tunnel to the
cluster, through which you can proxy requests to the UIs using a browser
https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html
~ Jonathan

Post by Mohammad Tariq
Hi Divya,
Do you you have inbounds enabled on port 50070 of your NN machine.
Also, it's a good idea to have the public DNS in your /etc/hosts for proper
name resolution.
[image: --]
Tariq, Mohammad
[image: https://]about.me/mti
<https://about.me/mti?promo=email_sig&utm_source=email_sig&utm_medium=external_link&utm_campaign=chrome_ext>
[image: http://] <http://about.me/mti>
Tariq, Mohammad
about.me/mti
[image: http://]
<http://about.me/mti>

Post by Divya Gehlot
Hi,
I am on EMR 4.7 with Spark 1.6.1 and Hadoop 2.7.2
When I am trying to view Any of the web UI of the cluster either
hadoop or Spark ,I am getting below error
"
This site can’t be reached
"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.
Would really appreciate the help.
Thanks,
Divya