'AWS EMR EMRFS Kerberos login on policy refresh
I installed Kerberos on a ec2 server and on a second ec2 server I installed Apache Ranger (with Kerberos auth added in core-site file,hadoop.security.authentication , this means that all the ranger plugins must login to kerberos before policy refresh) .
I made a security configuration to use the Kerberos as external KDC and the Ranger server with repositories for hive, spark and emrfs.
After cluster is ready to use I need to go to all ranger plugins (hive, spark and emrfs) to set the secureMode to true in order to make the kerberos auth before the policy refresh.
After this update (secureMode=true) I restart all the plugin services and the hive and spark ranger plugins are working as expected (I can see the kerberos auth in logs and the policy file is created under ranger_policy_cache directory). But the emrfs can not login to kerberos and the policy request to ranger is failed. In the logs I can see that the request is coming with id=null (for hive the id=hive and for spark is emr-record-server).
In /emr/secretagent/ I have a kerberos directory but it is empty. No other kerberos setting for emrfs.
My question is how to add kerberos details to emrfs? Or how to config the emrfs to get the ranger policies with secureMode=yes (kerberos auth)? It is possible to make emrfs working with kerberos?
Or is there another way to get the policies for all ranger plugins?
Note: The emrfs is getting the policies if I delete the kerberos auth (hadoop.security.authentication from core-site file) from Ranger Admin but I have issues on hive and spark. So I prefer to make the emrfs to work with kerberos auth enabled
Any suggestion is welcomed. I really do not find a way to fix this issue... Thank you
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
