hiveserver2 not listening on port 10000 and 10001'

Question

When I run:

hive --service hiveserver2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console

It shows

Starting HiveServer2

and nothing listens on port 10000 and 10001

Answer 1

The HiveServer2 service does not output error information, causing it hard to diagnostic the problem. You can try to start the metastore service provided by Hive, which listens on port 9083 and might give some information when your configuration is not properly set:

 hive --service metastore    # not detach from terminal to see logs

In my case, this service cannot be started, with error message:

MetaException(message:Hive Schema version 3.1.0 does not match metastore's schema 
version 1.2.0 Metastoed or corrupt)

One of the direct solution to resolve this error is to ignore the version difference by setting the hive-site.xml if there is only one hive version in your machine (another solution is to modify the metastore_db version):

<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
</property>

After this problem is resolved, the HiveServer2 service can be running and listening on port 10000.

hive --service hiveserver2 > /dev/null 2>&1 &

If your HiveServer2 access metastore via Derby or MySQL JDBC driver, then the aforementioned metastore service is not needed for HiveServer2. However, if HiveServer2 access metastore via thrift protocol, as configed in conf/hive-site.xml like

<property>
   <name>hive.metastore.uris</name>
   <value>thrift://hadoop-master:9083</value>
   <description>
     Thrift URI for the remote metastore.
     Used by metastore client to connect to remote metastore.
   </description>
 </property>

Then, the metastore service must be started at first.

Answer 2

I had a hard time to set up hive-3.1.2. I write this maybe it helps someone out. in order to diagnose the problem first try to launch metastore and hiveserver2 like this:

metastore:

hive --service metastore --hiveconf hive.root.logger=INFO,console 

hiveserver2:

hive --service hiveserver2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console

then carefully read the the exceptions were thrown.

my problem was user hive is not allowed to perform this api call and to solve that I added the following property to hive-site.xml:

<property>
  <name>hive.metastore.event.db.notification.api.auth</name>
  <value>false</value>
  <description>
    Should metastore do authorization against database notification related APIs such as get_next_notification.
    If set to true, then only the superusers in proxy settings have the permission
  </description>
</property>

also I add my full hive-site.xml as a sample:

<configuration> 
<property> 
    <name>datanucleus.schema.autoCreateTables</name> 
    <value>true</value> 
</property> 

<property> 
    <name>javax.jdo.option.ConnectionURL</name> 
    <value>jdbc:mysql://server-2:3306/metastore?createDatabaseIfNotExist=true</value> 
</property> 

<property>  
     <name>javax.jdo.option.ConnectionUserName</name>   
     <value>mysql_username</value>   
</property> 

<property> 
     <name>javax.jdo.option.ConnectionPassword</name> 
     <value>mysql_password</value>
</property> 

<property> 
     <name>javax.jdo.option.ConnectionDriverName</name> 
     <value>com.mysql.jdbc.Driver</value> 
</property> 

<property> 
    <name>hive.metastore.uris</name>    
    <value>thrift://server-2:9083</value> 
</property> 
    
<property> 
    <name>atanucleus.fixedDatastore</name> 
    <value>true</value> 
</property> 

<property> 
     <name>hive.server2.thrift.bind.host</name> 
     <value>server-2</value>
</property> 

<property> 
     <name>hive.server2.transport.mode</name> 
     <value>binary</value> 
</property> 

<property> 
     <name>hive.server2.enable.doAs</name> 
     <value>false</value>  
</property> 
 
<property> 
     <name>hive.metastore.event.db.notification.api.auth</name> 
     <value>false</value> 
</property> 
</configuration> 

Answer 3

Thanks. There is typo. It should hive.metastore not as shown below.

**metastore**.metastore.event.db.notification.api.auth false