How to fetch data from druid which is ingesting data through kafka in python?

Question

I made the schema of data for druid Then I enable druid kafka ingestion through this command

curl -XPOST -H'Content-Type: application/json' -d @quickstart/tutorial/my_schema.json http://localhost:8081/druid/indexer/v1/supervisor

so far so good. Then in python i made the script of kafka producer

        for line in lines:

            producer.send('vtintel', value=line)
            sleep(1)

I can fetch data through kafka consumer but not in pydruid in python.

When I try to do so like

from pydruid.db import connect

conn = connect(host='localhost', port=8082, path='/druid/v2/sql/', 
scheme='http')
curs = conn.cursor()
curs.execute("""
  SELECT detected,domain
  FROM vtintel
  LIMIT 10
""")
for row in curs:
   print(row)

I get the error:

pydruid.db.exceptions.ProgrammingError: Unknown error (Unknown):

Answer 1

Faced the same issue while using pydruid.db but was able to connect to Druid using SQLAlchemy

from sqlalchemy import *
from sqlalchemy.engine import create_engine
from sqlalchemy.schema import *

engine = create_engine('druid+http://<your_host>:8082/druid/v2/sql/')
# use 'https' if applicable as shown below
# engine = create_engine('druid+https://localhost:8082/druid/v2/sql/')

druid_test = Table('<ur_table_name>', MetaData(bind=engine), autoload=True)
print(select([func.count('*')], from_obj=druid_test).scalar())