Kafka Spark compatibility

Question

Thos code gives the error "no module named Kafka"

import os
from datetime import datetime
import time
import threading
import json
from kafka import KafkaProducer
from kafka.errors import KafkaError
import pandas as pd
from sklearn.model_selection import train_test_split

what are the compatible versions of K=kafka, oython and spark that prevent this error ?

Answer 1

You're not using any Spark libraries here.

You'll need to install kafka-python, it seems