'error: not found: value from_avro (Databricks)
Im trying to stream Avro data from Kafka and trying to use from_Avro by following https://docs.databricks.com/spark/latest/structured-streaming/avro-dataframe.html#avro-dataframe
Getting error : not found: value from_avro
val df = spark
.readStream
.format("kafka")
.option("kafka.bootstrap.servers", servers)
.option("subscribe", "t")
.load()
.select(
from_avro($"key", SchemaBuilder.builder().stringType()).as("key"),
from_avro($"value", SchemaBuilder.builder().intType()).as("value"))
using dependencies : bijection-avro_2.12-0.9.6.jar,spark-avro_2.11-4.0.0.jar
Solution 1:[1]
add this import statement import org.apache.spark.sql.avro.functions._
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | ableHercules |