'WriteStream is not able to write Data in Delta Table

I am trying to connect Streaming Json files from the streaming path using the below code

Schema1= "customerId STRING,orderId STRING,products ARRAY<STRUCT<productId: STRING,quantity: STRING,soldPrice: STRING>>,salesRepId STRING,shippingAddress STRUCT<address: STRING,attention: String,city: STRING,state: STRING,zip: STRING>,submittedAt TIMESTAMP";
streamingDF = (spark.readStream.schema(Schema1)\
  .option("maxFilesPerTrigger", 1).json(stream_path))

After few transformations in streamingDF Streaming Dataset and trying to write to a Delta Table using below code

streamingDF.writeStream.outputMode("append")\
  .option("checkpointLocation", orders_checkpoint_path)\
  .partitionBy("submitted_yyyy_mm")\
  .table("sachin")

But those records are not inserted into our delta table and also when I checked the dashboard it shows that numInputRows is 0

Screenshot of streaming while writestream being executed

Why those records are not append into delta table?



Solution 1:[1]

instead of using .table() use .start() then path the table path instead of the table name:

   streamingDF.writeStream.outputMode("append")
  .option("checkpointLocation", orders_checkpoint_path)
  .partitionBy("submitted_yyyy_mm")
  .start("/pathtotable/sachin")

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 user3322581