'How to update the batch dataset in structured streaming?
I am a beginner for spark. Currently, I am using Structured streaming. I have stream dataset join with a static dataset(which is batch dataset). But the static dataset could not be updated.
for example:
sparkSession.sql("select * from tableA") in the first batch.
But the batch dataset maybe changed in the next batch because of some reason.
sparkSession.sql("select * from tableB").
How to archive that?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|