'Scala Spark unit test Dataset.map function
I am having trouble writing unit test for the Dataset.map function for the below snippet, I am trying to write unit test using Flatspec and Matchers library
// Function within a Class
def dataValidation(ds: Dataset[someCaseClass]): Long = {
val validDs: Dataset[Booelan] = ds.map(entry => validate(entry))
validDs.filter(entry = !entry).count
}
def validate(data: someCaseClass): Boolean{
if(valid(data))
true
else
false
}
In the above code snippet I am facing hardtime in writing mock data for
val validDs: Dataset[Booelan] = ds.map(entry => validate(entry))
I get below error if I try to mock the map function as below
unit test code
when(
mock[Dataset[someCaseClass]].map(entry => validate(entry))
).thenReturn(mock[Dataset[Boolean]])
error that I get
SmartNull returned by this unstubbed method call on a mock:
dataset.map(
<function1>,
class[value[0]: boolean]
);
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
