'How to get all rows with initial values after the assertion in dataform?
I'm doing transformation with dataform , I added some assertions in my transformation
example of table input (all types of columns are STRING):
id | valueu1 | value2 | value3
1 |1?2 | 01/01/1900 00:00:00|x1
2 |1.2 | 01-01-1900 00:00:00|x2
example of stg transformation:
config {
type: "table",
database: "dev",
schema: "schema1",
tags: ["staging", "daily"],
/* DESCRIPTION */
description:"some description",
columns: {
id: "description",
value1: "description",
value2: "description",
value3: "description" }
,
assertions: {
uniqueKey: ["id"],
nonNull: ["valueu1","value2"]
}
}
select
id,
safe_cast(
REGEXP_REPLACE(
value1,
",",
"."
) as numeric
) value1,
SAFE.PARSE_DATETIME('%d/%m/%Y %H:%M:%S', value2) value2,
value3
from
${ref('input')}
The problem here is when the assertion is failed beacause of column value1 or value2 of table input when I want to know the raw value from the view in assertionSchema I can't beacause in the view the value is null (the result of safe_cast). How to get the raw value of column failed in the assertion, I know that I can get this value from input table, but I want to the value in the view of assertionSchema.
Thanks
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
