'Cumulative drop over a period of time
A column (col1) value decreasing over the time, How to find the cumulative drop in values for different window size?
Col1 is the Input field(Sensor) values sometimes decreases over time, Output is to send an alert at the time when the %drop values <-3.5% over a specific window.The formula to calculate the %drop is (present value-80)/80*100 , (i.e,Col1 start value for a window size of 6 is 80). For each window, the column start value changes. i.e, for window2, it become 79. Basically a rolling window of drop percentage to be calculated. How can this be achieved in Pyspark.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|