'Change value of a slice in pandas depending on the number of rows in the slice
I have a pandas dataframe that looks like this
import pandas as pd
df = pd.DataFrame({'Timestamp': ['1642847484', '1642847484', '1642847484', '1642847484', '1642847487', '1642847487','1642847487','1642847487','1642847487','1642847487','1642847487','1642847487', '1642847489', '1642847489', '1642847489'],
'value': [11, 10, 14, 20, 3, 2, 9, 48, 5, 20, 12, 20, 56, 12, 8]})
The data is collected in batches which results in multiple lines having the same timestamp . I need to index the dataframe with time and to do so the indexes must have unique values.
The problem as you can see is:
- The timestamp step is varriant
- The number of rows for each timestep is varriant
The approach I tried is
- Multiply timestamp by 1000 to get microseconds
- calculate the time beween timestep i and the next timestep j delta = j-i
- count the number of rows n between i and j
- for each row between i and j add ( 1/n+1 * rank) seconds
expected output:
Timestamp value
0 1642847484000 11
1 1642847484750 10
2 1642847485500 14
3 1642847484000 20
4 1642847487000 3
5 1642847487250 2
6 1642847487500 9
7 1642847487750 48
8 1642847488000 5
9 1642847488250 20
10 1642847488500 12
11 1642847488750 20
12 1642847489000 56
13 1642847489333 12
14 1642847489666 8
15 1642847490000 4
But I couldn't find a way to that efficiently, I used loops but I have 15M+ rows
Is there a simpler way to do it ? Thank you
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
