''NoneType' object is not subscriptable error when fetching the value
Editing the whole question and title to give more clarity...
I collected a set of rows from a dataframe df using data_collect = df.collect() for sample logic building. There will be millions of rows in the df without filter.
The rows in data_collect are
[Row(Origin='USA', YearWeek='202131', L4Weeks=['202132', '202133', '202134', '202135'], Value=2.0),
Row(Origin='USA', YearWeek='202123', L4Weeks=['202123', '202124', '202125', '202126'], Value=10.0),
Row(Origin='USA', YearWeek='202129', L4Weeks=['202127', '202128', '202129', '202130'], Value=22.0),
Row(Origin='USA', YearWeek='202127', L4Weeks=['202127', '202128', '202129', '202130'], Value=9.0),
Row(Origin='USA', YearWeek='202130', L4Weeks=['202132', '202133', '202134', '202135'], Value=0.0),
Row(Origin='USA', YearWeek='202150', L4Weeks=['202149', '202150', '202151', '202152'], Value=1.0),
Row(Origin='USA', YearWeek='202152', L4Weeks=['202149', '202150', '202151', '202152'], Value=1.0),
Row(Origin='USA', YearWeek='202201', L4Weeks=['202202', '202203', '202204', '202205'], Value=0.0),
Row(Origin='USA', YearWeek='202125', L4Weeks=['202123', '202124', '202125', '202126'], Value=13.0),
Row(Origin='USA', YearWeek='202147', L4Weeks=['202145', '202146', '202147', '202148'], Value=0.0),
Row(Origin='USA', YearWeek='202103', L4Weeks=['202101', '202102', '202103', '202104'], Value=0.0),
Row(Origin='USA', YearWeek='202128', L4Weeks=['202127', '202128', '202129', '202130'], Value=3.0)]
Now I used the following code block to build my logic
for row in data_collect:
value_sum = 0
for week in row['L4Weeks']:
value_sum += spark.sql("""select value from df where Origin = '{}' and YearWeek = {} """.format(row['Origin'], week)).first()[0]
print(value_sum )
This is giving me that None subscription error. If I fetch value from df without passing that week filter, it is working fine.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
