'How to fill spaces in dates fast? Pandas DataFrames

I have efficiency problem here, I have data like this: DataFrame

I want to fill spaces of dates with that specific date, and Quantity 0 (for every PrekesId) to look like this (screenshot only contains a part of data) DataFrame part-result

I am using this code right now :

unique_products = data['PrekesId'].unique()
date_range = pd.date_range(start=data.Date.min(), end=data.Date.max(), freq='M')
new_df = pd.DataFrame(columns=['PrekesId', 'Year', 'Month', 'Quantity'])
for product in unique_products:
  for date in date_range:
    # if test[(test['Year'] == date.year) & (test['Month'] == date.month)]['Quantity'].empty:
    try:
      new_df = new_df.append({'PrekesId': product, 'Year': date.year, 'Month': date.month, 'Quantity': test[(test['Year'] == date.year) & (test['Month'] == date.month)]['Quantity'].iloc[0], 'Date': date}, ignore_index=True)
    except IndexError:
      new_df = new_df.append({'PrekesId': product, 'Year': date.year, 'Month': date.month, 'Quantity': 0, 'Date': date}, ignore_index=True)
new_df

It has taken 8 minutes for 15k row initial dataframe.

Edit:

My intention is to have a dataframe of size len(date_range) * len(unique_products) initial dataframe doesn't have rows where quantity is 0, and thus i have rows with skipping months.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source