Pandas - Key Exception - Length mismatch: Some times Expected axis has 3 elements, Some times has 2 elemnts

Question

I have built a script to update stock values from yahoo finance with pandas.

Sometimes the script works fine, but at some point it gets an error:

Key Exception - Length mismatch: Expected axis has 3 elements, new values have 2 elements

Then I change the axis to 3 elements, but at some point, I start to get this error:

Key Exception - Length mismatch: Expected axis has 2 elements, new values have 3 elements

How could I solve that?

# update Stocks price
import pandas as pd
from django.core.management.base import BaseCommand
import yfinance as yf
from investments.models import Asset, Stocks


class Command(BaseCommand):

    def handle(self, *args, **options):
        print("Updating Stocks price from yahoo finance")

        # get assets from db that will be updated
        queryset = Stocks.objects.values_list("id", "ticker")
        app_df = pd.DataFrame(list(queryset), columns=["id", "ticker"])
        app_list = app_df["ticker"].astype(str).tolist()
        # print(app_list)

        # get price from yfinance (only from Stocks in db)
        yahoo_df = yf.download(app_list, period="1min")["Adj Close"]
        yahoo_df = yahoo_df.T.reset_index()
        # try in case of error
        try:
            yahoo_df.columns = ["ticker",  "price"]
            # if start to get axis error has 3 element, I change here to
            # yahoo_df.columns = ["ticker",  "price", "price2"]
            yahoo_df["price"] = yahoo_df["price"].round(2)
            yahoo_df = yahoo_df.set_index('ticker')
        except Exception as e:
            print(f' Key Exception - {e}')
            pass
        print(yahoo_df)

That's the output of yahoo_df is:

Date      index  2022-05-12 00:00:00  2022-05-13 00:00:00
0     ALZR11.SA                  NaN           115.699997
1     BARI11.SA                  NaN            99.989998
2     BRCO11.SA                  NaN            99.300003
3     BTAL11.SA                  NaN            95.370003
4     BTLG11.SA                  NaN           103.489998
5     BTRA11.SA                  NaN           100.040001
6     CPTS11.SA                  NaN            92.660004
7     DEVA11.SA                  NaN            94.000000
8     EQIR11.SA            97.010002                  NaN
9     FIIB11.SA                  NaN           426.850006

````

And sometimes yahoo_df is:

````

Date      index    2022-05-13 00:00:00
0     ALZR11.SA             115.699997
1     BARI11.SA              99.989998
2     BRCO11.SA              99.300003
3     BTAL11.SA              95.370003
4     BTLG11.SA             103.489998
5     BTRA11.SA             100.040001
6     CPTS11.SA              92.660004
7     DEVA11.SA              94.000000

````

Answer 1

Found the solution. Using if and else, so the command if yahoo_df.shape[1] == 3 , checks if the axis is equal 3.

And yahoo_df["price"] = yahoo_df["price"].fillna(yahoo_df["price2"]) fill the NaN of the column price with the value of column2.

if yahoo_df.shape[1] == 3:
   yahoo_df.columns = ["ticker",  "price", "price2"]
   yahoo_df["price"] = yahoo_df["price"].fillna(yahoo_df["price2"])
                
else:
   yahoo_df.columns = ["ticker",  "price"]