How to create table like structure in Python for alternative to avoid nesting of lists problem?

Question

I want to store data in a table like structure. The way i will populate it is generation of row after complex processing into different section of the codebase.

def insert_into_list(col1,col2,col3...):
   mytable=[]
   mytable.append([col1,col2,upper(col3)...])
   
   return mytable

my ideal table will be of below structure:

mytable=[
['john','doe','michigan',5000],
['jane','doe','michigan',10000],
['mona','johnson','florida',20000],
['sami','kahn','georgia',20000],
...and so on
]

Issue with using append is that mytable list is getting converted to a nested list at more than 2 levels and I am unable to do post processing on it easily.

On the other hand using insert() I am getting error with insert that only 2 values are allowed and as you can see from my example my table has more than 3 columns atleast

If I try to use pandas dataframe using append function, it threw error the append function in it is going to be deprecated and use concact. And concat is not easy to work with :|

Questions:

1. How can i modify the my use of list functions i.e. append or insert to correctly do it?
2. Should i explore some other data structure instead of list to do this kind of work? If what options do you suggest to look . The criteria is that data structure should be easy to move around in various functions and different .py files

You may ask why I am programming like this. I have been a Oracle Database Administrator dba and i am used to thinking table to be something like this :

first	last	state	salary
john	doe	michigan	5000
jane	doe	michigan	10000
mona	johnson	florida	20000
sami	kahn	georgia	20000

Answer 1

Most people use either a pandas dataframe or a numpy 2D array. Both are good options, although pandas tends to give more functionality (and it is built on top of numpy in any case).

If concat is giving you trouble, you should be able to do the same thing you are doing with your list and just add items row by row by using df.loc() or df.iloc().

I found a good example for you here: https://www.geeksforgeeks.org/how-to-add-one-row-in-an-existing-pandas-dataframe/