Category "pandas"

Fastest way to fill multiple columns by a given condition on other columns pandas

I'm working with a very long dataframe, so I'm looking for the fastest way to fill several columns at once given certain conditions. So let's say you have this

Adding Pandas column in custom function not working when using numpy

I have the following function: def create_col4(df): df['col4'] = df['col1'] + df['col2'] If I apply this function within my jupyter notebook as in create_c

Fastest way to count event occurences in a Pandas dataframe?

I have a Pandas dataframe with ~100,000,000 rows and 3 columns (Names str, Time int, and Values float), which I compiled from ~500 CSV files using glob.glob(pat

Getting the average value of each hour for specific columns in data frame

I have a data frame with the date/time passed as "parse_dates" and then set as the index column for the data frame. Flow Enter Leave

Pandas MultiIndex DataFrame

I have an array: w = np.array([1, 2, 3]) and I need to create a Dataframe with a MultiIndex looking like this: df= 0 1 2 0 0 1 1 1 1 1 1 1 2 1

Convert pandas dataframe hourly values in column names (H1, H2,... ) to a series in a separate column

I am trying to convert a dataframe in which hourly data appears in distinct columns, like here: ... to a dataframe that only contains two columns ['datetime',

mode values under a categorical column in python appear in form of a list

I am using this code to get the mode of a categorical column: df.groupby('user_id')['product'].agg(pd.Series.mode).reset_index().rename(columns = {'product': 'm

Drop Non English Rows Pandas [duplicate]

df.review: de la nada mi ya no se escucha I tried to set it up It is a good product The aim is to remove non-English rows. I tried this and

How to plot two plotly figures with common animation_frame

I am trying to plot both a scatterplot and a line plot, in the same figure. One is for objects and the other for lane markers. The outcome should be one figure

Pandas Method chaining: reassigning a column using df.assign()

I have a dataframe with stock returns in one column, strategy values in another & and another column called trades with boolean values (True, False). My de

How to merge two dfs in pandas (based on datetime period), and add rows if duplicates

I have the following 2 dfs: diag id encounter_key start_of_period end_of_period 1 AAA 2020-06-12 2021-07-07 1 BBB 2021-12-31 2022-01-04 drug id start_datetime

How to estimate similarity between sensor data based on the number of occurrence?

Following is my sample data: data = {850.0: 6, -852.0: 5, 992.0: 29, -993.0: 25, 990.0: 27, -992.0: 28, 965.0: 127, 988.0: 37, -994.0: 24, 996.0: 14, -996.0: 1

Python DataFrame manipulation: How to extract a set of columns in a fast way

I need to access and extract information from a Dataframe that is used for other colleagues in a research group. The DataFrame structure is: zee.loc[zee['layer'

Issues sorting dataframe using isin

I have a dataframe that was converted from a csv using pd.read_csv filled with information with California counties; it looks a little something like this: Cou

Python how ?comparing two columns data into one dataframe

so i have grouping data from this column and then i want to comparing 2 type of the country is 'US' & 'GB into one dataframe so i can make vissualization f

Dataframe add new row if the index does not exist like a dictionary without checking existence

import pandas as pd a = [['a', 1, 2, 3], ['b', 4, 5, 6], ['c', 7, 8, 9]] df = pd.DataFrame(a, columns=['alpha', 'one', 'two', 'three']) df.set_index(['alpha'],

Adding a summarised column back into a dataframe in python

as part of some data cleansing, i want to add the mean of a variable back into a dataframe to use if the variable is missing for a particular observation. so i'

Highlight element based on boolean pandas df

I have 2 data frames with identical indices/columns: df = pd.DataFrame({'A':[5.5, 3, 0, 3, 1], 'B':[2, 1, 0.2, 4, 5],

How to groupby a column but keep all rows as columns

I have a dataframe that was a result of a join operation. This operation had multiple matches, resulting in multiple rows. I want to move resulting match rows t

Python: Formatting a Pandas dataframe head with LaTex

I have made a Pandas dataframe from several NumPy arrays and tried to format columns heads using LaTex, but it looks awful. I'm working with Jupyter Notebook. i