Category "dataframe"

PySpark read data into Dataframe, transform in sql, then save to dataframe

New to Spark and Synapse....Need to do some transformation including adding a columns, changing datatypes, etc. I am reading a csv into a dataframe. I'd like t

Apply change to timedelta to columns containing a given string

Be the following DataFrame in python pandas: date time_SEL time_02_SEL_01 time_03_SEL_05 other 2022-01-01 34756 233232 3432423 756 2022-01-03 23322 4343 3334 3

How to duplicate each row having only one column different than the previous row pandas data frame?

I have a big data and I want to duplicate each row just below the original column by changing just one column value I want to copy the previous row value in pl

Calculate the difference in days between two date fields

I have a problem. I have two date fields fromDate and toDate. The toDate also contains a timestamp, e.g. 2021-03-22T18:59:59Z. The problem is that I want to cal

Select previous row every hour in pandas

I am trying to obtain the closest previous data point every hour in a pandas data frame. For example: time value 0 14:59:58 15 1 15:00:10 2

Error - replacement has [x] rows, data has [y]

I have a numeric column ("value") in a dataframe ("df"), and I would like to generate a new column ("valueBin") based on "value." I have the following condition

convert float64 (from excel import) to str using pandas

although the same question has been asked multiple times. I dont seem to make it work. I use python 3.8 and I rean an excel file like this df = pd.read_excel(r"

convert float64 (from excel import) to str using pandas

although the same question has been asked multiple times. I dont seem to make it work. I use python 3.8 and I rean an excel file like this df = pd.read_excel(r"

Deleting multiple rows under same App Name but with different number of reviews

I have a dataframe having many columns, 2 of them being 'App' and 'Reviews'. I discovered that for the same app there are multiple rows because they differ in t

Perform a merge by date field without creating an auxiliary column in the DataFrame

Be the following DataFrames in python pandas: | date | counter | |-----------------------------|------------------| | 2022-01-0

Concat null columns data with actual data in pandas?

I have set of columns need to be merged into single column where some columns have data and some don't have where it should be joined with the data to single co

pandas, creating dataframes based on tuple

I have a tuple that has data for several categories. Now I want to extract small dataframes from this tuple for each category based on a list I created. I want

Count occurrences within a specific range

I have a data frame that looks like this: Tag 0 skip_1 1 run 2 skip_1 3 run 4 skip_1 5

Multiply without eliminate information

I have a dataframe and I would like to maintain information. My data frame is like: a <- c("a","b", "c", "d") b <- c("e","f", "g", "h") c <- c(1, 2, 1,

Pandas Value Error: Cannot set item on a Categorical with a new category, set the categories first

I've been looking for other similar issues on this ValueError, but none of them has the same code as I have. So here it is. As I am still very new at this, I am

How to find the number of seconds elapsed from the start of the day in pandas dataframe

I have a pandas dataframe df in which I have a column named time_column which consists of timestamp objects. I want to calculate the number of seconds elapsed f

Python calculate increment rows till a condition

How to obtain the below result. Sample Data with Output Time To default is the column which is to be calculated. We need to get the increment number as Time to

Pandas Group by index Hour and keeping observation for each hour

I have a pandas dataframe containing one column and a datetime index, i need to group the data by hour and keep each obsevation (record) for each of the grouped

ParseError: Error tokenizing data. C error: Buffer overflow caught - possible malformed input file. (read_csv)

I cannot use read_csv method of pandas properly on kaggle. Error that I get is: ParseError: Error tokenizing data. C error: Buffer overflow caught - possible ma

how to add columns and values in a dataframe in python

In the below JSON array { "data": [ { "name": "page_call_phone_clicks_logged_in_unique", "period": "lifetime", "values": [ {