Category "dataframe"

Creating a new dataframe column with the number of overlapping words between dataframe and list

I'm having some trouble fixing the following problem: I have a dataframe with tokenised text on every row that looks (something) like the following index feelin

How to map single column in pandas using multiple columns (text and numbers) in a separate df

I'm trying to convert U.S. geolocation codes for states, counties and cities. The problem is, the county and city codes are duplicated -- meaning, multiple stat

How to select all columns whose names start with X in a pandas DataFrame

I have a DataFrame: import pandas as pd import numpy as np df = pd.DataFrame({'foo.aa': [1, 2.1, np.nan, 4.7, 5.6, 6.8], 'foo.fighters': [0

Show Method for Dynamic Frame in AWS glue returns empty field

When I try to use the dyF.show() it returns an empty field, even though I checked the schema and count() and I know the table is populated. I transformed it int

Retrieving data from the Air Quality Index (AQI) website through the API and only recieving small nr. of stations

I'm working on a personal project and I'm trying to retrieve air quality data from the https://aqicn.org website using their API. I've used this code, which I'v

Get specific rows which match condition pandas [duplicate]

I have the following dataframe My current code is as follows: Outcome is to only show instances where ImageFileName is services.exe and the P

How to join two very large dataframes together with same columns?

I have two datasets that look like this: df1: Date City State Quantity 2019-01 Chicago IL 35 2019-01 Orlando FL 322 ... .... ... ... 2021-07 Chicago IL 334 202

Get records that are a time interval away from a given date and specific conditions on a pandas DataFrame

Let it be the following Python Panda DataFrame: | ID | date | direction | country_ID | |-----------|-------------------------|----

how to get a single value from dataframe only in Python

I have dataframe df_my that looks like this id name age major ---------------------------------------- 0 1 Mark 34 Engli

Selecting a subset of a dataframe based on a list - pandas

I am working with a large dataframe (ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt) with pandas in Python 3, using PyCharm. The column

iterate over a df and multiply the values by the values of another df

My df1 looks like this:It contains 3 unique project id.The date starts on 01-01-22 and ends on 01-12-28 id date p50 p90 apv1 01-01-22 1000 1000 apv2 01-01-22 1

How to use pivot_longer in this case?

I have the following data frame: df =structure(list(Country = c("DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE",

ValueError: shapes and not aligned: (dim 2) != 4 (dim 0)

I am currently working on a script that does some array manipulating and calculations for modeling. I am running into an error and unsure how to solve it. from

Find differences between a set of csv files in folder 1 against a set of csv files in folder 2?

There are a number a files that need to be compared for differences in their rows; difference not as in subtraction but as in what values are different for each

Dropping NA-values with a MAX threshold and not MIN threshold

Is there any way to remove columns from a dataframe that has LESS NA-values than for instance 200? So instead of df.dropna(threshold = 200) we want the opposite

Why can one column of the pandas DataFrame not be filled?

I'm having some problems iteratively filling a pandas DataFrame with two different types of values. As a simple example, please consider the following initializ

How to fill na values of a column by checking another column

This image would help better: The column titled passengerId describes the group number and person number, people in the same group are usually families, hence

Trouble when trying to do a VLOOKUP like with two pandas dataframes

I've read a lot of questions regarding this matter, but none of it solved my problem. I have 2 dataframes, one containing a list of all students of graduation l

Remove not increasing rows based on other columns values

I have a data frame on R and I want to remove all rows that are not increasing in my column 3. Each row have to be higher or equal than the previous one. But m

Concat multiple dataframe and manage those that doesn't exist

I try to concat some dataframe - 30 dataframe of 24h data - that been created automatically with some csv, but sometimes csv doesn't exist, so the dataframe was