'How to edit data in CSV with equations to convert to new values?
I've been trying to find a solution. But most of all, I found only a few lines of csv editing. But the data that I have and have to solve consists of thousands of lines.
The data that I use is the value from the sensor-read. but it comes out in voltage. I need to convert it by using equations (which I already have).But I don't know how to use it to edit with many row in csv.
Data sample
| ID | Voltage |
|---|---|
| 1 | 1979 |
| 2 | 1978 |
| ... | ... |
| 49999 | 1976 |
| 50000 | 1976 |
I want to create new column and convert value from old existing column to new column and export to new csv file. Is there any way that can solve it?
I don't know if I should give an example of the equation. But let's just put it together. My equations will be like
Moisture = 100 - ( 0.01 x Voltage )
Here is sample of csv that I want.
| ID | Voltage | Moisture |
|---|---|---|
| 1 | 1979 | 80 |
| 2 | 1978 | 80 |
| ... | ... | ... |
| 49999 | 1976 | 81 |
| 50000 | 1976 | 81 |
Solution 1:[1]
load the csv into a dataframe, this is done easily with pandas
import pandas as pd
df = pd.read_csv('file_path.csv')
then you can create a new column and assign it a value base of existing columns
df['Moisture'] = 100 - (df['Voltage'] * 0.01)
then you can save the dataframe back to a csv
df.to_csv('file_path.csv', index=False)
Solution 2:[2]
You can use csv module for that. You can read data from csv into dictionaries and then just update those dictionaries and write it back. Something like this:
import csv
old_data = list()
with open('data.csv', newline='') as f:
old_data = list(csv.DictReader(f))
for d in old_data:
d['Moisture'] = 100 - (0.01 * d['Voltage'])
with open('new_data.csv', mode='w') as csv_file:
writer = csv.DictWriter(csv_file,
fieldnames=['ID', 'Voltage', 'Moisture'],
lineterminator='\n')
writer.writeheader()
writer.writerows(old_data)
Solution 3:[3]
The following approach just uses the built in CSV library and would work on any file size:
import csv
with open('input.csv') as f_input, open('output.csv', 'w', newline='') as f_output:
csv_input = csv.reader(f_input)
csv_output = csv.writer(f_output)
csv_output.writerow([*next(csv_input), 'Moisture'])
for id, voltage in csv_input:
voltage = float(voltage)
csv_output.writerow([id, voltage, f"{100 - ( 0.01 * voltage ):.1f}"]) # Use .0f if 80 needed (not 80.1)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Patrick |
| Solution 2 | |
| Solution 3 | Martin Evans |
