'Import CSV file to mariadb
Lately, I am facing problems importing from CSV files. I am using
MariaDB : 10.3.32-MariaDB-0ubuntu0.20.04.1
On Ubuntu : Ubuntu 20.04.3 LTS
I am using this command
LOAD DATA LOCAL INFILE '/path_to_file/data.csv' INTO TABLE tab
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES;
After searching and trying I found that I can only load File from tmp folder. i.e.
SELECT load_file('/tmp/data.csv');
But it didn't work on other paths.
And secondly, I found that even If the CSV file is present in tmp folder; If it contains a lot of fields then again MariaDB would fail to load. The main problem is that LOAD DATA command does not give any type of error or even warning; except if the file does not exist. Other than that nothing is shown. And nothing is imported.
I only succeeded to import very simple CSV from tmp folder
What I Suspected is that
MariaDB had been updated and in this new version there are some flags or configuration options that prohibit MariaDB from importing CSV files from other than tmp folder and
MariaDB would fail to load CSV because of some unknown problem, Maybe some special character (which I made sure nothing is in there).
There must be some option that makes MariaDB produce verbose error and warning log. Which I didn't know. Except for
/var/log/mysql/error.logfile. which does not contain any info containing failed to load CSV.
Any help would be appreciated.
Below is the first record of CSV. Actual CSV contains 49 fields and 1862 records (but the below sample contains only one record)
"S.No","Training Code","Intervention Type (NRM/Emp. Skill
Training)","Training Title/Activity","Start Month","Ending
Month","No. of Days Trainings","Start Date Training","End Date
Training","Name of Person","Father
Name","CNIC","Gender","Age","Education","Skill Level","CO Ref
#","COName","Village Name","Tehsil Name","District","Type of Farm
production","Total Land (if applicable)","Total Trees (if
applicable)","Sheeps/goats","Buffalo/Cows","Profession","Person's
Income Emp. Skill (Pre-Intervention)","Income from NRM (Pre-
Intervention)","HH Other Sources of Income","Total HH Income","Type
of Support provided","Tool Kit/Inputs Received or Not","Date of
Tool Kit receiving","Other intervention , like exposure market
trial, followup support, Advance Training etc","Production (Pre-
Intervention)","Production (Post-Intervention)","Change in
Production","Unit (kg, Maund,Liter, etc)","Income gain from
production (Post-Intervention)","Change in Income (NRM)","Income
gain by Employment -Emp.Skill (Post Intervention)","Change in
Income (Emp. Skill)","Outcome Trend","Employment/Self-
Employment/Other","Outcome Result","Remarks","Beneficiaries Contact
No.","Activity Location"
1,"AUP-0001","NRM","Dates Processing &
Packaging","Sep/2018","Sep/2018",2,"25/Sep/2018","26/Sep/2018",
"Some name","Barkat Gul",1234567891234,"Male",34,"Primary","Semi-
Skilled","AUP-NWD-073","MCO Haider Khel Welfare Committee","Haider
Khel","Mir Ali","North
Waziristan","Dates",,20,,,"Farming",,5000,"Farming",5000,"Training,
Packaging Boxes","Yes","10/10/2018",,180,320,140,"Kg",8000,3000,,,
"Positive","Self Employed","Value addition to the end product
(Packaging increase the Price per KG to 25%)",,,"Field NW"
Solution 1:[1]
BTW am NON-Technical :-O
While using Mariadb version 10.5.13-3.12.1 am able to import CSV files into Tables have set up.
Except with dates,
There am still struggling to import text-format-dates AND to convert text-dates into the (YYYY-MM-DD) date format.
end.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 |
