'csvkit mysql import fails with VARCHAR requires a length on dialect mysql

After installing csvkit with the following command

$ sudo -HE pip install --upgrade -e git+git://github.com/wireservice/csvkit.git@master#egg=csvkit

and trying to import a .csv as follows:

csvsql --db mysql://root:[email protected]:3306/jira_test --insert --table bugs_temp --no-constraints --overwrite --create-if-not-exists --no-inference --blanks bugs_temp.csv

I get the following error(s)

/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_2".
/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_3".
/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_4".
/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_5".
/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_6".
/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_7".
/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_8".
/usr/local/lib/python2.7/dist-packages/agate/utils.py:292: DuplicateColumnWarning: Column name "0" already exists in Table. Column will be renamed to "0_9".
(in table 'bugs_temp', column 'ESS-3146'): VARCHAR requires a length on dialect mysql
mysqlcsvkit


Solution 1:[1]

(This answer might be somewhat uncomplete. I don't have experience with csvsql, so I'm making it a wiki.)

csvkit depends on agate-sql, which depends on sqlalchemy.

sqlalchemy contains this line of code:

    "VARCHAR requires a length on dialect %s" % self.dialect.name

It seems like ”you need to specify lengths for all Strings“, citing this answer.

That means you cannot use the --no-constraints option:

  --no-constraints      Generate a schema without length limits or null
                        checks. Useful when sampling big tables.

See https://csvkit.readthedocs.io/en/latest/scripts/csvsql.html

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1

Related Questions