Skip to content

Instantly share code, notes, and snippets.

@raullenchai

raullenchai/gist:2e79a64187ac8ca62ef7198b14136b3d

Last active July 30, 2019 03:52
Show Gist options
  • Save raullenchai/2e79a64187ac8ca62ef7198b14136b3d to your computer and use it in GitHub Desktop.
Save raullenchai/2e79a64187ac8ca62ef7198b14136b3d to your computer and use it in GitHub Desktop.
Download ZIP
need a tool to import csv to mysql
Raw
gistfile1.txt
Have a csv of 500K+ lines, need a tool to ingest it to a mysql table in where one can specify X and Y, meaning for every insertion of X lines, sleep Y seconds.
CSV looks like
Name,Email
Philip Morris,[email protected]
Jane Webster,[email protected]
Mysql table looks like
+---------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| userID | int(11) | YES | | NULL | |
| name | varchar(100) | YES | | NULL | |
| email | varchar(100) | YES | MUL | NULL | |
| custom_fields | longtext | YES | | NULL | |
| list | int(11) | YES | MUL | NULL | |
| unsubscribed | int(11) | YES | MUL | 0 | |
| bounced | int(11) | YES | MUL | 0 | |
| bounce_soft | int(11) | YES | MUL | 0 | |
| complaint | int(11) | YES | MUL | 0 | |
| last_campaign | int(11) | YES | MUL | NULL | |
| last_ares | int(11) | YES | | NULL | |
| timestamp | int(100) | YES | MUL | NULL | |
| join_date | int(100) | YES | | NULL | |
| confirmed | int(11) | YES | MUL | 1 | |
| messageID | varchar(100) | YES | MUL | NULL | |
| ip | varchar(100) | YES | | NULL | |
| country | varchar(2) | YES | MUL | NULL | |
| referrer | varchar(500) | YES | MUL | NULL | |
| method | int(1) | YES | MUL | NULL | |
| added_via | int(1) | YES | MUL | NULL | |
| gdpr | int(1) | YES | MUL | 0 | |
| notes | text | YES | | NULL | |
+---------------+------------------+------+-----+---------+----------------+
e.g.,
mysql> select * from subscribers;
+-------+--------+------+----------------------------+---------------+------+--------------+---------+-------------+-----------+---------------+-----------+------------+-----------+-----------+-----------+------+---------+----------+--------+-----------+------+-------+
| id | userID | name | email | custom_fields | list | unsubscribed | bounced | bounce_soft | complaint | last_campaign | last_ares | timestamp | join_date | confirmed | messageID | ip | country | referrer | method | added_via | gdpr | notes |
+-------+--------+------+----------------------------+---------------+------+--------------+---------+-------------+-----------+---------------+-----------+------------+-----------+-----------+-----------+------+---------+----------+--------+-----------+------+-------+
| 13974 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13975 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13976 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13977 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13978 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13979 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13980 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13981 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13982 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13983 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13984 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
| 13985 | 1 | | [email protected] | NULL | 11 | 0 | 0 | 0 | 0 | NULL | NULL | 1564458600 | NULL | 1 | NULL | NULL | NULL | NULL | NULL | NULL | 0 | NULL |
+-------+--------+------+----------------------------+---------------+------+--------------+---------+-------------+-----------+---------------+-----------+------------+-----------+-----------+-----------+------+---------+----------+--------+-----------+------+-------+
12 rows in set (0.00 sec)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment