Tuesday 13 November 2018

How do I do a bulk insert in mySQL using node.js


 Answers


Bulk inserts are possible by using nested array, see the github page
Nested arrays are turned into grouped lists (for bulk inserts), 
e.g. [['a', 'b'], ['c', 'd']] turns into ('a', 'b'), ('c', 'd')
You just insert a nested array of elements.
An example is given in here
var mysql = require('node-mysql');
var conn = mysql.createConnection({
    ...
});

var sql = "INSERT INTO Test (name, email, n) VALUES ?";
var values = [
    ['demian', 'demian@gmail.com', 1],
    ['john', 'john@gmail.com', 2],
    ['mark', 'mark@gmail.com', 3],
    ['pete', 'pete@gmail.com', 4]
];
conn.query(sql, [values], function(err) {
    if (err) throw err;
    conn.end();
});
Note: values is an array of arrays wrapped in an array
[ [ [...], [...], [...] ] ]



All props to Ragnar123 for his answer.
I just wanted to expand it after the question asked by Josh Harington to talk about 
inserted IDs.
Hence you can just do this (notice what I did with the result.insertId):
  var statement = 'INSERT INTO ?? (' + sKeys.join() + ') VALUES ?';
  var insertStatement = [tableName, values];
  var sql = db.connection.format(statement, insertStatement);
  db.connection.query(sql, function(err, result) {
    if (err) {
      return clb(err);
    }
    var rowIds = [];
    for (var i = result.insertId; i < result.insertId + result.affectedRows; i++) {
      rowIds.push(i);
    }
    for (var i in persistentObjects) {
      var persistentObject = persistentObjects[i];
      persistentObject[persistentObject.idAttributeName()] = rowIds[i];
    }
    clb(null, persistentObjects);
  });



I was looking around for an answer on bulk inserting Objects.
function bulkInsert(connection, table, objectArray, callback) {
  let keys = Object.keys(objectArray[0]);
  let values = objectArray.map( obj => keys.map( key => obj[key]));
  let sql = 'INSERT INTO ' + table + ' (' + keys.join(',') + ') VALUES ?';
  connection.query(sql, [values], function (error, results, fields) {
    if (error) callback(error);
    callback(null, results);
  });
}

bulkInsert(connection, 'my_table_of_objects', objectArray, (error, response) => {
  if (error) res.send(error);
  res.json(response);
});

I was having similar problem. It was just inserting one from the list of arrays.
 It worked after making the below changes.

  1. Passed [params] to the query method.
  2. Changed the query from insert (a,b) into table1 values (?) ==> insert (a,b) into table1 values ? . 
ie. Removed the paranthesis around the question mark.

I am using mysql npm.

0 comments:

Post a Comment