Skip to content
Advertisement

how to avoid redundency while inserting into the database in scraping using nodejs and mysql

Am developing one using nodejs scraping and mysql. I want to store details into the mysql database. I have written the query it is saving sucessfully and it also checks for redundancy if we run the script again. If there is no data (empty table) in the table then, it is not checking for redundancy. All data is saving into the database.

The code what i have written to avoid redundency and insert into the table.

mysql.query(
    'select id, name from ' + product_details + ' where name = "'+$name+'"',                                                             
    function(err, result, fields) {
        if (err) throw err;
        else {
            var i;
            for (i in result) {
                var product = result[i];
                if(product.id>0){
                }
            }
            if(i==undefined){
                mysql.query('insert into '+ product_details +' (name) values ("' + $name + '")',
                    function selectCb(err, results, fields) {
                        if (err) throw err;
                        else {
                            console.log("inserted");
                    }
            });
        }
    }
});

Please help me to come out from this problem. Am new to nodejs scraping. Please help me. Thanks in advance. ** UPDATE **

Thank you user1027167. I tried your solution and i fixed the problem. But now mysql error occurs then script execution stops, but i want script to continue its execution by omitting (leaving) that detail to be saved in the database. How to do that please help me

Advertisement

Answer

you have to fetch the error like this:

mysql.query('insert into '+ product_details +' (name) values ("' + $name + '")',
    function errorhandler(err, results, fields) {
        if (err)
        {
            console.log("name is not unique");                
            // do something else
        }
    }
});

So you should not throw it.

User contributions licensed under: CC BY-SA
1 People found this is helpful
Advertisement