当前位置 博文首页 > shell脚本操作mysql数据库删除重复的数据

    shell脚本操作mysql数据库删除重复的数据

    作者:Scub 时间:2021-02-12 09:31

    由于之前的业务,造成数据库上产生了脏数据,写个脚本删除重复的数据。由于是开发测试环境,所以选择任意删除相同uid中的一条。由于每次执行只删除重复数据的一条,需要重复执行,如果本轮没有数据被删就OK

    #!/bin/sh
    # delete all company's duplicate uid
    MYSQL_BIN_PATH=/data/mysql/server/mysql_3306/bin
    MYSQL_SOCK_PATH=/data/mysql/server/mysql_3306/tmp
    DBUSER=dbuser
    DBPWD=userpwd
    DBHOSTNAME=192.168.1.105
    PORT=3306
    # get all company_id
    for company_id in `${MYSQL_BIN_PATH}/mysql -u${DBUSER} -p${DBPWD} -h ${DBHOSTNAME} -P ${PORT} --socket=${MYSQL_SOCK_PATH}/mysql.sock -e "
      SELECT company_id FROM company.companypage;"`
    do
     if [ $company_id != "company_id" ] ; then
    # if [ $company_id -eq 2733 ] ; then 
     suffix=`expr ${company_id} % 100`
     for user_id in `${MYSQL_BIN_PATH}/mysql -u${DBUSER} -p${DBPWD} -h ${DBHOSTNAME} -P ${PORT} --socket=${MYSQL_SOCK_PATH}/mysql.sock -e "
     SELECT user_id FROM company.company_candidate_${suffix} WHERE company_id=${company_id} AND user_id>0 GROUP BY company_id, user_id HAVING COUNT(user_id) > 1;"`
     do
     if [ $user_id != "user_id" ] ; then
      ${MYSQL_BIN_PATH}/mysql -u${DBUSER} -p${DBPWD} -h ${DBHOSTNAME} -P ${PORT} --socket=${MYSQL_SOCK_PATH}/mysql.sock -e "
      DELETE FROM company.company_candidate_${suffix} WHERE company_id=${company_id} and user_id=${user_id} limit 1;" 
      echo "delete from company_candidate_${suffix} where company_id=${company_id} and user_id=${user_id} limit 1"
     fi
     done
    # fi
     fi
    done
    exit 0

    总结

    以上就是这篇文章的全部内容了,希望本文的内容对大家的学习或者工作具有一定的参考学习价值,谢谢大家对站长博客的支持。如果你想了解更多相关内容请查看下面相关链接

    js