java - 从mysql中查询数据,然后插入到另一个数据库中,怎样提升这个代码的运行速度
巴扎黑
巴扎黑 2017-04-17 17:51:11
0
2
622

我从mysql中查询数据,然后将查询到的数据插入到另一个数据库中,目前速度查询5000个并插入的速度大概在45秒到200秒之间。怎样可以提升它的速度。
这条sql本身的执行时间在numiid很多的时候或者表中数据较多的时候就较慢,有时在1.6秒以上
sql:
select ir.id,iss.numIid,iss.bid,b.bname as bT,iss.cid,c.name as cT,iss.userId ,ss.shopTitle,iss.title as itemTitle,ir.type,ir.page,if(count(keyword) is null,0,count(keyword)) keywordCount1,IF(k.ssrq is null, 0,sum(k.ssrq )) ssrqCount1,if(iss.sales!=-1,iss.sales,IF(iss.trades!=-1,iss.trades,iss.receivers)) salesCount1

    from table iss
    inner join table0  ir  on iss.numIid=ir.numIid
    <if test="type != null"> and ir.type =#{type}</if> 
    <if test="type == null"> and ir.type in (1,2,3,4,11,12,13,21,22,31,32)</if>
    <if test="page != null"> and ir.page =#{page}</if> 
    <if test="page == null"> and ir.page in(1,2,3,4,5,6,7,8,9,10) </if> 
    left join table1_${batchId} k on ir.keyword =k.id
    left join table2_${batchId} ss on ss.userId =iss.userId
    left join table3  b  on iss.bid=b.bid 
    left join table4  c  on iss.cid=c.id
    where iss.numIid  in (
        <foreach collection="numIids" item="numIid" separator=",">
          #{numIid}
        </foreach>
    )
    group by iss.numIid,ir.page,ir.type
巴扎黑
巴扎黑

reply all(2)
大家讲道理

The time span from 45 seconds to 200 seconds is huge.
1. The SQL you write is too complicated. It is recommended to simplify the SQL and execute part of the logic processing in the code.
2. Save to another database and consider batch submission.

大家讲道理

1. In the first step of the query, see if there is room for optimization. Querying for more than 1 second feels a bit too much. Check if the index is used.
2. Can you post the pseudo code of your insertion logic? I don’t know how you do the insertion operation. I feel that it cannot take dozens of seconds to insert 5,000 pieces of data.

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template