java - centos服务器CPU被GC task占满，程序无内存溢出

Question

1. 环境：服务器为centos，2个8核16线CPU，内存64G，jdk版本1.7.0_67

2. 问题描述：服务器部署的java应用，jvm堆内存为8G，启动一段时间后CPU占用飙升到2300%

   • jstack查看获得如下信息:

     "VM Thread" prio=10 tid=0x00007f5ac4095800 nid=0xa599 runnable
     
     "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f5ac4021000 nid=0xa56b runnable
     "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f5ac4023000 nid=0xa56c runnable
     "GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f5ac4025000 nid=0xa56d runnable
     "GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f5ac4026800 nid=0xa56e runnable
     "GC task thread#4 (ParallelGC)" prio=10 tid=0x00007f5ac4028800 nid=0xa56f runnable
     "GC task thread#5 (ParallelGC)" prio=10 tid=0x00007f5ac402a800 nid=0xa570 runnable
     "GC task thread#6 (ParallelGC)" prio=10 tid=0x00007f5ac402c800 nid=0xa571 runnable
     "GC task thread#7 (ParallelGC)" prio=10 tid=0x00007f5ac402e800 nid=0xa572 runnable
     "GC task thread#8 (ParallelGC)" prio=10 tid=0x00007f5ac4030800 nid=0xa573 runnable
     "GC task thread#9 (ParallelGC)" prio=10 tid=0x00007f5ac4032800 nid=0xa574 runnable
     "GC task thread#10 (ParallelGC)" prio=10 tid=0x00007f5ac4034000 nid=0xa575 runnable
     "GC task thread#11 (ParallelGC)" prio=10 tid=0x00007f5ac4036000 nid=0xa576 runnable
     "GC task thread#12 (ParallelGC)" prio=10 tid=0x00007f5ac4038000 nid=0xa577 runnable
     "GC task thread#13 (ParallelGC)" prio=10 tid=0x00007f5ac403a000 nid=0xa578 runnable
     "GC task thread#14 (ParallelGC)" prio=10 tid=0x00007f5ac403c000 nid=0xa579 runnable
     "GC task thread#15 (ParallelGC)" prio=10 tid=0x00007f5ac403e000 nid=0xa57a runnable
     "GC task thread#16 (ParallelGC)" prio=10 tid=0x00007f5ac403f800 nid=0xa57b runnable
     "GC task thread#17 (ParallelGC)" prio=10 tid=0x00007f5ac4041800 nid=0xa57c runnable
     "GC task thread#18 (ParallelGC)" prio=10 tid=0x00007f5ac4043800 nid=0xa57d runnable
     "GC task thread#19 (ParallelGC)" prio=10 tid=0x00007f5ac4045800 nid=0xa57e runnable
     "GC task thread#20 (ParallelGC)" prio=10 tid=0x00007f5ac4047800 nid=0xa57f runnable
     "GC task thread#21 (ParallelGC)" prio=10 tid=0x00007f5ac4049800 nid=0xa580 runnable
     "GC task thread#22 (ParallelGC)" prio=10 tid=0x00007f5ac404b800 nid=0xa581 runnable
     "VM Periodic Task Thread" prio=10 tid=0x00007f5ac40d5800 nid=0xa5a3 waiting on condition
     
     JNI global references: 760

   • heap信息如下：

      PSYoungGen      
     total 5208064K, used 1186489K [0x00000006b1000000, 0x0000000800000000, 0x0000000800000000)
     eden space 4918272K, 20% used [0x00000006b1000000,0x00000006ef010b20,0x00000007dd300000)
     from space 289792K, 58% used [0x00000007ee500000,0x00000007f8b9db50,0x0000000800000000)
     to   space 280576K, 0% used [0x00000007dd300000,0x00000007dd300000,0x00000007ee500000)
      ParOldGen       
     total 1714688K, used 705292K [0x0000000413000000, 0x000000047ba80000, 0x00000006b1000000)
     object space 1714688K, 41% used [0x0000000413000000,0x000000043e0c3378,0x000000047ba80000)

   • dump下来6G多的内存快照用MAT做分析，疑似溢出只有38Mb，看了一下是设计的阻塞队列，基本可以排除内存溢出

   • 出问题部分的代码并行2个线程，一个处理json的序列化与反序列化以及zip流的写出操作，json用的是jackson2.3，zip流方面检查过了没有资源未关闭导致溢出的情况；还有一个对处理完的数据进行推送，收集到指定数量或等待指定时间才做一次推送

   现在能想到可能的情况都做了排查，本机单独运行出问题的模块也没什么问题，有没有遇到过类似问题的朋友，还望不吝赐教...