当前位置 博文首页 > 沙漏哟：《Java架构师的最佳实践》生产环境JVM调优之空间担保失

最大化 缩小

    沙漏哟：《Java架构师的最佳实践》生产环境JVM调优之空间担保失

    作者：沙漏哟 时间：2021-05-27 18:20

    1  问题现象

    应用prod-xxx-k8s，在内存足够的情况下，仍然会产生偶发FullGC的问题。

    JVM配置如下：

    -Xmx8192m -Dhsf.server.max.poolsize=2500 -Xms8192m -XX:MaxDirectMemorySize=1026m -XX:+UseGCLogFileRotation -XX:ParallelGCThreads=10 -XX:NewSize=3096m -XX:SurvivorRatio=8 -Xloggc:/home/admin/gc.log -XX:NumberOfGCLogFiles=1 -XX:ConcGCThreads=10 -XX:+PrintGCDateStamps -XX:MaxNewSize=3096m -XX:NewRatio=4 -XX:GCLogFileSize=1024m -XX:+UseConcMarkSweepGC -XX:+PrintGC -XX:+PrintGCDetails -Duser.timezone=Asia/Shanghai -Dlog4j.defaultInitOverride=false -XX:-OmitStackTraceInFastThrow -Djava.security.egd=file:/dev/./urandom

    从配置可以看出，prod-user-provider-k8s使用的CMS垃圾收集器。

    -Xmx8192m
-Xms8192m
-XX:MaxDirectMemorySize=1026m
-XX:NewRatio=4
-XX:NewSize=3096m
-XX:MaxNewSize=3096m
-XX:SurvivorRatio=8
-XX:+UseConcMarkSweepGC

    最终生效的配置

    命令：jmap -heap 1

    命令：jinfo -flags 1

    Non-default VM flags: -XX:CICompilerCount=3 -XX:ConcGCThreads=10 -XX:GCLogFileSize=1073741824 -XX:InitialHeapSize=8589934592 -XX:MaxDirectMemorySize=1075838976 -XX:MaxHeapSize=8589934592 -XX:MaxNewSize=3246391296 -XX:MinHeapDeltaBytes=196608 -XX:NewRatio=4 -XX:NewSize=3246391296 -XX:NumberOfGCLogFiles=1 -XX:OldPLABSize=16 -XX:OldSize=5343543296 -XX:-OmitStackTraceInFastThrow -XX:ParallelGCThreads=10 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:SurvivorRatio=8 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseGCLogFileRotation -XX:+UseParNewGC

    2  定位过程

    1）2021.05.26 21:52  检测服务GC情况，发现最近1小时GC次数较多，且此时没有发布变更。

    2）根据监控，缩小时间范围到2021-05-26 21:10:00 ~ 2021-05-26 21:20:00

    应用：prod-xxx-k8s

    时间段：2021-05-26 21:10:00 ~ 2021-05-26 21:20:00

    172.16.110.244，172.16.102.235，172.16.104.28 三台机器的平均响应事件较长，就取样这3台了。

    3）进入到 172.16.110.244 k8s终端，查看 /home/admin/gc.log，找到21:14分的GC日志（监控显示21:14分有一次FullGC，此时Old区和新生代容量均未满）

    2021-05-26T21:13:46.971+0800: 110332.313: [GC (Allocation Failure) 2021-05-26T21:13:46.972+0800: 110332.313: [ParNew: 2560944K->42868K(2853312K), 0.0163553 secs] 7265031K->4749630K(8071616K), 0.0171585 secs] [Times: user=0.11 sys=0.03, real=0.02 secs]
2021-05-26T21:13:50.840+0800: 110336.181: [GC (Allocation Failure) 2021-05-26T21:13:50.840+0800: 110336.182: [ParNew: 2579188K->49558K(2853312K), 0.0157663 secs] 7285950K->4758763K(8071616K), 0.0165931 secs] [Times: user=0.13 sys=0.00, real=0.02 secs]
2021-05-26T21:13:54.591+0800: 110339.933: [GC (Allocation Failure) 2021-05-26T21:13:54.592+0800: 110339.933: [ParNew: 2585878K->51614K(2853312K), 0.0163846 secs] 7295083K->4761839K(8071616K), 0.0171867 secs] [Times: user=0.10 sys=0.03, real=0.02 secs]
2021-05-26T21:13:58.559+0800: 110343.900: [GC (Allocation Failure) 2021-05-26T21:13:58.560+0800: 110343.901: [ParNew: 2587934K->54453K(2853312K), 0.0152570 secs] 7298159K->4765092K(8071616K), 0.0160738 secs] [Times: user=0.13 sys=0.00, real=0.01 secs]
2021-05-26T21:14:02.507+0800: 110347.848: [GC (Allocation Failure) 2021-05-26T21:14:02.507+0800: 110347.848: [ParNew: 2590773K->52962K(2853312K), 0.0158836 secs] 7301412K->4763945K(8071616K), 0.0168485 secs] [Times: user=0.10 sys=0.03, real=0.01 secs]
2021-05-26T21:14:06.253+0800: 110351.594: [GC (Allocation Failure) 2021-05-26T21:14:06.253+0800: 110351.595: [ParNew: 2589282K->53375K(2853312K), 0.0166401 secs] 7300265K->4764793K(8071616K), 0.0175633 secs] [Times: user=0.14 sys=0.00, real=0.02 secs]
2021-05-26T21:14:10.035+0800: 110355.376: [GC (Allocation Failure) 2021-05-26T21:14:10.035+0800: 110355.377: [ParNew: 2589695K->56544K(2853312K), 0.0160937 secs] 7301113K->4768240K(8071616K), 0.0168443 secs] [Times: user=0.10 sys=0.03, real=0.02 secs]
2021-05-26T21:14:13.680+0800: 110359.021: [GC (Allocation Failure) 2021-05-26T21:14:13.681+0800: 110359.022: [ParNew: 2592864K->57174K(2853312K), 0.0160996 secs] 7304560K->4769381K(8071616K), 0.0169435 secs] [Times: user=0.13 sys=0.00, real=0.02 secs]
2021-05-26T21:14:17.332+0800: 110362.673: [GC (Allocation Failure) 2021-05-26T21:14:17.332+0800: 110362.674: [ParNew: 2593494K->53500K(2853312K), 0.0166459 secs] 7305701K->4766029K(8071616K), 0.0174705 secs] [Times: user=0.11 sys=0.03, real=0.02 secs]
2021-05-26T21:14:21.113+0800: 110366.454: [GC (Allocation Failure) 2021-05-26T21:14:21.114+0800: 110366.455: [ParNew: 2589820K->50429K(2853312K), 0.0164917 secs] 7302349K->4763507K(8071616K), 0.0173216 secs] [Times: user=0.11 sys=0.03, real=0.02 secs]
2021-05-26T21:14:25.024+0800: 110370.365: [GC (Allocation Failure) 2021-05-26T21:14:25.024+0800: 110370.365: [ParNew: 2586749K->56649K(2853312K), 0.0171138 secs] 7299827K->4770415K(8071616K), 0.0179706 secs] [Times: user=0.15 sys=0.00, real=0.02 secs]
2021-05-26T21:14:28.679+0800: 110374.020: [GC (Allocation Failure) 2021-05-26T21:14:28.679+0800: 110374.020: [ParNew: 2592969K->50028K(2853312K), 0.0167487 secs] 7306735K->4764476K(8071616K), 0.0175809 secs] [Times: user=0.13 sys=0.01, real=0.02 secs]
2021-05-26T21:14:32.439+0800: 110377.780: [GC (Allocation Failure) 2021-05-26T21:14:32.440+0800: 110377.781: [ParNew: 2586348K->56920K(2853312K), 0.0160964 secs] 7300796K->4772346K(8071616K), 0.0168883 secs] [Times: user=0.11 sys=0.04, real=0.02 secs]
2021-05-26T21:14:36.294+0800: 110381.636: [GC (Allocation Failure) 2021-05-26T21:14:36.295+0800: 110381.636: [ParNew: 2593240K->52929K(2853312K), 0.0163589 secs] 7308666K->4770160K(8071616K), 0.0172542 secs] [Times: user=0.14 sys=0.00, real=0.02 secs]
2021-05-26T21:14:40.177+0800: 110385.518: [GC (Allocation Failure) 2021-05-26T21:14:40.177+0800: 110385.518: [ParNew: 2589249K->51830K(2853312K), 0.0157497 secs] 7306480K->4771515K(8071616K), 0.0166224 secs] [Times: user=0.11 sys=0.02, real=0.01 secs]
2021-05-26T21:14:44.009+0800: 110389.350: [GC (Allocation Failure) 2021-05-26T21:14:44.010+0800: 110389.351: [ParNew (promotion failed): 2588150K->2572606K(2853312K), 0.7510010 secs]2021-05-26T21:14:44.761+0800: 110390.102: [CMS: 4721703K->375449K(5218304K), 2.8244875 secs] 7307835K->375449K(8071616K), [Metaspace: 151787K->151787K(1189888K)], 3.5763945 secs] [Times: user=3.78 sys=0.16, real=3.57 secs]
2021-05-26T21:14:51.530+0800: 110396.871: [GC (Allocation Failure) 2021-05-26T21:14:51.531+0800: 110396.872: [ParNew: 2536320K->56158K(2853312K), 0.0890703 secs] 2911769K->431607K(8071616K), 0.0908828 secs] [Times: user=0.25 sys=0.00, real=0.09 secs]
2021-05-26T21:14:54.328+0800: 110399.670: [GC (Allocation Failure) 2021-05-26T21:14:54.330+0800: 110399.671: [ParNew: 2592478K->71256K(2853312K), 0.0873448 secs] 2967927K->446705K(8071616K), 0.0897922 secs] [Times: user=0.19 sys=0.06, real=0.09 secs]
2021-05-26T21:14:57.343+0800: 110402.685: [GC (Allocation Failure) 2021-05-26T21:14:57.345+0800: 110402.686: [ParNew: 2607576K->69383K(2853312K), 0.0773853 secs] 2983025K->444833K(8071616K), 0.0800916 secs] [Times: user=0.18 sys=0.00, real=0.08 secs]
2021-05-26T21:15:00.700+0800: 110406.041: [GC (Allocation Failure) 2021-05-26T21:15:00.701+0800: 110406.042: [ParNew: 2605703K->61466K(2853312K), 0.0252316 secs] 2981153K->436916K(8071616K), 0.0275881 secs] [Times: user=0.13 sys=0.05, real=0.03 secs]
2021-05-26T21:15:04.246+0800: 110409.587: [GC (Allocation Failure) 2021-05-26T21:15:04.247+0800: 110409.588: [ParNew: 2597786K->67316K(2853312K), 0.0226887 secs] 2973236K->442765K(8071616K), 0.0248441 secs] [Times: user=0.11 sys=0.04, real=0.02 secs]
2021-05-26T21:15:07.806+0800: 110413.147: [GC (Allocation Failure) 2021-05-26T21:15:07.808+0800: 110413.149: [ParNew: 2603636K->67948K(2853312K), 0.0245455 secs] 2979085K->443398K(8071616K), 0.0269191 secs] [Times: user=0.17 sys=0.00, real=0.02 secs]
2021-05-26T21:15:11.406+0800: 110416.748: [GC (Allocation Failure) 2021-05-26T21:15:11.408+0800: 110416.749: [ParNew: 2604268K->68840K(2853312K), 0.0241876 secs] 2979718K->444289K(8071616K), 0.0264560 secs] [Times: user=0.15 sys=0.00, real=0.02 secs]
2021-05-26T21:15:15.156+0800:
    上一篇：张传宁：.NET6系列：微软宣布Visual Studio 2022【转载】
    下一篇：没有了