一.问题现象
线上服务运行了5天,服务器的CPU使用率突然暴增到146%,顿时感觉情况不是很妙通过top 命令查看服务器运行情况如下:发现这个java程序的CPU使用率很高。
二.解决办法
- 执行top -Hp pid命令获取到当前进程中所有在执行的线程情况,再按大写的P进行排序,找到CPU使用率最高的线程 pid
- 将这个线程的pid输出为16进制
printf "%x\n" 1546#输出的16进制结果60a
3.执行jstack命令查看线程执行情况
jstack 25| grep 0x60a -C5
4. 通过打印的堆栈信息可以看到提示 ProductServiceImpl recommendSimilar 方法 2525行有问题。
- 查看代码(吐槽之前写的恶心代码)
public List<Map<String, Object>> recommendSimilar(String code) {
List<Map<String, Object>> empty = new ArrayList<>();String findOrderProductIdSQL = "select op.product_id from o_order_product op where op.order_code = :orderCode ";Map<String, Object> findOrderProductIdParams = new HashedMap<>();findOrderProductIdParams.put("orderCode", code);List<Long> orderProductIds = jdbcTemplate.queryForList(findOrderProductIdSQL, findOrderProductIdParams, Long.class);if (orderProductIds != null && orderProductIds.size() > 0) {
String aggreateModuleProductIdSQL = "SELECT t.product_id FROM a_aggregate_module_product t where t.aggregate_module_id in (SELECT amp.aggregate_module_id FROM a_aggregate_module_product amp where amp.product_id in (:productIds)) and t.product_id not in (:productIds)";Map<String, Object> aggreateModuleProductIdParams = new HashedMap<>();aggreateModuleProductIdParams.put("productIds", orderProductIds);List<Long> aggreateModuleProductIds = jdbcTemplate.queryForList(aggreateModuleProductIdSQL, aggreateModuleProductIdParams, Long.class);if (aggreateModuleProductIds != null && aggreateModuleProductIds.size() > 0) {
Set<Long> resultProductIds = new HashSet<>();while (resultProductIds.size() < 8) {
resultProductIds.add(aggreateModuleProductIds.get(RandomUtils.nextInt(0, aggreateModuleProductIds.size())));}String resultSQL = "SELECT p.id , p.title, p.min_price, p.max_price, p.original_price, p.currency,p.flag, (SELECT url FROM f_file_storage WHERE id = (SELECT file_storage_id FROM p_product_image WHERE product_id = p.id AND type = 0 AND yn = 1)) AS product_sku_img from p_product p where p.`status` = 10 and p.yn = 1 and p.id in (:resultProductIds)";Map<String, Object> resultParams = new HashedMap<>();resultParams.put("resultProductIds", resultProductIds);return jdbcTemplate.queryForList(resultSQL, resultParams);}}return empty;}
问题原因:
这个地方会出现死循环
while (resultProductIds.size() < 8) {
resultProductIds.add(aggreateModuleProductIds.get(RandomUtils.nextInt(0, aggreateModuleProductIds.size())));}
- 修改 优化,Bug修复完毕
public List<Map<String, Object>> recommendSimilar(String code) {
// 1. 根据订单code查询商品idString findOrderProductIdSQL = "select op.product_id from o_order_product op where op.order_code = :orderCode ";List<Long> orderProductIds = jdbcTemplate.queryForList(findOrderProductIdSQL, ImmutableMap.of("orderCode", code), Long.class);if (CollectionUtil.isEmpty(orderProductIds)) {
return Collections.emptyList();}// 2. 根据商品id 查询该聚合模块下相似商品ipString aggreateModuleProductIdSQL = "SELECT t.product_id FROM a_aggregate_module_product t where t.aggregate_module_id in (SELECT amp.aggregate_module_id FROM a_aggregate_module_product amp where amp.product_id in (:productIds)) and t.product_id not in (:productIds)";ImmutableMap<String, List<Long>> aggreateModuleParams = ImmutableMap.of("productIds", orderProductIds);List<Long> aggreateModuleProductIds = jdbcTemplate.queryForList(aggreateModuleProductIdSQL, aggreateModuleParams, Long.class);if (CollectionUtil.isEmpty(aggreateModuleProductIds)) {
return Collections.emptyList();}// 3. 随机查询8条相似商品信息Collection<Long> resultProductIds = randomElement(aggreateModuleProductIds, 8);String resultSQL = "SELECT p.id , p.title, p.min_price, p.max_price, p.original_price, p.currency,p.flag, (SELECT url FROM f_file_storage WHERE id = (SELECT file_storage_id FROM p_product_image WHERE product_id = p.id AND type = 0 AND yn = 1)) AS product_sku_img from p_product p where p.`status` = 10 and p.yn = 1 and p.id in (:resultProductIds)";Map<String, Object> resultParams = new HashedMap<>();resultParams.put("resultProductIds", resultProductIds);return jdbcTemplate.queryForList(resultSQL, resultParams);}private Collection<Long> randomElement(List<Long> list, int count) {
int size = list.size();// 总数小于 获取个数的3倍,直接取count条if (list.size() < count * 3) {
return list.stream().limit(count).collect(Collectors.toList());}Set<Long> result = new HashSet<>();while (result.size() < count) {
result.add(list.get(RandomUtils.nextInt(0, size)));}return result;}