Crs规则中默认inbound_anomaly_score_threshold=5阈值5
paranoia_level=1 严格等级默认为1，调高会匹配高等级规则
critical_anomaly_score=5命中严重异常规则的请求会加5分
error_anomaly_score=4 命中错误异常规则的请求会加4分
warning_anomaly_score=3 命中警告异常的请求会加3分
notice_anomaly_score=2 命中提示异常的请求会加2分
全部采用默认配置，选用全部默认规则结果如下
产生较多误报
分析误报请求
[12, 26, 28, 32, 39, 51, 56, 58, 89, 91, 94, 99, 111, 133, 140, 149, 152, 164, 197, 199, 209, 218, 226, 227, 229, 232, 236, 263, 276, 281, 288, 290, 307, 312, 315, 323, 330, 336, 341, 352, 353, 355, 383, 389, 391, 392, 399, 402, 403, 411, 412, 416, 428, 438, 454, 467, 472, 483, 484, 489, 491, 497, 498, 499, 501, 503, 526, 538, 549, 575, 587, 595, 596, 599, 608, 616, 618, 625, 651, 655, 657, 659, 677, 699, 736, 764, 767, 781, 785, 797, 845, 854, 884, 892, 908, 925, 932, 955, 959, 977, 982, 986, 997, 1003, 1004, 1048, 1068, 1073, 1095, 1102, 1125, 1160, 1173, 1192, 1193, 1195, 1205, 1215, 1221, 1224, 1235, 1236, 1240, 1259, 1262, 1264, 1270, 1274, 1296, 1314, 1320, 1328, 1332, 1333, 1342, 1362, 1365, 1369, 1380, 1382, 1392, 1403, 1404, 1424, 1445, 1451, 1466, 1477, 1481, 1490, 1509, 1517, 1546, 1552, 1574, 1584, 1595, 1596, 1626, 1627, 1646, 1658, 1691, 1698, 1700, 1731, 1733, 1741, 1756, 1760, 1771, 1777, 1808, 1824, 1831, 1846]
拦截日志如下
2019/09/26 11:46:43 [error] 10616#0: *42 [lua] waf.lua:494: _process_rule(): 943120 collection...0 pattern...0..value....0...rule.var...{["type"]="REQUEST_HEADERS",["length"]=true,["collection_key"]="REQUEST_HEADERS|specific|Referer",["parse"]={[1]="specific",[2]="Referer",},}, client: 10.232.132.53, server: cert.placuna.cn, request: "GET /mobsong_notfound.html HTTP/1.1", host: "cert.placuna.cn:8080"
2019/09/26 11:46:43 [error] 10616#0: *42 [lua] waf.lua:519: _process_rule(): after match rule.actions...{["disrupt"]="SCORE",["nondisrupt"]={[1]={["data"]={["value"]="%{rule.msg}",["key"]="MSG",["col"]="TX",},["action"]="setvar",},[2]={["data"]={["inc"]=true,["value"]="%{TX.CRITICAL_ANOMALY_SCORE}",["key"]="SESSION_FIXATION_SCORE",["col"]="TX",},["action"]="setvar",},[3]={["data"]={["inc"]=true,["value"]="%{TX.CRITICAL_ANOMALY_SCORE}",["key"]="ANOMALY_SCORE_PL1",["col"]="TX",},["action"]="setvar",},[4]={["data"]={["value"]="%{TX.0}",["key"]="943120-OWASP_CRS/WEB_ATTACK/SESSION_FIXATION-REQUEST_HEADERS",["col"]="TX",},["action"]="setvar",},},}, client: 10.232.132.53, server: cert.placuna.cn, request: "GET /mobsong_notfound.html HTTP/1.1", host: "cert.placuna.cn:8080"
从拦截日志中可以看出为id为943120的规则误拦
将规则文件REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf屏蔽掉
再测
误报基本没有
召回率由70.422%下降至66.549%、
研究session会话攻击的规则，id:943120为链式规则，第一步先匹配请求中是否有特殊sessionid字段，如果没有不拦截，如果有进入第二步，判断此请求的header中是否有referer，如果有则不拦截，没有则对此请求增加critical得分=5，默认阈值为5，所以直接触发拦截行为。所以对此逻辑造成误报率过高的初步解决方法是将critical得分改为warning得分3，来降低误报率。
修改后测试结果
相当于把943文件屏蔽掉了。
探究召回率下降的原因。修改后增加的漏报的请求为
[11, 82, 122, 128, 462, 463, 508, 552, 610, 634, 682, 722, 734, 746, 911, 987, 1098, 1138, 1176, 1199, 1223, 1263, 1371, 1439, 1498, 1570, 1614, 1619, 1692, 1727, 1847, 1848, 1849]
即这些请求在未修改前会被943规则阻拦。
（在此过程中发现500错误）
2019/09/27 17:44:45 [error] 7067#0: *5 lua entry thread aborted: runtime error: .../kswaf/openresty20190423/waf_lua_ng/resty/core/regex.lua:374: attempt to get length of local 'subj' (a number value)
原因是在初始化阶段向matched_var中写入数字，后面的等级2规则中需要对.data文件中的字符串和matched_var中的内容进行匹配，匹配方式为refind，这要求其不能为数字，否则会报错。
解决办法：在向写入matched_var写入数据时加一层数据类型校验避免这一错误
继续分析943处理后增加的漏报问题
[11, 82, 122, 128, 462, 463, 508, 552, 610, 634, 682, 722, 734, 746, 911, 987, 1098, 1138, 1176, 1199, 1223, 1263, 1371, 1439, 1498, 1570, 1614, 1619, 1692, 1727, 1847, 1848, 1849]
将943的规则文件调值默认发送上述单个请求。
请求的行号: 11
请求的label: 1
响应状态: 403
method: GET
urlpath: /jobs/jobs-list.php?key=%C7%EB%CA%E4%C8%EB%D6%B0%CE%BB%C3%FB%B3%C6%A1%A2%B9%AB%CB%BE%C3%FB%B3%C6%A1%A2%BC%BC%C4%DC%CC%D8%B3%A4%A1%A2%D1%A7%D0%A3%B5%C8%B9%D8%BC%FC%D7%D6...&Submit=%CB%D1%D6%B0%CE%BB%22%20and%20%22a%22=%22a
postbody: None
headers: {'Cookie': 'PHPSESSID=45a1i8dljhmcp4bfm5bdv7c6u0', 'User-Agent': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; .NET CLR 1.1.4322)'}
其中11,82,122,128,463,508,552,682,610,634,722,734,911,1176,1263,1371, 1498,1570,1614,1692,1727,1847,1848,1849为疑似sql注入请求,加粗部分sql注入在 body中，所以lable为1，但实际拦截为943文件，拦截依据是cookie中含有sessionid字段且无referer。
746,987,1098,1138,1199,1223,1439,1619为疑似会话固定攻击
漏报猜测：sql注入漏报，sql注入规则等级过低。
初步处理方式，单独调高sql注入拦截的等级，看误报是否会减少。
结果
因为每个等级有每个等级的得分，只有当默认等级设置为跟高等级，阻断时才会将各个等级的得分相加，所以单独设置某个规则文件的等级对阻断是没有意义的，但是可以从日志中查看是否会命中高等级的规则。
（调试过程中遇到转码问题，发现1849请求中的unicode在解析过程中没有被转码，导致规则匹配不成功，调试do_transform函数，解决办法：保留缓存命中率高的collection_key
的格式，在第一次遇到缓存未命中时对数据进行全部类型解码，之前进行的是replacetr和alltransform两种解码，造成解码不完全的问题。）
2019/09/29 18:21:02 [error] 12468#0: *11 [lua] waf.lua:496: _process_rule(): 942130 collection...{[1]="45a1i8dljhmcp4bfm5bdv7c6u0",[2]="λ??У...",[3]="λ and a=a",} pattern...(?i:([\s'"`\(\)]*?)([\d\w]++)([\s'"`\(\)]*?)(?:<(?:=(?:([\s'"`\(\)]*?)(?!\2)([\d\w]+)|>([\s'"`\(\)]*?)(?:\2))|>?([\s'"`\(\)]*?)(?!\2)([\d\w]+))|(?:not\s+(?:regexp|like)|is\s+not|>=?|!=|\^)([\s'"`\(\)]*?)(?!\2)([\d\w]+)|(?:(?:sounds\s+)?like|r(?:egexp|like)|=)([\s'"`\(\)]*?)(?:\2)))..value....table: 0x400b8250...rule.var...{["type"]="REQUEST_ARGS",["collection_key"]="REQUEST_ARGS|values|true",["parse"]={[1]="values",[2]=true,},}, client: 10.232.11.16, server: cert.placuna.cn, request: "GET /jobs/jobs-list.php?key=%C7%EB%CA%E4%C8%EB%D6%B0%CE%BB%C3%FB%B3%C6%A1%A2%B9%AB%CB%BE%C3%FB%B3%C6%A1%A2%BC%BC%C4%DC%CC%D8%B3%A4%A1%A2%D1%A7%D0%A3%B5%C8%B9%D8%BC%FC%D7%D6...&Submit=%CB%D1%D6%B0%CE%BB%22%20and%20%22a%22=%22a HTTP/1.1", host: "cert.placuna.cn:8080"
2019/09/29 18:21:02 [error] 12468#0: *11 [lua] waf.lua:521: _process_rule(): after match rule.actions...{["disrupt"]="SCORE",["nondisrupt"]={[1]={["data"]={["value"]="%{rule.msg}",["key"]="MSG",["col"]="TX",},["action"]="setvar",},[2]={["data"]={["inc"]=true,["value"]="%{TX.CRITICAL_ANOMALY_SCORE}",["key"]="SQL_INJECTION_SCORE",["col"]="TX",},["action"]="setvar",},[3]={["data"]={["inc"]=true,["value"]="%{TX.CRITICAL_ANOMALY_SCORE}",["key"]="ANOMALY_SCORE_PL2",["col"]="TX",},["action"]="setvar",},[4]={["data"]={["value"]="%{TX.0}",["key"]="%{RULE.ID}-OWASP_CRS/WEB_ATTACK/SQL_INJECTION-%{MATCHED_VAR_NAME}",["col"]="TX",},["action"]="setvar",},},}, client: 10.232.11.16, server: cert.placuna.cn, request: "GET /jobs/jobs-list.php?key=%C7%EB%CA%E4%C8%EB%D6%B0%CE%BB%C3%FB%B3%C6%A1%A2%B9%AB%CB%BE%C3%FB%B3%C6%A1%A2%BC%BC%C4%DC%CC%D8%B3%A4%A1%A2%D1%A7%D0%A3%B5%C8%B9%D8%BC%FC%D7%D6...&Submit=%CB%D1%D6%B0%CE%BB%22%20and%20%22a%22=%22a HTTP/1.1", host: "cert.placuna.cn:8080"
从日志中发现，sql注入规则文件的二三四等级均有对当前漏报产生critical等级的提醒，证明这些请求对于crs规则来说并不算是漏报，只是crs认为这些请求只有在严格要求的时候才会阻拦，正常模式下是会放过这些请求的。所以目前的调整策略是优先遵从crs规则，后期根据实际场景和业务再进行调整。
继续分析误报请求
[164, 352, 596, 659, 767, 1365, 1731]
149和596为底层默认规则拦截误报。
请求352请求内容及相应状态如下
拦截的日志如下：
分析日志可知，该请求被规则id941130（xss）的var拦截
[2] => table: 0x656b80 {
[type] => "COOKIES"
[collection_key] => "COOKIES|keys|true"
[parse] => table: 0x657000 {
[1] => "keys"
[2] => true
}
}
因为该请求的cookie的key为‘union-indexhtml’，匹配到了 (?i)[\s\S](?:x(?:link:href|html|mlns)|!ENTITY.*?SYSTEM|data:text\/html|pattern(?=.*?=)|formaction|\@import|base64)\b，命中后的action为下，直接加一个critical得分（5）导致拦截。
[3] => table: 0x658bb0 {
[data] => table: 0x659010 {
[value] => "%{TX.CRITICAL_ANOMALY_SCORE}"
[key] => "ANOMALY_SCORE_PL1"
[inc] => true
[col] => "TX"
}
[action] => "setvar"
}
规则解析如下
解决方法：先将此条规则屏蔽，后期研究此条规则（941130）的优化。
解决后误报消除，漏报没有增加。
至此，误报问题解决，接下来研究漏报请求
请求11如下：
urlpath: /jobs/jobs-list.php?key=%C7%EB%CA%E4%C8%EB%D6%B0%CE%BB%C3%FB%B3%C6%A1%A2%B9%AB%CB%BE%C3%FB%B3%C6%A1%A2%BC%BC%C4%DC%CC%D8%B3%A4%A1%A2%D1%A7%D0%A3%B5%C8%B9%D8%BC%FC%D7%D6...&Submit=%CB%D1%D6%B0%CE%BB%22%20and%20%22a%22=%22a
postbody: None
headers: {'Cookie': 'PHPSESSID=45a1i8dljhmcp4bfm5bdv7c6u0', 'User-Agent': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; .NET CLR 1.1.4322)'}
不会被crs拦截，但是sql注入引擎942130会在level2等级对其产生critical得分，尝试将此条规则写入level1,。查看结果漏报减少约10%，但是此条规则会增加10%的误报。误报请求如下{[1]="cnzz_eid=1850184636-1394596016-http%3a%2f%2fwww.yaose.net%2f&ntime=1394596016&cnzz_a=2&sin=none<ime=1394595991292"
会命中“n=n”产生误报
处理方法：将此条规则设置为level2等级，遵从crs本身设定，后期研究此条规则942130，消除误报
除信息泄露等低危请求漏报，剩余漏报如下：
[11, 13, 48, 49, 53, 67, 79, 81, 82, 105, 113, 122, 124, 127, 128, 131, 139, 142, 145, 161, 168, 175, 177, 180, 193, 212, 214, 238, 246, 254, 333, 344, 357, 367, 370, 375, 397, 409, 410, 413, 419, 430, 444, 445, 449, 459, 463, 488, 508, 517, 520, 533, 552, 554, 555, 558, 562, 566, 581, 591, 598, 610, 620, 629, 634, 654, 668, 674, 682, 704, 705, 722, 734, 737, 746, 768, 772, 780, 788, 808, 836, 856, 870, 873, 879, 906, 911, 915, 917, 924, 931, 935, 936, 949, 952, 953, 965, 987, 994, 1013, 1024, 1025, 1027, 1050, 1053, 1062, 1078, 1085, 1088, 1113, 1117, 1121, 1129, 1135, 1138, 1144, 1167, 1176, 1199, 1203, 1214, 1223, 1243, 1263, 1279, 1282, 1315, 1319, 1331, 1351, 1354, 1356, 1360, 1371, 1374, 1401, 1405, 1412, 1415, 1423, 1434, 1439, 1443, 1446, 1449, 1470, 1473, 1479, 1488, 1498, 1529, 1544, 1558, 1560, 1567, 1570, 1571, 1577, 1585, 1591, 1598, 1599, 1610, 1614, 1618, 1619, 1632, 1648, 1652, 1676, 1684, 1685, 1692, 1696, 1701, 1718, 1727, 1737, 1740, 1750, 1751, 1781, 1786, 1794, 1796, 1805, 1810, 1833, 1834, 1840, 1847, 1848, 1849]
此类漏报大部分为sql注入和php代码注入漏报
如1840-1849
请求如下method: GET
urlpath: /user/category/1)%20AND%20(SELECT%204037%20FROM(SELECT%20COUNT(*),CONCAT(CHAR(58,100,114,108,58),(SELECT%20(CASE%20WHEN%20(4037=4037)%20THEN%201%20ELSE%200%20END)),CHAR(58,122,103,111,58),FLOOR(RAND(0)*2))x%20FROM%20information_schema.tables%20GROUP%20BY%20x)a)%20AND%20(9909=9909
postbody: None
headers: {'User-Agent': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)'}
经一系列转码后出现sql注入特殊字符
解决方法：增加base64解码功能。结果如下
减少了漏报，持续优化中。。。