本文參考博客：https://blog.csdn.net/henshuia/article/details/111498753?utm_medium=distribute.pc_relevant.none-task-blog-2_defaultbaidujs_title~default-0.pc_relevant_default&spm=1001.2101.3001.4242.1&utm_relevant_index=3
再次感謝作者分享！

實現(xiàn)敏感詞過濾，一般都是遍歷敏感詞庫然后一個一個的用敏感詞進(jìn)行匹配，但是這樣效率不高并且如果敏感詞中間摻雜特殊符號或者空格就匹配不到了，其次高級點使用正則匹配，但是正則匹配效率低，所有就使用DFA算法解決。

DFA簡介
在實現(xiàn)文字過濾的算法中，DFA是唯一比較好的實現(xiàn)算法。DFA即Deterministic Finite Automaton，也就是確定有窮自動機(jī)，它是是通過event和當(dāng)前的state得到下一個state，即event+state=nextstate。下圖展示了其狀態(tài)的轉(zhuǎn)換

如果我們敏感詞是：黃色、黃賭毒、黃色絲襪、絲襪，則數(shù)據(jù)結(jié)構(gòu)如下所示

每次過濾文章的時候先從整片文章從第一個字開始往后過濾，如果當(dāng)前這個字存在根節(jié)點下則用這個字后面字個這個節(jié)點下一個節(jié)點匹配，如果存在并且end=1則是敏感詞。
首先創(chuàng)建對敏感詞處理，按格式寫入

private SoftReference<Map<String, Map>> sensitiveWordReference=null;private void setWordMap(List<String> result){Map<String, Map> sensitiveWordMap=new HashMap<>(result.size());Iterator<String> iterator = result.iterator();String key = null;Map nowMap = null;while(iterator.hasNext()){nowMap=sensitiveWordMap;key = iterator.next();for(int i=0;i< key.length();i++){char keyChar = key.charAt(i);if(nowMap.containsKey(keyChar)){nowMap=(Map)nowMap.get(keyChar);}else{Map<String,String>newWorMap = new HashMap();newWorMap.put("end", "0");nowMap.put(keyChar, newWorMap);nowMap = newWorMap;}if(i==(key.length()-1)){nowMap.put("end","1");}}}log.info("敏感詞組合{}", JSONObject.toJSONString(sensitiveWordMap));sensitiveWordReference=new SoftReference<Map<String, Map>>(sensitiveWordMap);}

敏感詞是：毒品,黑色,黃色絲襪,色情,絲襪,黃色,黃賭毒

敏感詞組合 {"黑": {"色": {"end": "1"},"end": "0"},"毒": {"品": {"end": "1"},"end": "0"},"色": {"情": {"end": "1"},"end": "0"},"黃": {"色": {"end": "1","絲": {"end": "0","襪": {"end": "1"}}},"end": "0","賭": {"毒": {"end": "1"},"end": "0"}},"絲": {"end": "0","襪": {"end": "1"}} }

檢查并返回匹配到的第一個敏感詞

private String checkWords(String sensitiveWords){if(StringUtils.isEmpty(sensitiveWords)){return StringUtils.EMPTY;}Map<String, Map> sensitiveWordMap=sensitiveWordReference.get();Map nowMap = null;StringBuffer sb=new StringBuffer();for(int i =0; i < sensitiveWords.length(); i++){if(!Character.toString(sensitiveWords.charAt(i)).matches("[\\u4E00-\\u9FA5]+")) {continue;}if(!sensitiveWordMap.containsKey(sensitiveWords.charAt(i))){continue;}sb=new StringBuffer().append(sensitiveWords.charAt(i));nowMap=sensitiveWordMap.get(sensitiveWords.charAt(i));int sensitiveWordLength=i;while (sensitiveWordLength<MAX_MAP_LENGTH){sensitiveWordLength++;if(!Character.toString(sensitiveWords.charAt(sensitiveWordLength)).matches("[\\u4E00-\\u9FA5]+")) {sb.append(sensitiveWords.charAt(sensitiveWordLength));continue;}nowMap=(Map)nowMap.get(sensitiveWords.charAt(sensitiveWordLength));if(CollectionUtils.isEmpty(nowMap)){break;}sb.append(sensitiveWords.charAt(sensitiveWordLength));if(nowMap.get("end").equals("1")){return sb.toString();}}}return sb.toString();}

本文增加了對特殊字符的處理，匹配敏感詞的時候會忽略特殊字符只匹配漢字。
執(zhí)行輸入檢查的文章：發(fā) 絲 0 襪送的發(fā)達(dá)色法士大夫黃 的色是打發(fā)士大 夫是
返回結(jié)果敏感詞是：絲 0 襪

總結(jié)

以上是生活随笔為你收集整理的java敏感词过滤算法的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。