一、kv介紹

用于解析key=value類型的消息，可以配置任意字符串來拆分數據，不一定非要用=符號，kv對的間隔也不一定非要用空格

二、allow_duplicate_values

• 功能：允許重復鍵值對
• 介紹：默認為true，兩個相同的鍵值對都會被放到數組中，如果設置為false，則相同的鍵值對只會顯示一個

filter{kv {source => "message"# 允許重復鍵值對allow_duplicate => "true"} } {"age": ["20","20"],"message": "name=瑞文 age=20 name=瑞文 age=20","name": ["瑞文","瑞文"] } filter{kv {source => "message"# 不允許重復鍵值對allow_duplicate => "false"} } {"age": "20","message": "name=瑞文 age=20 name=瑞文 age=20","name": "瑞文", }

三、allow_empty_values

• 功能：允許空值
• 介紹：默認為false，不允許空值，如果配置為true，則可以匹配name=這樣的字符

filter{kv {source => "message"# 允許空字段allow_empty_values => "true"} } {"message": "name=","name": "", } filter{kv {source => "message"# 不允許空字段allow_empty_values => "false"} } {"message": "name=" }

四、default_keys

• 功能：添加默認的key
• 介紹：如果匹配的內容中不含有指定的key，就將key添加到字段中，指定的數組內元素數量必須為雙數，單數下標的元素是value，雙數下標的元素是key（0基）

filter{kv {source => "message"# 添加默認鍵值對default_keys => ["from","123@com"]} } {"from": "123@com","message": "name=奧拉夫","name": "奧拉夫" } filter{kv {source => "message"} } {"message": "name=奧拉夫","name": "奧拉夫" }

五、recursive

• 功能：遞歸解析字段
• 介紹：默認為false，如果設置為true，字段中只要還有=符號就一直解析，放到子字段中

filter{kv {source => "message"# 遞歸解析字段recursive => "true"} } {"message" => "name=泰達米爾 age=20 type=type1=type2=戰士","name" => "泰達米爾","age" => "20","type" => {"type1" => {"type2" => "戰士"}} } filter{kv {source => "message"remove_field => ["log","@timestamp","@version","tags","service","host","event"]} } {"message" => "name=泰達米爾 age=20 type=type1=type2=戰士","name" => "泰達米爾","age" => "20","type" => "type1=type2=戰士" }

六、field_split

• 功能：字段拆分字符
• 介紹：默認為" "，可以更改拆分鍵值對的間隔字符，用正則表示，可以指定多個字符，多個字符是或的關系

filter{kv {source => "message"# 將鍵值對的分隔符修改為&field_split => "&"} } {"sex": "男","message": "name=泰隆&age=19&sex=男","name": "泰隆","age": "19" } filter{kv {source => "message"} } {"message": "name=泰隆&age=19&sex=男","name": "泰隆&age=19&sex=男" } filter{kv {source => "message"# 使用&或者?作為分隔字符field_split => "&?"} } {"sex": "男","message": "name=泰隆&age=19&sex=男?type=刺客","name": "泰隆","age": "19","type": "刺客" }

七、field_split_pattern

• 功能：字段分隔正則匹配模式
• 介紹：優先級高于field_split，可以自定義需要正則模式匹配鍵值對間隔

filter{kv {source => "message"# 匹配一個或多個:field_split_pattern => ":+"} } {"sex": "男","message": "name=泰隆:age=19::sex=男::::::type=刺客","name": "泰隆","age": "19","type": "刺客" } filter{kv {source => "message"# 匹配兩個+field_split_pattern => "\+\+"} } {"k4" => "v4","message" => "k1=v1++k2=v2++k3=v3++k4=v4","k3" => "v3","k2" => "v2","k1" => "v1" }

八、include_brackets

• 功能：排除括號
• 介紹：默認為true，會將(123)這種value識別為123，忽略兩遍的括號

filter{kv {source => "message"# 排除字段中的括號include_brackets => "true"} } {"age" => "20","message" => "name=(泰達米爾) age=(20)","name" => "泰達米爾" } filter{kv {source => "message"# 不排除字段中的括號include_brackets => "false"} } {"age" => "(20)","message" => "name=(泰達米爾) age=(20)","name" => "(泰達米爾)" }

九、exclude_keys

• 功能：排除字段
• 介紹：在匹配到的鍵值對中，把指定key的鍵值對排除掉

filter{kv {source => "message"# 排除age字段exclude_keys => ["age"]} } {"sex": "男","message": "name=泰隆 age=19 sex=男","name": "泰隆" } filter{kv {source => "message"} } {"sex": "男","age": "19","message": "name=泰隆 age=19 sex=男","name": "泰隆" }

十、include_keys

• 功能：添加字段
• 介紹：默認為匹配到的全體數組，如果設置了這一項，則只顯示這一項配置的匹配到的字段

filter{kv {source => "message"# 只顯示匹配到的name和age字段include_keys => ["name","age"]} } {"age" => "20","message" => "name=泰達米爾 age=20 type=戰士","name" => "泰達米爾" } filter{kv {source => "message"} } {"message" => "name=泰達米爾 age=20 type=戰士","name" => "泰達米爾","age" => "20","type" => "戰士" }

十一、prefix

• 功能：添加字段前綴
• 介紹：設置一個字符串，可以添加到所有匹配到的key中

filter{kv {source => "message"# 在匹配到的key中添加test_前綴prefix => "test_"} } {"message" => "name=泰達米爾 age=20 type=戰士","test_age" => "20","test_type" => "戰士","test_name" => "泰達米爾" } filter{kv {source => "message"} } {"message" => "name=泰達米爾 age=20 type=戰士","name" => "泰達米爾","age" => "20","type" => "戰士" }

十二、remove_char_key

• 功能：移除key中的字符串
• 介紹：指定符號，刪除key中的這些符號，支持正則表達式

filter{kv {source => "message"# 刪除key中包含的指定字符remove_char_key => "+-"} } {"type" => "戰士","message" => "+name=泰達米爾 a-ge=20 ty+-pe=戰士","name" => "泰達米爾","age" => "20" }

十三、remove_char_value

• 功能：移除value中的字符串
• 介紹：指定符號，刪除value中的這些符號，支持正則表達式

filter{kv {source => "message"# 刪除value中包含的指定符號remove_char_value => "<>"} } {"message" => "name=泰達<米爾 age=2>0 type=戰<>士","name" => "泰達米爾","type" => "戰士","age" => "20" }

十四、source

• 功能：指定要執行key=value的字段
• 介紹：指定一個字段按照key=value進行解析，默認解析message字段，也可以指定其他字段

input{syslog{port => "514"# 添加一個字段add_field => {"test" => "testKey=testValue"}}} filter{kv {# 將解析字段變為testsource => "test"} } {"message" => "name=泰達米爾 age=20 type=戰士","test" => "testKey=testValue","testKey" => "testValue" }

十五、target

• 功能：目標字段
• 介紹：將key=value解析出來的結果放到指定的字段下，默認鍵值對在最外層

filter{kv {source => "message"# 將解析出來的結果保存到test字段下target => "test"} } {"message" => "name=泰達米爾 age=20 type=戰士","test" => {"name" => "泰達米爾","age" => "20","type" => "戰士"} }

十六、transform_key

• 功能：改變key
• 介紹：可選值：lowercase、uppercase、capitalize，將key轉換為選擇的模式

filter{kv {source => "message"# 將key中的字母變為小寫transform_key => "uppercase"} } {"message" => "name=泰達米爾 age=20 type=戰士","AGE" => "20","NAME" => "泰達米爾","TYPE" => "戰士" }

十七、transform_value

• 功能：改變value
• 介紹：可選值：lowercase、uppercase、capitalize，將value轉換為選擇的模式

filter{kv {source => "message"# 將value中的字母變為大寫transform_value => "uppercase"} } {"message" => "name=泰達米爾 age=20 type=戰士 sex=m","name" => "泰達米爾","type" => "戰士","sex" => "M","age" => "20" }

十八、trim_key

• 功能：修建key字段
• 介紹：類似于strip，可以自定義字符，支持正則，將key前后包含的指定字符刪除

filter{kv {source => "message"# 將key中的指定字符刪除trim_key => "<>"} } {"message" => "<name=泰達米爾 <age>=20 type=戰士","name" => "泰達米爾","type" => "戰士","age" => "20" } filter{kv {source => "message"} } {"message" => "<name=泰達米爾 <age>=20 type=戰士","<name" => "泰達米爾","type" => "戰士","<age>" => "20" }

十九、trim_value

• 功能：修剪value字段
• 介紹：類似于strip，可以自定義字符，支持正則，將value前后包含的指定字符刪除

filter{kv {source => "message"# 將value中的指定字符刪除trim_value => "<>"} } {"message" => "name=泰達米爾<> age=<20> type=<><戰士","name" => "泰達米爾","type" => "戰士","age" => "20" } filter{kv {source => "message"} } {"message" => "name=泰達米爾<> age=<20> type=<><戰士","name" => "泰達米爾<>","type" => "<><戰士","age" => "20" }

二十、value_split

• 功能：鍵值對分隔符
• 介紹：默認按照=符號拆分，可以更改拆分的符號，支持正則

filter{kv {source => "message"# 將key、value按照:分隔value_split => ":"} } {"message" => "name:泰達米爾 age:20 type:戰士","name" => "泰達米爾","type" => "戰士","age" => "20" }

二十二、value_split_pattern

• 功能：設置多字符鍵值對分隔符
• 介紹：value_split的升級版，可以支持多個字符作為分隔符，優先級高魚value_split

filter{kv {source => "message"# 將鍵值對按照多個:進行匹配value_split_pattern => ":+"} } {"message" => "name::::泰達米爾 age:::20 type:戰士","name" => "泰達米爾","type" => "戰士","age" => "20" }

二十三、whitespace

• 功能：設置鍵值對匹配的空格模式
• 介紹：可選值為：lenient、strict，默認是lenient，等號兩邊有空格也可以匹配，如果改為strict，等號兩邊有空格就匹配不上了

filter{kv {source => "message"# 寬松模式匹配空格whitespace => "lenient"} } {"message" => "name= 泰達米爾 age= 20 type=戰士","name" => "泰達米爾","type" => "戰士","age" => "20" } filter{kv {source => "message"# 嚴格模式匹配空格whitespace => "strict"} } {"message" => "name= 泰達米爾 age= 20 type=戰士","type" => "戰士" }

總結

以上是生活随笔為你收集整理的Logstash~filter.kv插件使用教程（附带示例）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。