大家好，我是烤鴨：

? 最近用 springcloud gateway 時，想使用類似 logback-access的功能，用來做數據統計和圖表繪制等等，發現沒有類似的功能，只能自己開發了。

環境：

<dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter</artifactId></dependency><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter-gateway</artifactId></dependency>

整體思路

logback-access.jar 只需要在logback.xml 配置 LogstashAccessTcpSocketAppender 即可完成異步的日志上報。

如果采用相同的方式，考慮到同一個進程里異步上報占性能。(其實是開發太麻煩了)

這里采用的本地日志文件 + elk。

仿照 logback-access ，定義要收集的字段，開發過濾器收集字段，自定義 logstash.yml。

收集到的字段：
“User-Agent” ： 請求頭字段
“server_ip” ：服務器ip
“Content-Length” ： 請求參數長度
“request_uri” ：請求路徑(網關轉發的路徑)
“host” ：本機ip
“client_ip” ：請求ip
“method” ：get/post
“Host” : 請求頭字段
“params” ：請求參數
“request_url” ：請求全路徑
“thread_name” ：當前線程
“level” ：日志級別
“cost_time” ： 請求耗時
“logger_name” ：日志類
“Protocol” ： 請求頭字段

代碼實現

LoggingFilter

package com.xxx.gateway.filter;import com.alibaba.fastjson.JSONObject; import com.google.common.collect.Maps; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.StringUtils; import org.springframework.cloud.gateway.filter.GatewayFilterChain; import org.springframework.cloud.gateway.filter.GlobalFilter; import org.springframework.core.Ordered; import org.springframework.core.io.buffer.DataBuffer; import org.springframework.core.io.buffer.DataBufferUtils; import org.springframework.http.HttpHeaders; import org.springframework.http.HttpMethod; import org.springframework.http.server.reactive.ServerHttpRequest; import org.springframework.http.server.reactive.ServerHttpRequestDecorator; import org.springframework.stereotype.Component; import org.springframework.util.CollectionUtils; import org.springframework.util.MultiValueMap; import org.springframework.web.server.ServerWebExchange; import reactor.core.publisher.Flux; import reactor.core.publisher.Mono;import java.nio.charset.StandardCharsets; import java.util.Map;@Slf4j @Component public class LoggingFilter implements GlobalFilter, Ordered {private static final String UNKNOWN = "unknown";private static final String METHOD = "method";private static final String PARAMS = "params";private static final String REQUEST_URI = "request_uri";private static final String REQUEST_URL = "request_url";private static final String CLIENT_IP = "client_ip";private static final String SERVER_IP = "server_ip";private static final String HOST = "Host";private static final String COST_TIME = "cost_time";private static final String CID = "cid";private static final String CONTENT_LENGTH = "Content-Length";private static final String PROTOCOL = "Protocol";private static final String REQID = "reqid";private static final String USER_AGENT = "User-Agent";private static final String START_TIME = "gw_start_time";private static final String LOGINFOCOLLECTOR = "logInfoCollector";/*** Process the Web request and (optionally) delegate to the next {@code WebFilter}* through the given {@link GatewayFilterChain}.** @param exchange the current server exchange* @param chain provides a way to delegate to the next filter* @return {@code Mono<Void>} to indicate when request processing is complete*/@Overridepublic Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {ServerHttpRequest request = exchange.getRequest();String requestUrl = request.getPath().toString();Map<String, Object> logInfoCollector = Maps.newLinkedHashMap();logInfoCollector.put(CLIENT_IP, getIpAddress(request));logInfoCollector.put(SERVER_IP, request.getURI().getHost());logInfoCollector.put(HOST, getHeaderValue(request, HOST));logInfoCollector.put(METHOD, request.getMethodValue());logInfoCollector.put(REQUEST_URI, request.getURI().getPath());logInfoCollector.put(REQUEST_URL, getRequestUrl(request));logInfoCollector.put(PARAMS, request.getURI().getQuery());logInfoCollector.put(CID, getHeaderValue(request, CID));logInfoCollector.put(CONTENT_LENGTH, request.getHeaders().getContentLength());logInfoCollector.put(PROTOCOL, getHeaderValue(request, PROTOCOL));logInfoCollector.put(REQID, getHeaderValue(request, REQID));logInfoCollector.put(USER_AGENT, getHeaderValue(request, USER_AGENT));exchange.getAttributes().put(START_TIME, System.currentTimeMillis());exchange.getAttributes().put(LOGINFOCOLLECTOR, logInfoCollector);String requestMethod = request.getMethodValue();String contentType = exchange.getRequest().getHeaders().getFirst(HttpHeaders.CONTENT_TYPE);String contentLength = exchange.getRequest().getHeaders().getFirst(HttpHeaders.CONTENT_LENGTH);if (HttpMethod.POST.toString().equals(requestMethod) || HttpMethod.PUT.toString().equals(requestMethod)) {// 根據請求頭，用不同的方式解析Bodyif ((Character.DIRECTIONALITY_LEFT_TO_RIGHT + "").equals(contentLength) || StringUtils.isEmpty(contentType)) {MultiValueMap<String, String> getRequestParams = request.getQueryParams();log.info("\n 請求url:`{}` \n 請求類型：{} \n 請求參數：{}", requestUrl, requestMethod, getRequestParams);return chain.filter(exchange);}Mono<DataBuffer> bufferMono = DataBufferUtils.join(exchange.getRequest().getBody());return bufferMono.flatMap(dataBuffer -> {byte[] bytes = new byte[dataBuffer.readableByteCount()];dataBuffer.read(bytes);String postRequestBodyStr = new String(bytes, StandardCharsets.UTF_8);if (contentType.startsWith("multipart/form-data")) {log.info("\n 請求url:`{}` \n 請求類型：{} \n 文件上傳", requestMethod);} else {log.info("\n 請求url:`{}` \n 請求類型：{} \n 請求參數：{}", requestMethod, postRequestBodyStr);}// 后續需要用到參數的可以從這個地方獲取exchange.getAttributes().put("POST_BODY", postRequestBodyStr);logInfoCollector.put(PARAMS, postRequestBodyStr);DataBufferUtils.release(dataBuffer);Flux<DataBuffer> cachedFlux = Flux.defer(() -> {DataBuffer buffer = exchange.getResponse().bufferFactory().wrap(bytes);return Mono.just(buffer);});// 下面的將請求體再次封裝寫回到request里，傳到下一級，否則，由于請求體已被消費，后續的服務將取不到值ServerHttpRequest mutatedRequest = new ServerHttpRequestDecorator(exchange.getRequest()) {@Overridepublic Flux<DataBuffer> getBody() {return cachedFlux;}};// 封裝request，傳給下一級return chain.filter(exchange.mutate().request(mutatedRequest).build()).then(Mono.fromRunnable(() -> {Long startTime = exchange.getAttribute(START_TIME);Map<String, Object> logInfo = exchange.getAttribute(LOGINFOCOLLECTOR);if (startTime != null && !CollectionUtils.isEmpty(logInfo)) {Long executeTime = (System.currentTimeMillis() - startTime);logInfo.put(COST_TIME, executeTime);log.info(JSONObject.toJSONString(logInfo));}}));});} else if (HttpMethod.GET.toString().equals(requestMethod)|| HttpMethod.DELETE.toString().equals(requestMethod)) {return chain.filter(exchange).then(Mono.fromRunnable(() -> {Long startTime = exchange.getAttribute(START_TIME);Map<String, Object> logInfo = exchange.getAttribute(LOGINFOCOLLECTOR);if (startTime != null && !CollectionUtils.isEmpty(logInfo)) {Long executeTime = (System.currentTimeMillis() - startTime);logInfo.put(COST_TIME, executeTime);log.info(JSONObject.toJSONString(logInfo));}}));}return chain.filter(exchange);}public String getIpAddress(ServerHttpRequest request) {HttpHeaders headers = request.getHeaders();String ip = headers.getFirst("x-forwarded-for");if (StringUtils.isNotBlank(ip) && !UNKNOWN.equalsIgnoreCase(ip)) {// 多次反向代理后會有多個ip值，第一個ip才是真實ipif (ip.indexOf(",") != -1) {ip = ip.split(",")[0];}}if (StringUtils.isBlank(ip) || UNKNOWN.equalsIgnoreCase(ip)) {ip = headers.getFirst("Proxy-Client-IP");}if (StringUtils.isBlank(ip) || UNKNOWN.equalsIgnoreCase(ip)) {ip = headers.getFirst("WL-Proxy-Client-IP");}if (StringUtils.isBlank(ip) || UNKNOWN.equalsIgnoreCase(ip)) {ip = headers.getFirst("HTTP_CLIENT_IP");}if (StringUtils.isBlank(ip) || UNKNOWN.equalsIgnoreCase(ip)) {ip = headers.getFirst("HTTP_X_FORWARDED_FOR");}if (StringUtils.isBlank(ip) || UNKNOWN.equalsIgnoreCase(ip)) {ip = headers.getFirst("X-Real-IP");}if (StringUtils.isBlank(ip) || UNKNOWN.equalsIgnoreCase(ip)) {ip = request.getRemoteAddress().getAddress().getHostAddress();}return ip;}private String getRequestUrl(ServerHttpRequest request) {String url = request.getURI().toString();if (url.contains("?")) {url = url.substring(0, url.indexOf("?"));}return url;}private String getHeaderValue(ServerHttpRequest request, String key) {if (StringUtils.isEmpty(key)) {return "";}HttpHeaders headers = request.getHeaders();if (headers.containsKey(key)) {return headers.get(key).get(0);}return "";}/*** Get the order value of this object.* <p>Higher values are interpreted as lower priority. As a consequence,* the object with the lowest value has the highest priority (somewhat* analogous to Servlet {@code load-on-startup} values).* <p>Same order values will result in arbitrary sort positions for the* affected objects.** @return the order value* @see #HIGHEST_PRECEDENCE* @see #LOWEST_PRECEDENCE*/@Overridepublic int getOrder() {return HIGHEST_PRECEDENCE;} }

logstash.yml

input {file {path => "D:/data/logs/ccc-gateway/*.log"type => "ccc-gateway"codec => json {charset => "UTF-8"}} }filter {json {source => "message"skip_on_invalid_json => trueadd_field => { "@accessmes" => "%{message}" } remove_field => [ "@accessmes" ]} }output {elasticsearch {hosts => "localhost:9200" index => "ccc-gateway_%{+YYYY.MM.dd}"} }

上面的 logstash.yml 兼容 json和非json格式，loggingFilter 會保證數據打印為json格式，其他的地方log也可以是非json的。

效果如圖

accesslog：

其他的log：

圖表繪制

其實netty 作為容器本身也是有 acesslog的，可以開啟。

-Dreactor.netty.http.server.accessLogEnabled=true

AccessLog的log方法直接通過logger輸出日志，其日志格式為COMMON_LOG_FORMAT({} - {} [{}] "{} {} {}" {} {} {} {} ms)，分別是address, user, zonedDateTime, method, uri, protocol, status, contentLength, port, duration

沒有請求參數和自定義參數(一般鏈路id放在請求頭里的)和響應參數(這次也沒加)，所以算是對accesslog做了改進。下圖是訪問量和平均耗時，后續還可以加tp99，請求路徑等等

訪問量：

平均耗時：

總結

以上是生活随笔為你收集整理的springcloud gateway 自定义 accesslog elk的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。