簡介
在項目中,存在傳遞超大 json 數據的場景。直接傳輸超大 json 數據的話,有以下兩個弊端
- 占用網絡帶寬,而有些云產品就是按照帶寬來計費的,間接浪費了錢
- 傳輸數據大導致網絡傳輸耗時較長 為了避免直接傳輸超大 json 數據,可以對 json 數據進行 Gzip 壓縮后,再進行網絡傳輸。
- 請求頭添加 Content-Encoding 標識,傳輸的數據進行過壓縮
- Servlet Filter 攔截請求,對壓縮過的數據進行解壓
- HttpServletRequestWrApper 包裝,把解壓的數據寫入請求體
4.0.0com.oliverequest-uncompression0.0.1-SNAPSHOTjarrequest-uncompressionhttp://maven.Apache.orgorg.springframework.bootspring-boot-starter-parent2.5.14UTF-888org.springframework.bootspring-boot-starter-testtestorg.projectlomboklombokorg.springframework.bootspring-boot-starter-webcom.alibaba.fastjson2fastjson22.0.14commons-iocommons-io2.9.0
創建壓縮工具類
GzipUtils 類提供壓縮解壓相關方法
package com.olive.utils;import com.alibaba.fastjson2.JSON;import com.olive.vo.ArticleRequestVO;import lombok.extern.slf4j.Slf4j;import org.apache.commons.io.FileUtils;import JAVA.io.*;import java.util.zip.GZIPInputStream;import java.util.zip.GZIPOutputStream;@Slf4jpublic class GzipUtils {private static final String GZIP_ENCODE_UTF_8 = "UTF-8";* 字符串壓縮為GZIP字節數組* @param str* @returnpublic static byte[] compress(String str) {return compress(str, GZIP_ENCODE_UTF_8);* 字符串壓縮為GZIP字節數組* @param str* @param encoding* @returnpublic static byte[] compress(String str, String encoding) {if (str == null || str.length() == 0) {return null;ByteArrayOutputStream out = new ByteArrayOutputStream();GZIPOutputStream gzip = null;try {gzip = new GZIPOutputStream(out);gzip.write(str.getBytes(encoding));} catch (IOException e) {log.error("compress>>", e);}finally {if(gzip!=null){try {gzip.close();} catch (IOException e) {return out.toByteArray();* GZIP解壓縮* @param bytes* @returnpublic static byte[] uncompress(byte[] bytes) {if (bytes == null || bytes.length == 0) {return null;ByteArrayOutputStream out = new ByteArrayOutputStream();ByteArrayInputStream in = new ByteArrayInputStream(bytes);GZIPInputStream unGzip = null;try {unGzip = new GZIPInputStream(in);byte[] buffer = new byte[256];int n;while ((n = unGzip.read(buffer)) >= 0) {out.write(buffer, 0, n);} catch (IOException e) {log.error("uncompress>>", e);}finally {if(unGzip!=null){try {unGzip.close();} catch (IOException e) {return out.toByteArray();* 解壓并返回String* @param bytes* @returnpublic static String uncompressToString(byte[] bytes) throws IOException {return uncompressToString(bytes, GZIP_ENCODE_UTF_8);* @param bytes* @returnpublic static byte[] uncompressToByteArray(byte[] bytes) throws IOException {return uncompressToByteArray(bytes, GZIP_ENCODE_UTF_8);* 解壓成字符串* @param bytes 壓縮后的字節數組* @param encoding 編碼方式* @return 解壓后的字符串public static String uncompressToString(byte[] bytes, String encoding) throws IOException {byte[] result = uncompressToByteArray(bytes, encoding);return new String(result);* 解壓成字節數組* @param bytes* @param encoding* @returnpublic static byte[] uncompressToByteArray(byte[] bytes, String encoding) throws IOException {if (bytes == null || bytes.length == 0) {return null;ByteArrayOutputStream out = new ByteArrayOutputStream();ByteArrayInputStream in = new ByteArrayInputStream(bytes);GZIPInputStream unGzip = null;try {unGzip = new GZIPInputStream(in);byte[] buffer = new byte[256];int n;while ((n = unGzip.read(buffer)) >= 0) {out.write(buffer, 0, n);return out.toByteArray();} catch (IOException e) {log.error("uncompressToByteArray>>", e);throw new IOException("解壓縮失敗!");}finally {if(unGzip!=null){unGzip.close();* 將字節流轉換成文件* @param filename* @param data* @throws Exceptionpublic static void saveFile(String filename, byte[] data) throws Exception {FileOutputStream fos = null;try {if (data != null) {String filepath = "/" + filename;File file = new File(filepath);if (file.exists()) {file.delete();fos = new FileOutputStream(file);fos.write(data, 0, data.length);fos.flush();System.out.println(file);}catch (Exception e){throw e;}finally {if(fos!=null){fos.close();
對Request進行包裝
UnZipRequestWrapper 讀取輸入流,然進行解壓;解壓完后,再把解壓出來的數據封裝到輸入流中。
package com.olive.filter;import com.olive.utils.GzipUtils;import lombok.extern.slf4j.Slf4j;import javax.servlet.ReadListener;import javax.servlet.ServletInputStream;import javax.servlet.http.HttpServletRequest;import javax.servlet.http.HttpServletRequestWrapper;import java.io.*;* Json String 經過壓縮后保存為二進制文件 -> 解壓縮后還原成 Jso nString轉換成byte[]寫回body中@Slf4jpublic class UnZipRequestWrapper extends HttpServletRequestWrapper {private final byte[] bytes;public UnZipRequestWrapper(HttpServletRequest request) throws IOException {super(request);try (BufferedInputStream bis = new BufferedInputStream(request.getInputStream());ByteArrayOutputStream baos = new ByteArrayOutputStream()) {final byte[] body;byte[] buffer = new byte[1024];int len;while ((len = bis.read(buffer)) > 0) {baos.write(buffer, 0, len);body = baos.toByteArray();if (body.length == 0) {log.info("Body無內容,無需解壓");bytes = body;return;this.bytes = GzipUtils.uncompressToByteArray(body);} catch (IOException ex) {log.error("解壓縮步驟發生異常!", ex);throw ex;@Overridepublic ServletInputStream getInputStream() throws IOException {final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);return new ServletInputStream() {@Overridepublic boolean isFinished() {return false;@Overridepublic boolean isReady() {return false;@Overridepublic void setReadListener(ReadListener readListener) {public int read() throws IOException {return byteArrayInputStream.read();@Overridepublic BufferedReader getReader() throws IOException {return new BufferedReader(new InputStreamReader(this.getInputStream()));
定義GzipFilter對請求進行攔截
GzipFilter 攔截器根據請求頭是否包含Content-Encoding=application/gzip,如果包含就對數據進行解壓;否則就直接放過。
package com.olive.filter;import lombok.extern.slf4j.Slf4j;import org.springframework.stereotype.Component;import javax.servlet.*;import javax.servlet.http.HttpServletRequest;import java.io.IOException;* 解壓filter@Slf4j@Componentpublic class GzipFilter implements Filter {private static final String CONTENT_ENCODING = "Content-Encoding";private static final String CONTENT_ENCODING_TYPE = "application/gzip";@Overridepublic void init(FilterConfig filterConfig) throws ServletException {log.info("init GzipFilter");@Overridepublic void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {long start = System.currentTimeMillis();HttpServletRequest httpServletRequest = (HttpServletRequest)request;String encodeType = httpServletRequest.getHeader(CONTENT_ENCODING);if (encodeType!=null && CONTENT_ENCODING_TYPE.equals(encodeType)) {log.info("請求:{} 需要解壓", httpServletRequest.getRequestURI());UnZipRequestWrapper unZipRequest = new UnZipRequestWrapper(httpServletRequest);chain.doFilter(unZipRequest, response);}else {log.info("請求:{} 無需解壓", httpServletRequest.getRequestURI());chain.doFilter(request,response);log.info("耗時:{}ms", System.currentTimeMillis() - start);@Overridepublic void destroy() {log.info("destroy GzipFilter");
注冊 GzipFilter 攔截器package com.olive.config;import com.olive.filter.GzipFilter;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.web.servlet.FilterRegistrationBean;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;* 注冊filter@Configurationpublic class FilterRegistration {@Autowiredprivate GzipFilter gzipFilter;@Beanpublic FilterRegistrationBean gzipFilterRegistrationBean() {FilterRegistrationBean registration = new FilterRegistrationBean<>();//Filter可以new,也可以使用依賴注入Beanregistration.setFilter(gzipFilter);//過濾器名稱registration.setName("gzipFilter");//攔截路徑registration.addUrlPatterns("/*");//設置順序registration.setOrder(1);return registration;
定義 Controller
該 Controller 非常簡單,主要是輸入請求的數據
package com.olive.controller;import java.util.HashMap;import java.util.Map;import com.alibaba.fastjson2.JSON;import com.olive.vo.ArticleRequestVO;import org.springframework.web.bind.annotation.RequestBody;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RestController;@RestControllerpublic class TestController {@RequestMapping("/getArticle")public Map getArticle(@RequestBody ArticleRequestVO articleRequestVO){Map result = new HashMap<>();result.put("code", 200);result.put("msg", "success");System.out.println(JSON.toJSONString(articleRequestVO));return result;
Controller 參數接收VO
package com.olive.vo;import lombok.Data;import java.io.Serializable;@Datapublic class ArticleRequestVO implements Serializable {private Long id;private String title;private String content;
定義 Springboot 引導類package com.olive;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;@SpringBootApplicationpublic class Application {public static void main(String[] args) {SpringApplication.run(Application.class);
測試
- 非壓縮請求測試
curl -X POST http://127.0.0.1:8080/getArticle -H 'content-type: application/json' "id":1,"title": "java樂園","content":"xxxxxxxxxx"
- 壓縮請求測試
不要直接將壓縮后的 byte[] 數組當作字符串進行傳輸,否則壓縮后的請求數據比沒壓縮后的還要大得多! 項目中一般采用以下兩種傳輸壓縮后的 byte[] 的方式:
- 將壓縮后的 byet[] 進行 Base64 編碼再傳輸字符串,這種方式會損失掉一部分 GZIP 的壓縮效果,適用于壓縮結果要存儲在 redis 中的情況
- 將壓縮后的 byte[] 以二進制的形式寫入到文件中,請求時直接在 body 中帶上文件即可,用這種方式可以不損失壓縮效果
小編測試采用第二種方式,采用以下代碼把原始數據進行壓縮
public static void main(String[] args) {ArticleRequestVO vo = new ArticleRequestVO();vo.setId(1L);vo.setTitle("bug弄潮兒");try {byte[] bytes = FileUtils.readFileToByteArray(new File("C:\Users\2230\Desktop\凱平項目資料\改裝車項目\CXSSBOOT_DB_DDL-1.0.9.sql"));vo.setContent(new String(bytes));byte[] dataBytes = compress(JSON.toJSONString(vo));saveFile("d:/vo.txt", dataBytes);} catch (Exception e) {e.printStackTrace();
壓縮后數據存儲到d:/vo.txt,然后在 postman 中安裝下圖選擇