Please wait a minute...
浙江大学学报(工学版)
J4  2009, Vol. 43 Issue (09): 1585-1590    DOI: 10.3785/j.issn.1008973X.2009.09.007
自动化技术、计算机技术     
高速网络环境下数据的压缩存储
张宝军1,平玲娣1,潘雪增1,王界兵2
(1.浙江大学 计算机科学与技术学院,浙江 杭州 310027；2. 汉帆（杭州）信息技术有限公司,浙江 杭州 310012)
Data compression and storage under high speed network
ZHANG Bao-jun1, PING Ling-di1, PAN Xue-zeng1, WANG Jie-bing2
(1.College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China;
2.Hifn (Hangzhou) Information Technology Limited Company, Hangzhou 310012, China) 
 全文: PDF(969 KB)   HTML
摘要：

针对高速网络环境下数据的压缩存储问题,提出一种实用的解决方案,该方案采用软硬件相结合的压缩存储方法,在不影响网络速度的前提下,可以对数据进行实时的压缩存储.方案在硬件上使用汉帆公司（HIFN）的DR系列高性能无损数据压缩卡,该卡由多个9 630压缩芯片组成,采用拥有汉帆自主专利的增强型LZS技术,能够在满足压缩比的前提下提供快速的数据加解压功能.当硬件出现故障时,该方案将采用设备驱动程序中内嵌的eLZS软件模块对数据进行无缝的加解压处理,以保障系统的正常运行.为了使DR压缩卡在Windows环境下有效工作,开发DR压缩卡的Windows驱动程序.该方案在Linux及Windows环境下得到实现,在测试过程中表现出较好的性能.
Abstract:

A practical scheme was proposed aiming at the problem of data compression and storage under  high speed network. The scheme compressed the data with hardware and software, and without any performance affection to the network speed. HIFN data reduction (DR) series high performance lossless data compression card was used in hardware. The card had multi 9630 compression chips and used the eLZS compression algorithm to provide fast data compression and decompression with satisfactory compression ratio. An eLZS module embedded in the DR device driver  compressed and decompressed the data seamlessly when the hardware was failure in order to make the system run normally. The Windows driver of DR was developed in order to make DR compression card work on Windows effectively. The scheme was realized on both Linux and Windows platform and showed good performance under the test．
:  TP 39  
基金资助:

浙江省重大专项重点资助项目(2007C11068，2007C11088)；国家“863”高技术研究发展计划资助项目(2006AA01Z431).
通讯作者: 潘雪增，男，教授，博导.     E-mail: xzpan@cs.zju.edu.cn
作者简介: 张宝军（1977-），男，湖北京山人，博士生，从事计算机系统结构及网络安全的研究.
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章　　

引用本文:

张宝军, 平玲娣, 潘雪增, 等. 高速网络环境下数据的压缩存储[J]. J4, 2009, 43(09): 1585-1590.

ZHANG Bao-Jun, BENG Ling-Di, BO Xue-Ceng, et al. Data compression and storage under high speed network. J4, 2009, 43(09): 1585-1590.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008973X.2009.09.007    或    http://www.zjujournals.com/eng/CN/Y2009/V43/I09/1585

［1］ SHANNON C E. A mathematical theory of communication ［J］. The Bell System Technical Journal, 1948, 27(3): 379423．
［2］ HUFFMAN D A. A method for the construction of minimum redundancy codes ［J］. Proceedings of the Institute of Radio Engineers, 1952, 40(9): 10981101．
［3］ RISSANEN J. Generalized kraft inequality and arithmetic coding ［J］. IBM Journal of Research and Development, 1976, 20(3): 198203．
［4］ RISSANEN J, LANGDON G G. Arithmetic coding ［J］. IBM Journal of Research and Development, 1979, 23(2): 149162．
［5］ WITTEN I H, NEAL R M, CLEARY J G. Arithmetic coding for data compression ［J］. Communications of the ACM, 1987, 30(6): 520538．
［6］ 吴珉. 数据压缩技术简史 ［EB/OL］. ［20030926］. http:∥dev.csdn.net/article/48/48668.shtm．
［7］ ZIV J, LEMPEL A. A universal algorithm for sequential data compression ［J］. Information Theory, IEEE Transactions on, 1977, 23(3): 337343．
［8］ ZIV J, LEMPEL A. Compression of individual sequences via variablerate coding ［J］. Information Theory, IEEE Transactions on, 1978, 24(5): 530536．
［9］ WELCH T. Technique for highperformance data compression ［J］. IEEE Computer, 1984, 17(6): 819．
［10］ BHANJA S, RANGANATHAN N. Hardware implementation of data compression［M］∥Lossless compression handbook. United States: Academic, 2002: 405446．
［11］ AGUSTIN R A, RAFAEL G G, RICARDO C P. A hardware design of a massiveparallel, modular NNbased vector quantizer for realtime video coding ［J］. Microprocessors and Microsystems, 2007, 32(4): 3344．
［12］ CHAIKALIS D, SGOUROS N, MAROULIS D, et al. Hardware implementation of a disparity estimation scheme for realtime compression in 3D imaging applications ［J］. Journal of Visual Communication and Image Representation, 2008, 19(1): 111．
［13］ 戴明明. 高性能无损数据压缩卡赛驰1000 ［EB/OL］. ［20060721］.http:∥tech.sina.com.cn/roll/20060721/124245305.shtml．
［14］ BARBARA N. Enhanced LZS algorithm ［EB/OL］. ［20070407］. http:∥www.hifn.com/products.aspx？id=2456．
[1] 解聪, 雷辉, 徐星, 陈伟锋, 陈海东,杨劲松, 严丹方, 陈为, 严森祥. 基于并行欧式距离变换的三维障碍距离场计算[J]. J4, 2014, 48(2): 360-367.
[2] 全卉, 马利庄. 基于生物电阻抗的人体内脏脂肪预测方法[J]. J4, 2011, 45(2): 301-305.