|
|
|
| Data compression and storage under high speed network |
| ZHANG Bao-jun1, PING Ling-di1, PAN Xue-zeng1, WANG Jie-bing2 |
(1.College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China;
2.Hifn (Hangzhou) Information Technology Limited Company, Hangzhou 310012, China) |
|
|
|
Abstract A practical scheme was proposed aiming at the problem of data compression and storage under high speed network. The scheme compressed the data with hardware and software, and without any performance affection to the network speed. HIFN data reduction (DR) series high performance lossless data compression card was used in hardware. The card had multi 9630 compression chips and used the eLZS compression algorithm to provide fast data compression and decompression with satisfactory compression ratio. An eLZS module embedded in the DR device driver compressed and decompressed the data seamlessly when the hardware was failure in order to make the system run normally. The Windows driver of DR was developed in order to make DR compression card work on Windows effectively. The scheme was realized on both Linux and Windows platform and showed good performance under the test．
|
|
|
|
|
高速网络环境下数据的压缩存储
针对高速网络环境下数据的压缩存储问题,提出一种实用的解决方案,该方案采用软硬件相结合的压缩存储方法,在不影响网络速度的前提下,可以对数据进行实时的压缩存储.方案在硬件上使用汉帆公司（HIFN）的DR系列高性能无损数据压缩卡,该卡由多个9 630压缩芯片组成,采用拥有汉帆自主专利的增强型LZS技术,能够在满足压缩比的前提下提供快速的数据加解压功能.当硬件出现故障时,该方案将采用设备驱动程序中内嵌的eLZS软件模块对数据进行无缝的加解压处理,以保障系统的正常运行.为了使DR压缩卡在Windows环境下有效工作,开发DR压缩卡的Windows驱动程序.该方案在Linux及Windows环境下得到实现,在测试过程中表现出较好的性能.
|
|
| ［1］ SHANNON C E. A mathematical theory of communication ［J］. The Bell System Technical Journal, 1948, 27(3): 379423．
［2］ HUFFMAN D A. A method for the construction of minimum redundancy codes ［J］. Proceedings of the Institute of Radio Engineers, 1952, 40(9): 10981101．
［3］ RISSANEN J. Generalized kraft inequality and arithmetic coding ［J］. IBM Journal of Research and Development, 1976, 20(3): 198203．
［4］ RISSANEN J, LANGDON G G. Arithmetic coding ［J］. IBM Journal of Research and Development, 1979, 23(2): 149162．
［5］ WITTEN I H, NEAL R M, CLEARY J G. Arithmetic coding for data compression ［J］. Communications of the ACM, 1987, 30(6): 520538．
［6］ 吴珉. 数据压缩技术简史 ［EB/OL］. ［20030926］. http:∥dev.csdn.net/article/48/48668.shtm．
［7］ ZIV J, LEMPEL A. A universal algorithm for sequential data compression ［J］. Information Theory, IEEE Transactions on, 1977, 23(3): 337343．
［8］ ZIV J, LEMPEL A. Compression of individual sequences via variablerate coding ［J］. Information Theory, IEEE Transactions on, 1978, 24(5): 530536．
［9］ WELCH T. Technique for highperformance data compression ［J］. IEEE Computer, 1984, 17(6): 819．
［10］ BHANJA S, RANGANATHAN N. Hardware implementation of data compression［M］∥Lossless compression handbook. United States: Academic, 2002: 405446．
［11］ AGUSTIN R A, RAFAEL G G, RICARDO C P. A hardware design of a massiveparallel, modular NNbased vector quantizer for realtime video coding ［J］. Microprocessors and Microsystems, 2007, 32(4): 3344．
［12］ CHAIKALIS D, SGOUROS N, MAROULIS D, et al. Hardware implementation of a disparity estimation scheme for realtime compression in 3D imaging applications ［J］. Journal of Visual Communication and Image Representation, 2008, 19(1): 111．
［13］ 戴明明. 高性能无损数据压缩卡赛驰1000 ［EB/OL］. ［20060721］.http:∥tech.sina.com.cn/roll/20060721/124245305.shtml．
［14］ BARBARA N. Enhanced LZS algorithm ［EB/OL］. ［20070407］. http:∥www.hifn.com/products.aspx？id=2456． |
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
