CN104348490B - A kind of data splitting compression method preferred based on effect - Google Patents
A kind of data splitting compression method preferred based on effect Download PDFInfo
- Publication number
- CN104348490B CN104348490B CN201410647747.6A CN201410647747A CN104348490B CN 104348490 B CN104348490 B CN 104348490B CN 201410647747 A CN201410647747 A CN 201410647747A CN 104348490 B CN104348490 B CN 104348490B
- Authority
- CN
- China
- Prior art keywords
- compression
- data
- variety
- algorithm
- algorithms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007906 compression Methods 0.000 title claims abstract description 180
- 230000006835 compression Effects 0.000 title claims abstract description 174
- 230000000694 effects Effects 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 title claims abstract description 22
- 230000006837 decompression Effects 0.000 claims abstract description 21
- 238000013144 data compression Methods 0.000 claims abstract description 14
- 230000035945 sensitivity Effects 0.000 claims description 10
- 230000008901 benefit Effects 0.000 abstract description 6
- 238000005457 optimization Methods 0.000 description 5
- 241001269238 Data Species 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 241000545442 Radix Species 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001595 contractor effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The present invention relates to a kind of data splitting compression method preferred based on effect, including：A variety of compression algorithms are chosen according to data type to be compressed, principal character and/or time restriction factor, compression algorithm collection is constituted, the average decompression speed D of the corresponding decompression algorithm of a variety of compression algorithms is obtainedi；Demand data to be compressed is analyzed, multiple data blocks are divided into, the common parameter of computer system is obtained；Calculate the compression effectiveness value that each data block uses a variety of compression algorithms；Compare the compression effectiveness value of a variety of compression algorithms, choose the minimum algorithm of compression effectiveness value and be used as optimal algorithm；Data compression calculating is carried out according to the corresponding optimal algorithm of each data block.The present invention preferably combines compression algorithm based on effect and has merged the advantage of a variety of compression algorithms, while consider the influence of data reading speed and disk space resource, can maximized raising data compression effects, and improve the performance of the actual reading of data.
Description
Technical field
The present invention relates to computerized algorithm technical field, more particularly to a kind of data splitting compression side preferred based on effect
Method.
Background technology
At present, having differences property of programmed algorithm principle, different compression algorithm to the compression effectiveness of same type data simultaneously
Differ, similarly, identical compression algorithm is also differed for the compression effectiveness of different types of data.Due to specific data characteristics
It is unpredictable, cause any single data compression algorithm to be all unable to reach optimal compression effect.For different types of data
How compression, can be only achieved preferable compression effectiveness, be the hot issue that current people study and inquired into always.
The as shown by data of our experiment statisticses, the combination compression algorithm of feature based value can cause 15% or so data to select
The compression algorithm selected can not be optimal compression effectiveness, its compression effectiveness about optimal compression effect 40%~80% it
Between.Therefore, for conceptual data, there is 15%* (1-60%)=6% optimization sky in the combination compression algorithm of feature based value
Between.
Under the little scene of the number of times difference compressed and decompressed, the optimization in 6% above-mentioned optimization space is worth not
Greatly, and in scene of the decompression number of times much larger than compression number of times, such as analytic type database sacrifices certain compression performance, so that
Improve above-mentioned 6% compression effectiveness, can greatly improve overall decompression reading performance in multiple decompression procedure.
The optimization space that the present invention exists for the combination compression algorithm of feature based value, it is proposed that preferred based on effect
Compression algorithm is combined, this algorithm is by the contrast to actual compression result, from alternative compression algorithms selection optimal compression algorithm, energy
The actual compression effect of data is improved to greatest extent.
The content of the invention
The technical problems to be solved by the invention are how to solve to be currently based on the compression algorithm of characteristic value to have part number
According to the key issue for being unable to reach compression optimization.
For this purpose, the present invention proposes a kind of data splitting compression method preferred based on effect, including in detail below
Step：
S1：A variety of compression algorithms are chosen according to data type to be compressed, principal character and/or time restriction factor, constituted
Compression algorithm collection, and
Obtain the average decompression speed D of the corresponding decompression algorithm of a variety of compression algorithmsi；
S2：The demand data to be compressed is analyzed, multiple data blocks are divided into, and
Obtain the common parameter of the computer system；
S3：Calculate the compression effectiveness value that each data block uses a variety of compression algorithms；
S4：Compare the compression effectiveness value of a variety of compression algorithms, choose the minimum algorithm conduct of compression effectiveness value
Optimal algorithm；
S5：Data compression calculating is carried out according to the corresponding optimal algorithm of each data block.
Further, the step S3 also includes：
S31：For the multiple data block, preset data size is C0；
S32：The compression algorithm collection a variety of compression algorithms therein are traveled through to be compressed each data block
Calculate, and count the size CR of compression resulti。
Further, the step S3 also includes：
S31’：Calculating obtains total read access time of a variety of compression algorithms for the compression result of each data block
Ti；
S32’:The data in each data block are calculated without reading total time T during compression0And compression effectiveness
Value.
Specifically, formula is passed through：
Ti=(CRi/Di)+(CRi/V)
Calculating obtains total read access time T of a variety of compression algorithms for the compression result of each data blocki, its
In, the size CR of the compression resulti, the average decompression rate of every kind of algorithm in a variety of compression algorithms is Di, average I/
O reading speed is V.
Specifically, formula is passed through：
T0=C0/V
Calculate and obtain a variety of compression algorithms for the data in each data block without reading during compression
Total time T0。
Specifically, formula is passed through：
CEV0=(T0/T0)+C0* DSR=1+C0*DSR
Calculate and obtain a variety of compression algorithms for the data in each data block without compression during compression
Effect value, wherein, the preset data size is C0, disk sensitivity coefficient is DSR.
Further, the step S3 also includes：
Pass through formula：
CEVi=(Ti/T0)+CRi*DSR
The compression effectiveness value that each data block uses a variety of compression algorithms is calculated, wherein, calculating obtains described
Total read access time T of a variety of compression algorithms for the compression result of each data blocki, calculate in each data block
Data are without reading total time T during compression0, the size CR of the compression resulti, the disk sensitivity coefficient is DSR.
Further, also include after the step S5：Each data block compression result is recorded, and it is described optimal
Algorithm information.
The invention discloses a kind of data splitting compression method preferred based on effect, the combination compression method based on effect
The advantage of a variety of compression algorithms has been merged, while the influence of data reading speed and disk space resource is considered, can be maximum
The raising data compression effects of change, and the performance of the actual reading of data can be improved；Further, combination compression method is combined
LZ4, Huffman encoding algorithm, gzip compress three kinds of compression algorithms advantage, can maximized raising data compression effects, so as to have
The reading performance of the raising data of effect.
Brief description of the drawings
The features and advantages of the present invention can be more clearly understood from by reference to accompanying drawing, accompanying drawing is schematical without that should manage
Solve to carry out any limitation to the present invention, in the accompanying drawings：
Fig. 1 shows a kind of step flow of data splitting compression method preferred based on effect in the embodiment of the present invention
Figure；
The step of Fig. 2 shows a kind of preferred based on effect data splitting compression method in another embodiment of the present invention
Flow chart.
Embodiment
Following noun is explained in detail first：1) compression algorithm collection：The set of a variety of compression algorithm compositions, according to not
Same data characteristicses, scene etc., can select different compression algorithms to constitute compression algorithm collection；2) average decompression rate：Use
When the corresponding decompression algorithm of compression algorithm performs decompression operations, the average data for the compressed data that can be handled in the unit interval
Amount；DSR (Disk Sensitivity Ratio, disk sensitivity coefficient)：A number of data are when disk is stored
Sensitivity coefficient, the value of the coefficient is TDV (Total Disk Volume：Disk total capacity) inverse with ES (Expert
Score：Expert estimation) product：DSR=ES/TDV.Wherein, TDV is the parameter of system hardware, and ES is to application scenarios by expert
And the marking provided after system progress overall merit.DSR is smaller, represents more insensitive for coefficient.Under extreme case, work as disk
When infinitely great, DSR is 0；4)CEV(Compress Effect Value:Compression effectiveness value) compressed by the evaluation calculated
The numerical value of effect, unit is 1.Below in conjunction with accompanying drawing, embodiments of the present invention is described in detail.
As shown in figure 1, the invention provides a kind of data splitting compression method preferred based on effect, including in detail below
Step：
Step S1：A variety of compression algorithms are chosen according to data type to be compressed, principal character and/or time restriction factor,
Compression algorithm collection is constituted, and obtains the average decompression speed D of the corresponding decompression algorithm of a variety of compression algorithmsi。
Step S2：Demand data to be compressed is analyzed, data to be compressed are divided into multiple data blocks, and obtain department of computer science
The common parameter of system.
Step S3：Calculate the compression effectiveness value that each data block uses a variety of compression algorithms.
Specifically, step S3 also includes：
Step S31：For multiple data blocks, preset data size is C0。
Step S32：Traversal compression algorithm collection a variety of compression algorithms therein are compressed calculating to each data block, and unite
Count the size CR of compression resulti。
Further, step S3 also includes：
Step S31 '：Calculating obtains total read access time T of a variety of compression algorithms for the compression result of multiple data blocksi。
Step S32 ':The data in multiple data blocks are calculated without reading total time T during compression0And compression effectiveness
Value.
Specifically, formula is passed through：Ti=(CRi/Di)+(CRi/V)
Calculating obtains total read access time T of a variety of compression algorithms for the compression result of each data blocki, wherein, compression
As a result size CRi, the average decompression rate of every kind of algorithm in a variety of compression algorithms is Di, average I/O reading speed is V.
Further, formula is passed through：T0=C0/V
Calculate and obtain a variety of compression algorithms for the data in each data block without reading total time T during compression0。
Further, formula is passed through：
CEV0=(T0/T0)+C0* DSR=1+C0*DSR
Calculate obtain a variety of compression algorithms for the data in each data block without compression when compression effectiveness value, its
In, preset data size is C0, disk sensitivity coefficient is DSR.
Further, because the final purpose of data compression is able to read available data faster, number is worked as
During according to without compression, although data volume is bigger than data after compression, but need not be decompressed when reading, when there is total reading
Between smaller possibility, i.e., it is necessary to which the total read access time of the data of not compressed data is added with reference to alternative when selecting compression algorithm
In algorithm.
Specifically, formula is passed through：
CEVi=(Ti/T0)+CRi*DSR
The compression effectiveness value that each data block uses a variety of compression algorithms is calculated, wherein, calculating obtains a variety of compression algorithms
For total read access time T of the compression result of each data blocki, the data in each data block are calculated without reading during compression
Take total time T0, the size CR of compression resulti, disk sensitivity coefficient is DSR.
Further, the time of digital independent includes from the time of data after disk reading compression and data is solved
The time of pressure.
Step S4：Compare the compression effectiveness value of a variety of compression algorithms, choose the minimum algorithm of compression effectiveness value as optimal
Algorithm.
Step S5：Data compression calculating is carried out according to the corresponding optimal algorithm of each data block.
Further, also include after step S5：Record each data block compression result, and optimal algorithm letter
Breath, so that when decompressing the data of this compression algorithm, decompression algorithm can use corresponding algorithm to be decompressed, effectively, accurately
Obtain decompress data.
In order to more fully understand with using a kind of data splitting compression method preferred based on effect proposed by the present invention, tying
Close Fig. 2 and the example below is carried out to the present invention, the present invention not only limits to the example below.
Specifically, in the big table data instance to be commonly used in row deposit data storehouse, the detailed single table data of a communication common carrier,
Three class data are divided into according to data characteristicses：Ordered datas are largely repeated by representative of exchange hour, using user area as generation
Table it is a large amount of repeat non-ordered datas and with tranaction costs (be accurate to point) for the less repeatedly non-ordered data of representative.
In conventional compressed encoding model, belong to the Run- Length Coding of dictionary model and the ordering relation of initial data
Closely, it is and unrelated with its frequency of occurrences；The huffman coding and the initial data frequency of occurrences for belonging to statistical model are in close relations, and
It is unrelated with its ordering.Therefore, the characteristics of using exchange hour as the ordered data of representative, LZ4 algorithms are selected；According to user area
Domain is the characteristics of radix of representative is compared with small data, selects Huffman encoding algorithm；It is representative according to tranaction costs (be accurate to point)
The characteristics of less repetition non-ordered data, the gzip algorithms of the comprehensive two kinds of algorithm characteristics of selection.Therefore, in this example, compression is calculated
Method collection is made up of LZ4 algorithms, Huffman encoding algorithm, three kinds of gzip algorithms.
Further, a large amount of to repeat the LZ4 compression algorithms that ordered data uses Run- Length Coding, this algorithm uses Run- Length Coding
To realize, the number of times of this algorithm numeral description Data duplication, when data are largely repeated in order, this algorithm has high
Compression ratio, all data compressions can be adapted to one in the primitive of storage, therefore all data to be compressed
Only need to be divided into 1 data block.Largely repeat unordered data selection and use Huffman encoding algorithm, when base value is larger,
This algorithm needs to use more space to store all radixes.Therefore, optimum organization compression algorithm only has appropriate reduction single
Data compression result, could be saved in and be adapted in the data cell of storage, so as to realize optimal pressure by the data total amount of compression
Contracting effect.Data to be compressed need to be divided into 10~20 data blocks or so in the big table of database in this example.For less repetition
Unordered data selection uses gzip algorithms, because its average compression ratio is less than the algorithm that other two types data are used.Cause
This, single table in this example needs total data being divided into 50~100 data blocks or so, just can guarantee that compression result can be deposited
Storage is in primitive.
Specifically, calculating is compressed to data block first by LZ4 algorithms, the size CR of data after being compressed1, so
Calculating is compressed to data block using Huffman encoding algorithm afterwards, the size CR of data after being compressed2, finally compressed using gzip
Algorithm is compressed calculating to data block, the size of data CR after being compressed3。
Further, LZ4 algorithms, Huffman encoding algorithm, the average decompression rate D of gzip algorithms are obtained respectively1、D2、D3And
Average I/O reading speed V, and combine the size of data CR after the compression of the three kinds of algorithms obtained in aforesaid operations1、CR2、CR3,
Calculating obtains total read access time T of three kinds of algorithms for the compression result of the data block1、T2、T3。
Further, calculate respectively and obtain LZ4 algorithms, Huffman encoding algorithm, the CEV of gzip algorithms1、CEV2、CEV3.Its
In, compression algorithm collection includes LZ4, Huffman encoding algorithm, gzip and compresses three kinds of algorithms and uncompressed CEV, i.e. compression effectiveness value
Respectively CEV1、CEV2、CEV3、CEV0.Therefrom choose minimum value, it is assumed that be CEV2, its corresponding algorithm be Huffman encoding algorithm just
It is the optimal algorithm of the data block.Huffman encoding algorithm calculates obtained compression result as the data during preserving aforesaid operations
The compression result of block.
Further, it is necessary to which recording the data block uses Huffman while record data block compression result
Compression algorithm, so could use correct decompression algorithm in decompression.
The invention discloses a kind of data splitting compression method preferred based on effect, the combination compression algorithm based on effect
The advantage of a variety of compression algorithms has been merged, while the influence of data reading speed and disk space resource is considered, can be maximum
While the raising data compression effects of change, the performance of the actual reading of data is improved；Further, combination compression algorithm is combined
LZ4, Huffman encoding algorithm, gzip compress three kinds of compression algorithms advantage, can maximized raising data compression effects, so as to have
The reading performance of the raising data of effect；Further, although during loading, the combinational algorithm uses LZ4, Hough respectively to data
Graceful algorithm, gzip compress three kinds of compression algorithms and are compressed calculating, reduce data loading performance, but greatly improve data reading
Take performance, the usage scenario repeatedly read for once loading, for example, big data query analysis etc. its data multiple reading
In, the lifting of reading performance is far longer than the reduction of loading performance, so as to provide efficient data compression algorithm for this kind of scene.
Although being described in conjunction with the accompanying embodiments of the present invention, those skilled in the art can not depart from this hair
Various modifications and variations are made in the case of bright spirit and scope, such modifications and variations are each fallen within by appended claims
Within limited range.
Claims (7)
1. a kind of data splitting compression method preferred based on effect, it is characterised in that including step in detail below：
S1：A variety of compression algorithms are chosen according to data type to be compressed, principal character and/or time restriction factor, compression is constituted
Set of algorithms, and
Obtain the average decompression speed D of the corresponding decompression algorithm of a variety of compression algorithmsi；
S2：The demand data to be compressed is analyzed, multiple data blocks are divided into, and
Obtain the common parameter of computer system；
S3：Calculate the compression effectiveness value that each data block uses a variety of compression algorithms；
S4：Compare the compression effectiveness value of a variety of compression algorithms, choose the minimum algorithm of compression effectiveness value as optimal
Algorithm；
S5：Data compression calculating is carried out according to the corresponding optimal algorithm of each data block；
Wherein, the step S3 also includes：
Pass through formula：
CEVi=(Ti/T0)+CRi*DSR
The compression effectiveness value that each data block uses a variety of compression algorithms is calculated, wherein, calculating obtains described a variety of
Total read access time T of the compression algorithm for the compression result of each data blocki, calculate the data in each data block
Without reading total time T during compression0, the size CR of the compression resulti, the DSR is disk sensitivity coefficient；
The DSR is sensitivity coefficient of a number of data when disk is stored, and is by formula：
DSR=ES/TDV
Calculate what is obtained, wherein, TDV is disk total capacity, is the parameter of system hardware；ES is by expert is to application scenarios and is
System carries out the marking provided after overall merit；DSR is smaller, represents more insensitive for coefficient.
2. a kind of data splitting compression method preferred based on effect as claimed in claim 1, it is characterised in that the step
S3 also includes：
S31：For the multiple data block, preset data size is C0；
S32：Travel through the compression algorithm collection a variety of compression algorithms therein and calculating be compressed to each data block,
And count the size CR of compression resulti。
3. a kind of data splitting compression method preferred based on effect as claimed in claim 2, it is characterised in that the step
S3 also includes：
S31’：Calculating obtains total read access time T of a variety of compression algorithms for the compression result of each data blocki；
S32’:The data in each data block are calculated without reading total time T during compression0And compression effectiveness value.
4. a kind of data splitting compression method preferred based on effect as claimed in claim 3, it is characterised in that pass through public affairs
Formula：
Ti=(CRi/Di)+(CRi/V)
Calculating obtains total read access time T of a variety of compression algorithms for the compression result of each data blocki, wherein, institute
State the size CR of compression resulti, the average decompression rate of every kind of algorithm in a variety of compression algorithms is Di, average I/O reading
It is V to take speed.
5. a kind of data splitting compression method preferred based on effect as claimed in claim 4, it is characterised in that pass through public affairs
Formula：
T0=C0/V
Calculate obtain a variety of compression algorithms for the data in each data block without compression when reading it is total when
Between T0。
6. a kind of data splitting compression method preferred based on effect as described in claim 3 or 5, it is characterised in that pass through
Formula：
CEV0=(T0/T0)+C0* DSR=1+C0*DSR
Calculate and obtain a variety of compression algorithms for the data in each data block without compression effectiveness during compression
Value, wherein, the preset data size is C0, disk sensitivity coefficient is DSR.
7. a kind of data splitting compression method preferred based on effect as claimed in claim 1, it is characterised in that the step
Also include after S5：Record each data block compression result, and the optimal algorithm information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410647747.6A CN104348490B (en) | 2014-11-14 | 2014-11-14 | A kind of data splitting compression method preferred based on effect |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410647747.6A CN104348490B (en) | 2014-11-14 | 2014-11-14 | A kind of data splitting compression method preferred based on effect |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104348490A CN104348490A (en) | 2015-02-11 |
| CN104348490B true CN104348490B (en) | 2017-09-19 |
Family
ID=52503430
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410647747.6A Active CN104348490B (en) | 2014-11-14 | 2014-11-14 | A kind of data splitting compression method preferred based on effect |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104348490B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713394A (en) | 2015-11-16 | 2017-05-24 | 华为技术有限公司 | Data transmission method and device |
| CN105630999A (en) * | 2015-12-28 | 2016-06-01 | 华为技术有限公司 | Data compressing method and device of server |
| CN105653698A (en) * | 2015-12-30 | 2016-06-08 | 北京奇艺世纪科技有限公司 | Data loading method and apparatus for database table Hive Table |
| CN107783990B (en) * | 2016-08-26 | 2021-11-19 | 华为技术有限公司 | Data compression method and terminal |
| CN106610616B (en) * | 2016-11-30 | 2019-12-03 | 交控科技股份有限公司 | A kind of performance automated tuning method and device based on big data analysis |
| CN107451266A (en) * | 2017-07-31 | 2017-12-08 | 北京京东尚科信息技术有限公司 | For processing data method and its equipment |
| CN109962710A (en) * | 2017-12-14 | 2019-07-02 | 阿里巴巴集团控股有限公司 | Data compression method, electronic device, and computer-readable storage medium |
| CN110334124B (en) * | 2018-06-15 | 2023-05-16 | 腾讯科技（深圳）有限公司 | Compression algorithm selection method, device and equipment |
| CN110163367B (en) * | 2018-09-29 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Terminal deployment method and device |
| CN109710239B (en) * | 2018-12-29 | 2022-08-16 | 北京航天数据股份有限公司 | Industrial model generation method and device, digital asset processing method and electronic equipment |
| CN110035299B (en) * | 2019-04-18 | 2021-02-05 | 雷欧尼斯（北京）信息技术有限公司 | Compression transmission method and system for immersive object audio |
| CN110311687B (en) * | 2019-07-09 | 2022-10-04 | 上海天数智芯半导体有限公司 | Time sequence data lossless compression method based on integration algorithm |
| CN110535869B (en) * | 2019-09-05 | 2021-11-02 | 厦门市美亚柏科信息股份有限公司 | A data transmission method, terminal device and storage medium based on compression algorithm |
| CN110704408A (en) * | 2019-09-10 | 2020-01-17 | 南京天数智芯科技有限公司 | Clustering-based time sequence data compression method and system |
| WO2022155921A1 (en) * | 2021-01-22 | 2022-07-28 | 京东方科技集团股份有限公司 | Display panel and driving method therefor, compensation data compression method, and compensation data decompression method |
| CN114265821A (en) * | 2021-12-28 | 2022-04-01 | 深圳市云语科技有限公司 | File compression algorithm selection method based on capability negotiation |
| CN115062009A (en) * | 2022-03-09 | 2022-09-16 | 国电内蒙古东胜热电有限公司 | Flexible data compression method based on K-means clustering |
| CN118838879A (en) * | 2024-09-20 | 2024-10-25 | 江苏华库数据技术有限公司 | Self-adaptive data compression method, system, equipment and product for database |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1251449A (en) * | 1998-10-18 | 2000-04-26 | 华强 | Combined use with reference of two category dictionary compress algorithm in data compaction |
| CN101459616A (en) * | 2009-01-04 | 2009-06-17 | 成都市华为赛门铁克科技有限公司 | Method, customer terminal and access system for network access |
| US7765346B2 (en) * | 2007-12-14 | 2010-07-27 | Bmc Software, Inc. | Dynamic compression of systems management data |
| WO2011129818A1 (en) * | 2010-04-13 | 2011-10-20 | Empire Technology Development Llc | Adaptive compression |
| CN102594360A (en) * | 2012-02-01 | 2012-07-18 | 浪潮(北京)电子信息产业有限公司 | Method and device for computer data compression |
| CN103166983A (en) * | 2011-12-09 | 2013-06-19 | 中国移动通信集团广东有限公司 | Network data optimization processing method and device, network server |
-
2014
- 2014-11-14 CN CN201410647747.6A patent/CN104348490B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1251449A (en) * | 1998-10-18 | 2000-04-26 | 华强 | Combined use with reference of two category dictionary compress algorithm in data compaction |
| US7765346B2 (en) * | 2007-12-14 | 2010-07-27 | Bmc Software, Inc. | Dynamic compression of systems management data |
| CN101459616A (en) * | 2009-01-04 | 2009-06-17 | 成都市华为赛门铁克科技有限公司 | Method, customer terminal and access system for network access |
| WO2011129818A1 (en) * | 2010-04-13 | 2011-10-20 | Empire Technology Development Llc | Adaptive compression |
| CN103166983A (en) * | 2011-12-09 | 2013-06-19 | 中国移动通信集团广东有限公司 | Network data optimization processing method and device, network server |
| CN102594360A (en) * | 2012-02-01 | 2012-07-18 | 浪潮(北京)电子信息产业有限公司 | Method and device for computer data compression |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104348490A (en) | 2015-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104348490B (en) | A kind of data splitting compression method preferred based on effect | |
| CN101241508B (en) | Structured data sequence compression method | |
| JP5466232B2 (en) | Efficient column-based data encoding for large-scale data storage | |
| CN101968806A (en) | Data storage method, querying method and device | |
| CN102609491A (en) | Column-storage oriented area-level data compression method | |
| CN104298736B (en) | Data acquisition system connection method, device and Database Systems | |
| CN105144157B (en) | System and method for compressing data in a database | |
| CN103995887A (en) | Bitmap index compressing method and bitmap index decompressing method | |
| CN102708183A (en) | Method and device for data compression | |
| US20170221153A1 (en) | Systems and Methods for Use in Compressing Data Structures | |
| CN105824574A (en) | Memory data storage method | |
| CN116170027B (en) | Data management system and processing method for poison detection equipment | |
| CN103500224B (en) | A kind of method for writing data and device, method for reading data and device | |
| CN107273471A (en) | A kind of binary electric power time series data index structuring method based on Geohash | |
| CN110442557A (en) | Data compression and decompression method, electronic equipment and computer readable storage medium | |
| CN105099460B (en) | Dictionary compression method, dictionary decompression method and dictionary constructing method | |
| CN114064753B (en) | Time series data processing method, device and electronic device | |
| TW202044129A (en) | Computing system and compressing method thereof for neural network parameters | |
| CN112101548A (en) | Data compression method and device, data decompression method and device, and electronic device | |
| AU2017248412A1 (en) | Information processing apparatus, and data management method | |
| CN107433032A (en) | Chess game data processing method and device | |
| CN109698703B (en) | Gene sequencing data decompression method, system and computer readable medium | |
| US20200099391A1 (en) | Method for compressing a quantum state vector and process for storing a quantum state vector | |
| JP2022545644A5 (en) | ||
| CN108182283A (en) | A kind of many body simulation lossy compression based on single snapshot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |