Customer Relationship Management (CRM)
Data Farm
Inc.®
|
The Technical "Know-How"
|
Compression - Reducing but not Mapping
|
Compression is a reduction in the number of bits needed to represent data. Compressing
data can save storage capacity, speed file transfer, and decrease costs for storage
hardware and network bandwidth. Data compression is also widely used in backup
utilities, spreadsheet applications, and database management systems.
|
Our Post office Project is using compression in the following:
• Data Stream
• Reducing Data Size in Big Data
• Compression-Encryption Security Approach
Our Post Office (PO) Compression-Encryption is not competing with Hadoop MapReduce
framework, but PO can be used by Hadoop MapReduce as complementing tools in both data reduction and security.
Compression Methods:
|
•
|
Lossy Compression: provides high degrees of compression and result in smaller compressed
files, but there is a certain amount of visual loss when restored.
|
|
•
|
Lossless Compression: is where no original data is lost when a file is zipped. Lossless
compression is preferred for archival purposes and often for medical imaging, technical
drawings. Usage of Image, image transmission and Data Streaming are on the raise and
the most that can be reduced in term of image size in lossless compression is 20% at
best. Not to mention that the best known algorithms are very slow and sometimes is not
practical for real-time applications.
|
Streaming Data:
Streaming or media streaming is a technique for transferring data so that it can be
processed as a steady and continuous stream. Streaming technologies are becoming
increasingly important with the growth of the Internet because most users do not have
fast enough access to download large multimedia files quickly. To illustrate the
importance of Data Streaming, let us look at a fighter jet or surveillance airplane
as an example. In real combat scenario, the fighter jet or surveillance airplane cameras
would capture ground, enemy positions, enemy personal and enemy fighters images and
transmits these images to the air force base computer(s) and satellites to analyze
and reply back with recommended set of actions or maneuvers in real time. The images
streaming time or the images transmission is time-critical to for both fighter jet
computer and air force base computer(s). The size of these images depend on image
resolutions. The higher the resolution the more accurate the data, but the larger
the image size and the longer time of transmission. Image Compression can help if
the following are true:
|
•
|
Compression-Decompression is lossless with no loss of data
|
|
•
|
Compressed image is reduced significantly to reduce images transmission-time
|
|
•
|
Compression and decompression algorithms speed is as fast as the CPU speed of processing, so there is almost no delay in compression and decompression.
|
We believe we have an answer to Data Streaming including the fighter jet
example. We literally stumbled on three algorithms which would save in
compression size as follows:
|
•
|
First algorithm would save 30%
|
|
•
|
Second algorithm would save 40% to 50%
|
|
•
|
Third algorithm would save over 60%
|
The speed of compressing and decompressing is as fast as CPU and compressing and decompressing
would be a fraction of what we have today. These algorithms can be a part of the CPU architect, or
put into its own chip and be installed in cameras, computers, phones, cars, etc. The compressed
format is also automatic encryption and the compressed-encrypted data is impossible to break and
the variation of encryption implementation is endless.
Compression-Encryption Security Approach:
Our Compression-Encryption Security Approach is a total package to secure data, reduce the
data size and implement a number of processes to automate, manage and train staff on these
security processes. There a number pages on this site which address these processes and the technologies.
|
|
