Customer Relationship Management (CRM)
Data Farm
Inc.®
|
The Technical "Know-How"
|
Data Compression - Definitely Ours is KISS
What compression got to do with data and databases?
Data compression is widely used in data management to save storage space and network
bandwidth. A large number of compression schemes have been devised based on character
encoding or on detection of repetitive strings.
Compression comes with a price. For example, database performance strongly depends on
the amount of available memory due to I/O buffering, the decompression processes
as well as the work space for query processing algorithms. Not to mention security is not
addressed with such compression.
We have three approaches in our Post Office Services:
|
•
|
DAO-XML files that is stored in the database as is (CLOB) and use the XML-database vendor
query services to perform the perform the database tasks. This approach does not address security also.
|
|
•
|
DAO-XML with compression-encryption files are stored as BLOB or CLOB. It addresses security, but
there is no query on such these files and must retrieve the entire file.
|
|
•
|
DAO-XML with compression-encryption plus index tables. This use the same principle
as our second approach, but we create integer index values for frequently used data values
and store them in tables. Index also addresses security since index-integer-values are hashed
and must be un-hashed or translated back to their meaningful value - normal. This approach
addresses security in the XML files and Index tables.
|
Our approaches are very fast and without heavy memory usage.
|
|
