Future Work
The Octane Toolkit was started in July 2003, and is still in the early stages of development. Many features remain to be written, and we are hoping that other developers will help to make Octane a useful resource.
What follows is a list of some of the things we would like to see added to Octane. This list does not represent a commitment on our part to code these features, but rather a wishlist and public request for help in adding these features.
-
Add custom training class, for custom segmentation of training files
-
An Efficient Arithmetic Coder - Arithmetic coding is considered the most space-efficient coding algorithm (though see [witten] for a discussion of why they ultimately chose to use a Huffman coder for their system). See [moffat] for implementation details of a good arithmetic coder. See also: http://www.cs.mu.oz.au/~alistair/arith_coder/.
-
Class Framework for Dictionary Compressors - Much like we have provided a set of integrated classes for statistical compression, we would like to have a set of classes to support Lempel-Ziv type dictionary compressor, and make writing new dictionary compressors easy.
-
More Compressor Implementations - Octane is currently a framework lacking a real collection of implemented compressors. We would like to see it used to implement some of the more popular compression algorithms (ppm, dmc, bwt, lz*, mtf, etc.). See also: http://www.datacompression.info/.
-
Experimental Compressors - We would like to see more experimental compressors, such as neural network statistical compressors, for use with external dictionary type compression tasks.
-
Cryptographic Extensions - We would like to see cryptographic functionality added to Octane. Currently we believe the best approach to this is to in fact generalize the concept of compressor so that we work with Transformer classes, which can involve compression, encryption, etc. We would then allow for transformers to be "chained" together, so that compressors and encryptors could be chained together to produce a pipeline of transformations.
-
More Testing/Reporting Tools - The testing utility of Octane needs to be extended to provide additional ways to do batch comparison of compression algorithms on known corpora. The output of such routines should be standardized for easy comparison to published results.
-
Custom Memory Use Tracking - It is often desirable when testing compression algorithms to evaluate memory usage during compression and decompression. Octane currently implements a simple memory use reporting function which is not very accurate. A custom memory manager is needed which can track and report memory usage exactly.
-
Compresor Switching Code - Octane was built to support applications like chat programs which might switch dynamically between a set of compressors/dictionaries during operation in order to track the most efficient compressor over time. A supervisor class which manages such switching and balances efficiency vs. computational effort is needed.
-
Higher Level Package Archiver - We would like to see the Octane Classes used to build a multi-file archiver, using Compressor Switching to choose the best compressor for each file (or in fact switching within a file). See also: http://datacompression.info/ArchiveFormats/.
-
Platform (endian) compatibility tests and final fixes; almost all of the bitio code is completely platform neutral, and can be used to compress on one platform (endianness) and decompress on another. The only area that needs to be fixed is the writing of floating point values, which is not used in normal compression but could be used by compressor's saving their state.
We need help implementing these features! Join us on the efnet irc network in #octane.
Generated on 20 May 2004 by
doxygen 1.3.3