Octane v1.01.20 - The Open Compression Toolkit for C++

http://octane.sourceforge.net/

OctaneCompressor_Statistical Class Reference
[Compressors]

#include <compressor_statistical.hpp>

Inheritance diagram for OctaneCompressor_Statistical:

List of all members.

---

Detailed Description

The Statistical compressor class is a base class from which to derive specific statistical compressors that make use of the parser/modeler/coder framework.

Definition at line 39 of file compressor_statistical.hpp.

Public Member Functions
	OctaneCompressor_Statistical (bool registerme=false)
	constructor
virtual	~OctaneCompressor_Statistical ()
	destructor
virtual std::string	GetClassName ()
	Get the assigned name of an instantiated compressor.
virtual std::string	GetDescription ()
	optionally provide a longer (maybe 20-60 characters) description
virtual std::string	GetHelpInformation ()
	optionally provide more information about the object on request for help
virtual void	ShowDebuggingInfo ()
	show any debugging info on request (used by various derived classes) [optional]
virtual unsigned int	GetMemoryUsed ()
	Report actual current memory usage (in bytes).
virtual unsigned int	GetDiskspaceUsed (bool fortempdecompressiononly)
	Report disk space (in bytes) that would be used when saving state to file (in bytes).
virtual std::string	GetParametersInformation ()
	Reports information about any parameters available.
virtual void	SetDefaultParameters ()
	Reset any parameters to their default values.
virtual bool	SetParameter (const std::string &parametername, const std::string &parametervalue)
virtual void	ResetState ()
	Reset the state of the compressor to its initial state.
virtual void	SynchronizeStateForNewStream ()
	Synchronize state for a new stream - this will be called each time before beginning a new parsing stream.
virtual bool	PrepareForCompression ()
	Prepare for parsing mode, this will be called parsing begins.
virtual void	SetupAnyDefaultParser ()
virtual bool	IsReadyToCompress ()
	is the compressor ready to compress? Normally this is false until a dictionary is built or loaded.
Protected Member Functions
virtual bool	DoProtectedCompress (bitreader &from, bitwriter &to)
	This is the member function which does the actual compression, and this is the function that should be subclasses by derived classes.
virtual bool	DoProtectedDecompress (bitreader &from, bitwriter &to)
	This is the member function which does the actual decompression, and this is the function that should be subclasses by derived classes.
virtual bool	DoProtectedCreateSymbolsAndModelsUsingStream (bitreader &from)
virtual bool	DoProtectedSaveState (bitwriter &to, bool fortempdecompressiononly)
	This is the member function which does the actual saving of state, and this is the function that should be subclasses by derived classes.
virtual bool	DoProtectedLoadState (bitreader &from)
Protected Attributes
OctaneParser *	parserp
	Pointer to the derived (and dynamically allocated) statistical parser.
OctaneModeler *	modelerp
	Pointer to the derived (and dynamically allocated) statistical modeler.
OctaneCoder *	coderp
	Pointer to the derived (and dynamically allocated) statistical coder.

---

Member Function Documentation

virtual std::string OctaneCompressor_Statistical::GetDescription (   )  [inline, virtual]

optionally provide a longer (maybe 20-60 characters) description Returns: a one line description Reimplemented from OctaneCompressor. Reimplemented in SampleStatCompressor, and SubStrHuffCompressor. Definition at line 58 of file compressor_statistical.hpp. {return "Statistical Compressor Base Class";}

string OctaneCompressor_Statistical::GetHelpInformation (   )  [virtual]

optionally provide more information about the object on request for help Returns: a long string (can be multiple newlines) Reimplemented from OctaneCompressor. Definition at line 69 of file compressor_statistical.cpp. References coderp, OctaneCoder::GetHelpInformation(), OctaneModeler::GetHelpInformation(), OctaneParser::GetHelpInformation(), modelerp, parserp, and OctaneClass::PrefixNonEmptyString(). { // show help info from our parser,model,coder string helpinfo; // generic help information for children helpinfo += PrefixNonEmptyString("Parser Help:\n-------------------------\n",parserp->GetHelpInformation()); helpinfo += PrefixNonEmptyString("Modeler Help:\n-------------------------\n",modelerp->GetHelpInformation()); helpinfo += PrefixNonEmptyString("Coder Help:\n-------------------------\n",coderp->GetHelpInformation()); // return amalgamated string return helpinfo; }

bool OctaneCompressor_Statistical::SetParameter (  const std::string &  parametername, const std::string &  parametervalue )  [virtual]

Note: this is a little weird; there are two ways we could do this, we could use a prefix to indicate whether they want to set a parameter in parser/modeler/coder, or we could simply ask each in turn if they own the variable Reimplemented from OctaneClass. Definition at line 131 of file compressor_statistical.cpp. References coderp, modelerp, parserp, and OctaneClass::SetParameter(). { bool bretv; bretv=parserp->SetParameter(parametername,parametervalue); if (!bretv) bretv=modelerp->SetParameter(parametername,parametervalue); if (!bretv) bretv=coderp->SetParameter(parametername,parametervalue); return bretv; }

bool OctaneCompressor_Statistical::PrepareForCompression (   )  [virtual]

Prepare for parsing mode, this will be called parsing begins. Returns: true on success Todo: is this a duplication of functionality in ResetState() and SynchronizeStateForNewStream()? Reimplemented from OctaneCompressor. Definition at line 233 of file compressor_statistical.cpp. References coderp, modelerp, parserp, OctaneCoder::PrepareForCoding(), OctaneModeler::PrepareForModeling(), and OctaneParser::PrepareForParsing(). { // prepare for parsing mode, must be called after zero or more Updating calls, and before parsing begins; returns true on success bool bretv; // first prepare the parser (this should ensure we have a valid symbol set) bretv=parserp->PrepareForParsing(); // now prepare the model (give it parser info in case it wants it) if (bretv) modelerp->PrepareForModeling(parserp); // now build coder based on modeler if (bretv) coderp->PrepareForCoding(modelerp); // return success return bretv; }

void OctaneCompressor_Statistical::SetupAnyDefaultParser (   )  [virtual]

now tell the coder that all the weights on symbols have changed Reimplemented from OctaneCompressor. Definition at line 187 of file compressor_statistical.cpp. References coderp, OctaneModeler::CreateModelUsingParser(), OctaneModeler::IsReadyToModel(), OctaneParser::IsReadyToParse(), modelerp, parserp, and OctaneCoder::ReceiveNotification_ModelChange_AllSymbolWeights(). Referenced by SampleStatCompressor::SampleStatCompressor(), and SubStrHuffCompressor::SubStrHuffCompressor(). { // if a parser starts out ready to parse, we might need to build a model/coder from it if (parserp->IsReadyToParse()) { modelerp->CreateModelUsingParser(parserp); if (modelerp->IsReadyToModel()) { coderp->ReceiveNotification_ModelChange_AllSymbolWeights(modelerp); } } }

bool OctaneCompressor_Statistical::IsReadyToCompress (   )  [virtual]

is the compressor ready to compress? Normally this is false until a dictionary is built or loaded. Returns: true on success Reimplemented from OctaneCompressor. Definition at line 251 of file compressor_statistical.cpp. References coderp, OctaneCoder::IsReadyToCode(), OctaneModeler::IsReadyToModel(), OctaneParser::IsReadyToParse(), modelerp, and parserp. { // is the compressor ready to compress? Normally this is false until a dictionary is built or loaded. bool bretv1,bretv2,bretv3; if (!(bretv1=parserp->IsReadyToParse())) cout << "the parser is not ready."<<endl; if (!(bretv2=modelerp->IsReadyToModel())) cout << "the modeler is not ready."<<endl; if (!(bretv3=coderp->IsReadyToCode())) cout << "the coder is not ready."<<endl; return (bretv1 && bretv2 && bretv3); }

bool OctaneCompressor_Statistical::DoProtectedCompress (  bitreader &  from, bitwriter &  to )  [protected, virtual]

This is the member function which does the actual compression, and this is the function that should be subclasses by derived classes. It is wrapped by the public API call Compress() which performs some measurements on compression speed and streamsize. Note: this function should never be called by an outside class - the API interface should be called instead. Returns: true on success Implements OctaneCompressor. Definition at line 280 of file compressor_statistical.cpp. References OctaneCoder::AlignOutputStreamToByteBoundry(), coderp, modelerp, OctaneParser::ParseNextSymbolFromInput(), parserp, SynchronizeStateForNewStream(), OctaneModeler::UpdateModelAfterReceivingSymbol(), and OctaneCoder::WriteSymbolBits(). { // compress all of from and send to to // ATTN: the only thing slightly troubling about this otherwise elegant routine, is the // fact that there is no way for the components to change the symbol set during compression bool bretv=true; int symbolnumber; // we *must* inform our components that we are about to compress or decompress a new stream SynchronizeStateForNewStream(); // loop parsing a symbol from parser until we hit end (note the last symbol will be an End-Of-Stream) while (parserp->ParseNextSymbolFromInput(from,symbolnumber)) { // debug show some info? // cout << "$READ SYMBOL "<<symbolnumber<<": '"<<parserp->LookupSymbolText(symbolnumber)<<"'."<<endl; // code the symbol given current model and codes if (!(bretv=coderp->WriteSymbolBits(symbolnumber,to))) break; // now update the model based on this symbol (pass coderp so it can inform it if probabilities change). // note that for four zero-order model this doesn't do anything, but we call it for completeness. if (!(bretv=modelerp->UpdateModelAfterReceivingSymbol(symbolnumber,coderp))) break; } // when we finish coding a stream we want to align to byte boundary in the output // this will happen automatically if we close the output stream now, but we do it manually in case we want to pack multiple // input streams into the same output. coderp->AlignOutputStreamToByteBoundry(to); // return success return bretv; }

bool OctaneCompressor_Statistical::DoProtectedDecompress (  bitreader &  from, bitwriter &  to )  [protected, virtual]

This is the member function which does the actual decompression, and this is the function that should be subclasses by derived classes. It is wrapped by the public API call Decompress() which performs some measurements on decompression speed and streamsize. Note: this function should never be called by an outside class - the API interface should be called instead. Returns: true on success Implements OctaneCompressor. Definition at line 314 of file compressor_statistical.cpp. References OctaneCoder::AlignInputStreamToByteBoundry(), coderp, OctaneCoder::DecodeSymbolFromInput(), modelerp, parserp, SynchronizeStateForNewStream(), OctaneModeler::UpdateModelAfterReceivingSymbol(), and OctaneParser::WriteSymbolText(). { // decompress all of from and send to to // ATTN: the only thing slightly troubling about this otherwise elegant routine, is the // fact that there is no way for the components to change the symbol set during compression bool bretv=true; int symbolnumber; bool isendofstreamsymbol; // we *must* inform our components that we are about to compress or decompress a new stream SynchronizeStateForNewStream(); // loop decoding a symbol from coder until we hit end while (coderp->DecodeSymbolFromInput(symbolnumber,from)) { // debug show some info? // cout << "$DECODED SYMBOL "<<symbolnumber<<": '"<<parserp->LookupSymbolText(symbolnumber)<<"'."<<endl; // deParse the symbol given current model and symbol set if (!(bretv=parserp->WriteSymbolText(to,symbolnumber,isendofstreamsymbol))) break; // did we just parse an EOS(end-of-stream) symbol? if so we need to align input to byte boundary (normally its end of file) if (isendofstreamsymbol) { // when we finish coding a stream we want to align to byte boundary in the output // this will happen automatically if we close the output stream now, but we do it manually in case we want to pack multiple // input streams into the same output. note that this we will almost always be leaving this loop after this symbol. coderp->AlignInputStreamToByteBoundry(from); // cout << "$DEBUG aligning input."<<endl; } // now update the model based on this symbol (pass coderp so it can inform it if probabilities change). // note that for four zero-order model this doesn't do anything, but we call it for completeness. if (!(bretv=modelerp->UpdateModelAfterReceivingSymbol(symbolnumber,coderp))) break; } // return success return bretv; }

bool OctaneCompressor_Statistical::DoProtectedCreateSymbolsAndModelsUsingStream (  bitreader &  from  )  [protected, virtual]

now tell the coder that all the weights on symbols have changed Reimplemented from OctaneCompressor. Definition at line 202 of file compressor_statistical.cpp. References coderp, OctaneModeler::CreateModelUsingStream(), OctaneParser::CreateSymbolSetUsingStream(), modelerp, parserp, OctaneCoder::ReceiveNotification_ModelChange_AllSymbolWeights(), bitreader::seek_bit(), and bitreader::tell_bit(). { // optionally (pre)process an input stream to update/create a symbol set from it; returns true on success bool bretv; // build parser symbol set using stream // HOWEVER this can be a bit tricky, because we have 2 structures (parser and modeler) which may want to process from stream // so we need to rewind it after the parser gets done with it. // record starting position of from stream (probably 0) size_t fromstartpos=from.tell_bit(); // give it to parser bretv = parserp->CreateSymbolSetUsingStream(from); // now build the model using the symbols created by the parser, and again on the stream from if (bretv) { // reset from stream to start from.seek_bit(fromstartpos); // now have model process it bretv = modelerp->CreateModelUsingStream(parserp,from); if (bretv) { coderp->ReceiveNotification_ModelChange_AllSymbolWeights(modelerp); } } // return success return bretv; }

bool OctaneCompressor_Statistical::DoProtectedSaveState (  bitwriter &  to, bool  fortempdecompressiononly )  [protected, virtual]

This is the member function which does the actual saving of state, and this is the function that should be subclasses by derived classes. It is wrapped by the public API call SaveState() which performs some measurements on speed and streamsize. Note: this function should never be called by an outside class - the API interface should be called instead. Returns: true on success Reimplemented from OctaneCompressor. Definition at line 145 of file compressor_statistical.cpp. References coderp, modelerp, parserp, and OctaneClass::SaveState(). { // save state; returns true on success bool bretv; bretv=parserp->SaveState(to,fortempdecompressiononly); if (bretv) bretv=modelerp->SaveState(to,fortempdecompressiononly); if (bretv) bretv=coderp->SaveState(to,fortempdecompressiononly); return bretv; }

bool OctaneCompressor_Statistical::DoProtectedLoadState (  bitreader &  from  )  [protected, virtual]

now tell the coder that the modeler (the weights on symbols) have changed Reimplemented from OctaneCompressor. Definition at line 157 of file compressor_statistical.cpp. References coderp, OctaneClass::LoadState(), modelerp, parserp, and OctaneCoder::ReceiveNotification_ModelChange_AllSymbolWeights(). { // save state; returns true on success bool bretv; bretv=parserp->LoadState(from); if (bretv) bretv=modelerp->LoadState(from); if (bretv) { bretv=coderp->LoadState(from); coderp->ReceiveNotification_ModelChange_AllSymbolWeights(modelerp); } return bretv; }

---

The documentation for this class was generated from the following files:

• compressors/compressor_statistical.hpp
• compressors/compressor_statistical.cpp

 

Generated on 20 May 2004 by doxygen 1.3.3