Accelerating the pace of engineering and science

# Vector Quantizer Design

Design vector quantizer using Vector Quantizer Design Tool (VQDTool)

Quantizers

dspquant2

## Description

Double-click on the Vector Quantizer Design block to start VQDTool, a GUI that allows you to design and implement a vector quantizer. You can also start VQDTool by typing vqdtool at the MATLAB® command prompt. Based on your specifications, VQDTool iteratively calculates the codebook values that minimize the mean squared error between the training set and the codebook until the stopping criteria for the design process is satisfied. The block uses the resulting codebook values to implement your vector quantizer.

For the Training Set parameter, enter a k-by-M matrix of values you want to use to train the quantizer codebook. The variable k, where $k\ge 1$, is the length of each training vector. It also represents the dimension of your quantizer. The variable M, where $M\ge 2$, is the number of training vectors. This data can be created using a MATLAB function, such as the default value randn(10,1000), or it can be any variable defined in the MATLAB workspace.

You have two choices for the Source of initial codebook parameter. Select Auto-generate to have the block choose the values of the initial codebook. In this case, the block picks N random training vectors as the initial codebook, where N is the Number of levels parameter and $N\ge 2$. When you select User defined, enter the initial codebook values in the Initial codebook field. The initial codebook matrix must have the same number of rows as the training set. Each column of the codebook is a codeword, and your codebook must have at least two codewords.

For the given training set and initial codebook, the block performs an iterative process, using the Generalized Lloyd Algorithm (GLA), to design a final codebook. For each iteration of the GLA, the block first associates each training vector with its nearest codeword by calculating the distortion. You can specify one of the two possible methods for calculating distortion using the Distortion measure parameter.

When you select Squared error for the Distortion measure parameter, the block finds the nearest codeword by calculating the squared error (unweighted). Consider the codebook $CB=\left[\begin{array}{cccc}C{W}_{1}& C{W}_{2}& ...& C{W}_{N}\end{array}\right]$. This codebook has N codewords; each codeword has k elements. The i-th codeword is defined as $C{W}_{i}=\left[\begin{array}{cccc}{a}_{1i}& {a}_{2i}& ...& {a}_{ki}\end{array}\right]$. The training set has M columns and is defined as $U=\left[\begin{array}{cccc}{U}_{1}& {U}_{2}& ...& {U}_{M}\end{array}\right]$, where the p-th training vector is ${U}_{p}={\left[\begin{array}{cccc}{u}_{1p}& {u}_{2p}& ...& {u}_{kp}\end{array}\right]}^{\prime }$. The squared error (unweighted) is calculated using the equation

$D=\sum _{j=1}^{k}{\left({a}_{ji}-{u}_{jp}\right)}^{2}$

When you select Weighted squared error for the Distortion measure parameter, enter a vector or matrix for the Weighting factor parameter. When the weighting factor is a vector, its length must be equal to the number of rows in the training set. This weighting factor is used for each training vector. When the weighting factor is a matrix, it must be the same size as the training set matrix. The block finds the nearest codeword by calculating the weighted squared error. If the weighting factor for the p-th column of the training vector, ${U}_{p}$, is defined as $Wp={\left[\begin{array}{cccc}{w}_{1p}& {w}_{2p}& ...& {w}_{kp}\end{array}\right]}^{\prime }$, then the weighted squared error is defined by the equation

$D=\sum _{j=1}^{k}{w}_{jp}{\left({a}_{ji}-{u}_{jp}\right)}^{2}$

Once the block has associated all the training vectors with their nearest codeword vectors, the block calculates the mean squared error for the codebook and checks to see if the stopping criteria for the process has been satisfied.

The two possible options for the Stopping criteria parameter are Relative threshold and Maximum iteration. When you want the design process to stop when the fractional drop in the squared error is below a certain value, select Relative threshold. Then, type the maximum acceptable fractional drop in the Relative threshold field. The fraction drop in the squared error is defined as

When you want the design process to stop after a certain number of iterations, choose Maximum iteration. Then, enter the maximum number of iterations you want the block to perform in the Maximum iteration field. For Stopping criteria, you can also choose Whichever comes first and enter Relative threshold and Maximum iteration values. The block stops iterating as soon as one of these conditions is satisfied.

When a training vector has the same distortion for two different codeword vectors, the algorithm uses the Tie-breaking rule parameter to determine which codeword vector the training vector is associated with. When you want the training vector to be associated with the lower indexed codeword, select Lower indexed codeword. To associate the training vector with the higher indexed codeword, select Higher indexed codeword.

With each iteration, the block updates the codeword values in order to minimize the distortion. The Codebook update method parameter defines the way the block calculates these new codebook values.

 Note   If, for the Distortion measure parameter, you choose Squared error, the Codebook update method parameter is set to Mean.

If, for the Distortion measure parameter, you choose Weighted squared error and you choose Mean for the Codebook update method parameter, the new codeword vector is found as follows. Suppose there are three training vectors associated with one codeword vector. The training vectors are

The new codeword vector is calculated as $C{W}_{new}=\left[\begin{array}{c}\frac{1+10+11}{3}\\ \frac{2+12+12}{3}\end{array}\right]$

where the denominator is the number of training vectors associated with this codeword. If, for the Codebook update method parameter, you choose Centroid and you specify the weighting factors ${W}_{1}=\left[\begin{array}{c}0.1\\ 0.2\end{array}\right]$, ${W}_{3}=\left[\begin{array}{c}1\\ 0.6\end{array}\right]$, and ${W}_{7}=\left[\begin{array}{c}0.3\\ 0.4\end{array}\right]$, the new codeword vector is calculated as

$C{W}_{new}=\left[\begin{array}{c}\frac{\left(0.1\right)\left(1\right)+\left(1\right)\left(10\right)+\left(0.3\right)\left(11\right)}{0.1+1+0.3}\\ \frac{\left(0.2\right)\left(2\right)+\left(0.6\right)\left(12\right)+\left(0.4\right)\left(12\right)}{0.2+0.6+0.4}\end{array}\right]$

Click Design and Plot to design the quantizer with the parameter values specified on the left side of the GUI. The performance curve and the entropy of the quantizer are updated and displayed in the figures on the right side of the GUI.

 Note   You must click Design and Plot to apply any changes you make to the parameter values in the VQDTool dialog box.

The following is an example of how the block calculates the entropy of the quantizer at each iteration. Suppose you have a codebook with four codewords and a training set with 200 training vectors. Also suppose that, at the i-th iteration, 40 training vectors are associated with the first codeword, 60 training vectors are associated with the second codeword, 20 training vectors are associated with the third codeword, and 80 training vectors are associated with the fourth codeword. The probability that a training vector is associated with the first codeword is $\frac{40}{200}$. The probabilities that training vectors are associated with the second, third, and fourth codewords are $\frac{60}{200}$, $\frac{20}{200}$, and $\frac{80}{200}$, respectively. The GUI uses these probabilities to calculate the entropy according to the equation

$H=\sum _{i=1}^{N}-{p}_{i}{\mathrm{log}}_{2}{p}_{i}$

where N is the number of codewords. Based on these probabilities, the GUI calculates the entropy of the quantizer at the i-th iteration as

$\begin{array}{l}H=-\left(\frac{40}{200}{\mathrm{log}}_{2}\frac{40}{200}+\frac{60}{200}{\mathrm{log}}_{2}\frac{60}{200}+\frac{20}{200}{\mathrm{log}}_{2}\frac{20}{200}+\frac{80}{200}{\mathrm{log}}_{2}\frac{80}{200}\right)\\ H=1.8464\end{array}$

VQDTool can export parameter values that correspond to the figures displayed in the GUI. Click the Export Outputs button, or press Ctrl+E, to export the Final Codebook, Mean Square Error, and Entropy values to the workspace, a text file, or a MAT-file.

In the Model section of the GUI, specify the destination of the block that will contain the parameters of your quantizer. For Destination, select Current model to create a block with your parameters in the model you most recently selected. Type gcs in the MATLAB Command Window to display the name of your current model. Select New model to create a block in a new model file.

From the Block type list, select Encoder to design a Vector Quantizer Encoder block. Select Decoder to design a Vector Quantizer Decoder block. Select Both to design a Vector Quantizer Encoder block and a Vector Quantizer Decoder block.

In the Encoder block name field, enter a name for the Vector Quantizer Encoder block. In the Decoder block name field, enter a name for the Vector Quantizer Decoder block. When you have a Vector Quantizer Encoder and/or Decoder block in your destination model with the same name, select the Overwrite target block check box to replace the block's parameters with the current parameters. When you do not select this check box, a new Vector Quantizer Encoder and/or Decoder block is created in your destination model.

Click Generate Model. VQDTool uses the parameters that correspond to the current plots to set the parameters of the Vector Quantizer Encoder and/or Decoder blocks.

## Dialog Box

Training Set

Enter the samples of the signal you would like to quantize. This data set can be a MATLAB function or a variable defined in the MATLAB workspace. The typical length of this data vector is 1e5.

Source of initial codebook

Select Auto-generate to have the block choose the initial codebook values. Choose User defined to enter your own initial codebook values.

Number of levels

Enter the number of codeword vectors, N, in your codebook matrix, where N ≥ 2.

Initial codebook

Enter your initial codebook values. From the Source of initial codebook list, select User defined in order to activate this parameter. The codebook must have the same number of rows as the training set. You must provide at least two codeword vectors.

Distortion measure

When you select Squared error, the block finds the nearest codeword by calculating the squared error (unweighted). When you select Weighted squared error, the block finds the nearest codeword by calculating the weighted squared error.

Weighting factor

Enter a vector or matrix. The block uses these values to compute the weighted squared error. When the weighting factor is a vector, its length must be equal to the number of rows in the training set. This weighting factor is used for each training vector. When the weighting factor is a matrix, it must be the same size as the training set matrix. The individual weighting factors cannot be negative. The weighting factor vector or matrix cannot contain all zeros.

Stopping criteria

Choose Relative threshold to enter the maximum acceptable fractional drop in the squared quantization error. Choose Maximum iteration to specify the number of iterations at which to stop. Choose Whichever comes first and the block stops the iteration process as soon as the relative threshold or maximum iteration value is attained.

Relative threshold

This parameter is available when you choose Relative threshold or Whichever comes first for the Stopping criteria parameter. Enter the value that is the maximum acceptable fractional drop in the squared quantization error.

Maximum iteration

This parameter is available when you choose Maximum iteration or Whichever comes first for the Stopping criteria parameter. Enter the maximum number of iterations you want the block to perform.

Tie-breaking rules

When a training vector has the same distortion for two different codeword vectors, select Lower indexed codeword to associate the training vector with the lower indexed codeword. Select Higher indexed codeword to associate the training vector with the lower indexed codeword.

Codebook update method

When you choose Mean, the new codeword vector is calculated by taking the average of all the training vector values that were associated with the original codeword vector. When you choose Centroid, the block calculates the new codeword vector by taking the weighted average of all the training vector values that were associated with the original codeword vector Note that if, for the Distortion measure parameter, you choose Squared error, the Codebook update method parameter is set to Mean.

Destination

Choose Current model to create a Vector Quantizer block in the model you most recently selected. Type gcs in the MATLAB Command Window to display the name of your current model. Choose New model to create a block in a new model file.

Block type

Select Encoder to design a Vector Quantizer Encoder block. Select Decoder to design a Vector Quantizer Decoder block. Select Both to design a Vector Quantizer Encoder block and a Vector Quantizer Decoder block.

Encoder block name

Enter a name for the Vector Quantizer Encoder block.

Decoder block name

Enter a name for the Vector Quantizer Decoder block.

Overwrite target block

When you do not select this check box and a Vector Quantizer Encoder and/or Decoder block with the same block name exists in the destination model, a new Vector Quantizer Encoder and/or Decoder block is created in the destination model. When you select this check box and a Vector Quantizer Encoder and/or Decoder block with the same block name exists in the destination model, the parameters of these blocks are overwritten by new parameters.

Generate Model

Click this button and VQDTool uses the parameters that correspond to the current plots to set the parameters of the Vector Quantizer Encoder and/or Decoder blocks.

Design and Plot

Click this button to design a quantizer using the parameters on the left side of the GUI and to update the performance curve and entropy plots on the right side of the GUI.

You must click Design and Plot to apply any changes you make to the parameter values in the VQDTool GUI.

Export Outputs

Click this button, or press Ctrl+E, to export the Final Codebook, Mean Squared Error, and Entropy values to the workspace, a text file, or a MAT-file.

## References

Gersho, A. and R. Gray. Vector Quantization and Signal Compression. Boston: Kluwer Academic Publishers, 1992.

## Supported Data Types

• Double-precision floating point