Tune Fuzzy Trees

You can tune the parameters of a FIS tree using a similar two-step process as shown in Tuning Fuzzy Inference Systems.

• Learn and tune the rules of the FISs in the tree.

• Learn the MF parameters of the FISs in the tree.

Create a FIS tree to model for $\frac{\mathrm{sin}\left(\mathit{x}\right)+\mathrm{cos}\left(\mathit{x}\right)}{\mathrm{exp}\left(\mathit{x}\right)}$ as shown in the following figure. For more information on creating FIS trees, see Fuzzy Trees.

Create `fis1` as a Sugeno type FIS, which results in a faster tuning process due to computationally efficient defuzzification method. Add two inputs, both with range the [0, 10] and with three MFs each. Use a smooth differentiable MF, such as `gaussmf`, to match the characteristics of the data type you are modeling.

```fis1 = sugfis('Name','fis1'); fis1 = addInput(fis1,[0 10],'NumMFs',3,'MFType','gaussmf'); fis1 = addInput(fis1,[0 10],'NumMFs',3,'MFType','gaussmf');```

Add an output with the range [–1.5, 1.5] having nine MFs corresponding to the nine possible input MF combinations. Doing so provides maximum granularity for the FIS rules. Set the output range according to the possible values of $\mathrm{sin}\left(\mathit{x}\right)+\mathrm{cos}\left(\mathit{x}\right)$.

`fis1 = addOutput(fis1,[-1.5 1.5],'NumMFs',9);`

Create `fis2` as a Sugeno type FIS. Add two inputs. Set the range of the first input to [-1.5, 1.5], which matches the range of the output of `fis1`. The second input is the same as the inputs of `fis1`. Therefore, use the same input range, [0, 10]. Add three MFs for each of the inputs.

```fis2 = sugfis('Name','fis2'); fis2 = addInput(fis2,[-1.5 1.5],'NumMFs',3,'MFType','gaussmf'); fis2 = addInput(fis2,[0 10],'NumMFs',3,'MFType','gaussmf');```

Add an output with range [0 1] and nine MFs. The output range is set according to the possible values of $\frac{\mathrm{sin}\left(\mathit{x}\right)+\mathrm{cos}\left(\mathit{x}\right)}{\mathrm{exp}\left(\mathit{x}\right)}$.

`fis2 = addOutput(fis2,[0 1],'NumMFs',9);`

Connect the inputs and the outputs as shown in the diagram. Output 1 of `fis1` connects to input 1 of `fis2`, inputs 1 and 2 of `fis1` connect to each other, and input 2 of `fis1` connects to input 2 of `fis2`.

```con1 = ["fis1/output1" "fis2/input1"]; con2 = ["fis1/input1" "fis1/input2"]; con3 = ["fis1/input2" "fis2/input2"];```

Finally, create a FIS tree using the specified FISs and connections.

`fisT = fistree([fis1 fis2],[con1;con2;con3]);`

Add an additional output to the FIS tree to access the output of `fis1`.

`fisT.Outputs = ["fis1/output1";fisT.Outputs];`

Generate input and output training data.

```x = (0:0.1:10)'; y1 = sin(x)+cos(x); y2 = y1./exp(x); y = [y1 y2];```

Tune the FIS tree parameters in two steps. First, learn the rules of the FIS tree using a global optimization method (particle swarm for this example).

`options = tunefisOptions('Method','particleswarm','OptimizationType','learning');`

This tuning step uses a small number of iterations to learn a rule base without overfitting the training data. The rule base provides an educated initial condition for the second step to optimize all the FIS tree parameters together. Set the maximum iteration number to 5, and learn the rule base.

```options.MethodOptions.MaxIterations = 5; rng('default') % for reproducibility fisTout1 = tunefis(fisT,[],x,y,options);```
``` Best Mean Stall Iteration f-count f(x) f(x) Iterations 0 100 0.6682 0.9395 0 1 200 0.6682 1.023 0 2 300 0.6652 0.9308 0 3 400 0.6259 0.958 0 4 500 0.6259 0.918 1 5 600 0.5969 0.9179 0 Optimization ended: number of iterations exceeded OPTIONS.MaxIterations. ```

Next, to tune all the FIS tree parameters, use a local optimization method (pattern search for this example). Local optimization is generally faster than global optimization and can produce better results when the input fuzzy system parameters are already consistent with the training data.

Use the `patternsearch` method for optimization. Set the number of iterations to 25.

```options.Method = 'patternsearch'; options.MethodOptions.MaxIterations = 25;```

Use `getTunableSettings` to obtain input, output, and rule parameter settings from the FIS tree.

`[in,out,rule] = getTunableSettings(fisTout1);`

Tune the FIS tree parameters.

```rng('default') % for reproducibility fisTout2 = tunefis(fisTout1,[in;out;rule],x,y,options);```
```Iter Func-count f(x) MeshSize Method 0 1 0.596926 1 1 3 0.551284 2 Successful Poll 2 13 0.548551 4 Successful Poll 3 20 0.546331 8 Successful Poll 4 33 0.527482 16 Successful Poll 5 33 0.527482 8 Refine Mesh 6 61 0.511532 16 Successful Poll 7 61 0.511532 8 Refine Mesh 8 92 0.505355 16 Successful Poll 9 92 0.505355 8 Refine Mesh 10 128 0.505355 4 Refine Mesh 11 175 0.487734 8 Successful Poll 12 212 0.487734 4 Refine Mesh 13 265 0.487734 2 Refine Mesh 14 275 0.486926 4 Successful Poll 15 328 0.486926 2 Refine Mesh 16 339 0.483683 4 Successful Poll 17 391 0.483683 2 Refine Mesh 18 410 0.442624 4 Successful Poll 19 462 0.442624 2 Refine Mesh 20 469 0.44051 4 Successful Poll 21 521 0.44051 2 Refine Mesh 22 542 0.435381 4 Successful Poll 23 594 0.435381 2 Refine Mesh 24 614 0.398872 4 Successful Poll 25 662 0.398385 8 Successful Poll 26 698 0.398385 4 Refine Mesh Maximum number of iterations exceeded: increase options.MaxIterations. ```

The optimization cost reduces from 0.59 to 0.39 in the second step.

Alternatively, you can tune the specific fuzzy systems within a FIS tree. For this example, after learning the rule base of the FIS tree, separately tune `fis1` and `fis2` parameters.

To obtain parameter settings of a FIS within the FIS tree, use `getTunableSettings`, specifying the FIS name. First, get the parameter settings for `fis1`.

`[in,out,rule] = getTunableSettings(fisTout1,"FIS","fis1");`

Tune the parameters of `fis1`.

```rng('default') fisTout2 = tunefis(fisTout1,[in;out;rule],x,y,options);```
```Iter Func-count f(x) MeshSize Method 0 1 0.596926 1 1 3 0.551284 2 Successful Poll 2 18 0.510362 4 Successful Poll 3 28 0.494804 8 Successful Poll 4 56 0.494804 4 Refine Mesh 5 84 0.493422 8 Successful Poll 6 107 0.492883 16 Successful Poll 7 107 0.492883 8 Refine Mesh 8 136 0.492883 4 Refine Mesh 9 171 0.492883 2 Refine Mesh 10 178 0.491534 4 Successful Poll 11 213 0.491534 2 Refine Mesh 12 229 0.482682 4 Successful Poll 13 264 0.482682 2 Refine Mesh 14 279 0.446645 4 Successful Poll 15 313 0.446645 2 Refine Mesh 16 330 0.44657 4 Successful Poll 17 364 0.44657 2 Refine Mesh 18 384 0.446495 4 Successful Poll 19 418 0.446495 2 Refine Mesh 20 461 0.445938 4 Successful Poll 21 495 0.445938 2 Refine Mesh 22 560 0.422421 4 Successful Poll 23 594 0.422421 2 Refine Mesh 24 597 0.397265 4 Successful Poll 25 630 0.397265 2 Refine Mesh 26 701 0.390338 4 Successful Poll Maximum number of iterations exceeded: increase options.MaxIterations. ```

In this case, the optimization cost is improved by tuning only `fis1` parameter values.

Next, obtain the parameter settings for `fis2` and tune the `fis2` parameters.

```[in,out,rule] = getTunableSettings(fisTout2,"FIS","fis2"); rng('default') fisTout3 = tunefis(fisTout2,[in;out;rule],x,y,options);```
```Iter Func-count f(x) MeshSize Method 0 1 0.390338 1 1 2 0.374103 2 Successful Poll 2 5 0.373855 4 Successful Poll 3 10 0.356619 8 Successful Poll 4 33 0.356619 4 Refine Mesh 5 43 0.350715 8 Successful Poll 6 65 0.349417 16 Successful Poll 7 65 0.349417 8 Refine Mesh 8 87 0.349417 4 Refine Mesh 9 91 0.349356 8 Successful Poll 10 112 0.349356 4 Refine Mesh 11 138 0.346102 8 Successful Poll 12 159 0.346102 4 Refine Mesh 13 172 0.345938 8 Successful Poll 14 193 0.345938 4 Refine Mesh 15 222 0.342721 8 Successful Poll 16 244 0.342721 4 Refine Mesh 17 275 0.342721 2 Refine Mesh 18 283 0.340727 4 Successful Poll 19 312 0.340554 8 Successful Poll 20 335 0.340554 4 Refine Mesh 21 366 0.340554 2 Refine Mesh 22 427 0.337873 4 Successful Poll 23 457 0.337873 2 Refine Mesh 24 521 0.33706 4 Successful Poll 25 551 0.33706 2 Refine Mesh 26 624 0.333193 4 Successful Poll Maximum number of iterations exceeded: increase options.MaxIterations. ```

The optimization cost is further reduced by tuning the `fis2` parameter values. To avoid overfitting of individual FIS parameter values, you can further tune both `fis1` and `fis2` parameters together.

```[in,out,rule] = getTunableSettings(fisTout3); rng('default') fisTout4 = tunefis(fisTout3,[in;out;rule],x,y,options);```
```Iter Func-count f(x) MeshSize Method 0 1 0.333193 1 1 8 0.326804 2 Successful Poll 2 91 0.326432 4 Successful Poll 3 116 0.326261 8 Successful Poll 4 154 0.326261 4 Refine Mesh 5 205 0.326261 2 Refine Mesh 6 302 0.326092 4 Successful Poll 7 352 0.326092 2 Refine Mesh 8 391 0.325964 4 Successful Poll 9 441 0.325964 2 Refine Mesh 10 478 0.32578 4 Successful Poll 11 528 0.32578 2 Refine Mesh 12 562 0.325691 4 Successful Poll 13 612 0.325691 2 Refine Mesh 14 713 0.229273 4 Successful Poll 15 763 0.229273 2 Refine Mesh 16 867 0.22891 4 Successful Poll 17 917 0.22891 2 Refine Mesh 18 1036 0.228688 4 Successful Poll 19 1086 0.228688 2 Refine Mesh 20 1212 0.228688 1 Refine Mesh 21 1266 0.228445 2 Successful Poll 22 1369 0.228441 4 Successful Poll 23 1381 0.227645 8 Successful Poll 24 1407 0.226125 16 Successful Poll 25 1407 0.226125 8 Refine Mesh 26 1447 0.226125 4 Refine Mesh Maximum number of iterations exceeded: increase options.MaxIterations. ```

Overall, the optimization cost is smaller after using the three tuning steps.