How to manually construct or modify a cross-validation object in MATLAB?

12 views (last 30 days)
Short question:
I want your help to manually construct or modify Matlab cross-validation (CV) object created using function cvpartition when it's known which observations belong to training and which to validation partitions.
Extended question:
I have a kind of specific dataset in which several observations (each represented by a separate row) belong to the same object of investigation (e.g. measurements of the same object were performed several times). Each object (and it's observation) belongs to a certain class (e.g. yellow object, green object, etc.) as well. The problem is this: it's needed a CV object in which all observations of the same object belong to the same (either training or validation) partition as a block of data (unfortunately I'm not sure how to implement this) and the data would be distributed to partitions in stratified way (luckily, this is implemented in function cvpartition). I tried to access indices of CV object (cvo.indices) but they are locked. Here cvo is:
cvType = 'kfold';
nPartitions = 5;
cvo = cvpartition(my_classes,cvType,nPartitions);
Your insights and ideas how to construct or modify cvo manually are welcomed.
  2 Comments
Jan
Jan on 20 Dec 2018
[MOVED from flags] Flagged as Unclear by James Ratti on 19 Oct 2018.
This solution no longer works - was there a concurrent patch that provided a workaround?
@James: This is the question. Which solution do you mean?
Rik
Rik on 29 Mar 2021
Comment posted as flag by patrck rich:
the solution is no longer working in recent versions
That is always a risk when trying to modify internal files. However, I ran the lines below in the online interface (I cleared the output, as I don't expect Mathworks would be happy with me sharing their files..)
%type(which('cvpartition'))
OS={'Windows','macOS','Linux'};fprintf('This is version %s, running on %s.',version,OS{[ispc ismac isunix&&~ismac]})
This is version 9.10.0.1613233 (R2021a), running on Linux.
As far as I can judge from the code I can read, I don't see a reason why it wouldn't still work, but feel free to post a comment explaining which exact steps you took and which lines you modified.

Sign in to comment.

Accepted Answer

Vilmantas Gegzna
Vilmantas Gegzna on 22 Apr 2015
Edited: Vilmantas Gegzna on 22 Apr 2015
I found, that I can modify the cross-validation object (CVO) which class is cvpartition by modifying several lines in cvpartition.m file from:
properties(GetAccess = 'private', SetAccess = 'private')
to:
properties(GetAccess = 'public', SetAccess = 'public')
This allowed me to modify indices of CVO manually.
I wonder if there are any other way to set indices of CVO without changing the lines in cvpartition.m? This is the case when I want to run Matlab on my campus computer.
  2 Comments
RZM
RZM on 6 Jul 2018
I cannot change the properties, it pops up an error message that Access is denied!
Sergio Gutierrez
Sergio Gutierrez on 11 Aug 2018
I have done the same modification as Albert Sama said in the below comment. You require administrative rights to make changes. I used Notepad++ in administrator mode.

Sign in to comment.

More Answers (3)

Jo
Jo on 4 Nov 2016
Hi,
It seems that one year later your solution is not working anymore, maybe they changed the function.
Indeed, I tried to modify as you suggested "properties(GetAccess = 'public', SetAccess = 'public')", but it only allows me to access to the number of observations and not the indices. Indices are created by "methods" defined in cvpartition.m (test, training). And I do not understand how to change these to choose my own indices.
Maybe if you have progessed in all that since you first posted your question.... I would be really greatful to have some help in there.
Using random crossvalidation is of none interest for me, I need to choose the block myself to have a more robust solution. Thanks to anyone that can help me!!!
  2 Comments
Albert Sama
Albert Sama on 2 Nov 2017
I have done a similar modification on the source code that works on MATLAB R2017b.
Basically, the file cvpartitionImpl.m (at ...\toolbox\stats\stats\+internal\+stats) has to be modified as the first answer specifies, making indices public:
properties(GetAccess = public, SetAccess = public)
indices = [];
Group = [];
holdoutT = [];
end
This way, indices are updateable, for instance:
cvp = cvpartition(500,'KFold',9);
cvp.Impl.indices=cvp.Impl.indices(end:-1:1);
Evelyn Tang
Evelyn Tang on 9 Nov 2018
This is wonderful and just what I need, thank you so much! Works perfectly :)

Sign in to comment.


Roberto Herrera-Lara
Roberto Herrera-Lara on 14 Apr 2015
try to do this https://chrisjmccormick.wordpress.com/2013/07/31/k-fold-cross-validation-with-matlab-code/
  1 Comment
Vilmantas Gegzna
Vilmantas Gegzna on 22 Apr 2015
Edited: Vilmantas Gegzna on 22 Apr 2015
Thank you, Roberto, your link provided an extremely good description of k-fold cross-validation and updated my knowledge. Unfortunately, I was searching for a bit different thing.

Sign in to comment.


John Smith
John Smith on 21 Nov 2018
Two small additions:
2.Restart Matlab after that

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!