Providing better feedback to students when assessing FUNCTION problems in MATLAB Grader

I'm working on some MATLAB coding assignments for students and am using MATLAB Grader to provide assessment and feedback. I would like to have students write functions. This is to achieve two objectives, one to further their understanding of concepts from the course and so that when they are done, they can use their function (that they know and understand) to solve other homework problems, use it in labs, homeworks and exams for computing values, etc.. Much like they might do in industry.

Since within Grader one can't assess intermediate variables within the student's functions I was looking for a way to better help them debug their code. To do this I added a variable to the return variables that is a structure. I called it debug. Then for my assessments I assigned fields to intermediate values that I can check in the assessment to provide feedback to the students if these values are wrong.

For example a snippet of a learner template for a function to perform the inverse DFT is shown below. The reference template would be similar and pass back the variables that I think would be useful to assess and provide feedback on if not correct.

%
%  IDFT function
%
%  The function definition is locked in the learner template
%
function [xOut, debug] = IDFT( ReX, ImX )


%  Create a variable debug that is a structure.
%  This line is locked in the learner template
debug = struct;

%
%  ... Other code and comments
%

%  Save debug return variables for my assessment and checking 
%  These lines are locked in the learner template

debug.variable1 = variable1;   %  Save and return for debug and assessment
debug.variable2 = variable2;   %  Save and return for debug and assessment

%  Students can define their own fields to return variables for
%  checking in their calling code.
%  These lines are written by the learner and not locked

debug.learnerVar1 = learnerVar1;
debug.learnerVar2 = learnerVar2;

The assessment can compare the learner values to the reference values. However I had to do the comparison using the assert function as the assessVariableEqual function wouldn't accept the structure and field format for the first variable (in quotations) (e.g. assessVariableEqual('debug.variable1' , debug_ref.variable1) does not work)

For example:

%  Call the learner function.  Include both xOut and debug if you want to get both
%  the output and the debug values passed back to the calling function

[xOut, debug] = IDFT(ReX, ImX);  %  Call your function.  Include the debug output

%  Call the reference function.  Include the debug output
[xOut_ref, debug_ref] = reference.IDFT( ReX, ImX )

assert( debug.variable1 == debug_ref.variable1, 'The scaled real values of the IDFT input are incorrect.  Check your code starting there' );

This works, but seems complicated. I was wondering if anyone else has solved this in a different way? Any feedback would be great!

Thanks,

Mark

Answer 1

[Cris LaPierre]

You could assign the value of a specific field to a separate variable, and then check that using assessVariableEqual.

variable1 = debug.variable1;
assessVariableEqual('variable1' , debug_ref.variable1)

One considernation to keep in mind when deciding between assessVariableEqual and assert is the default feedback in assessVariableEqual will include the variable name. This may actually cause confusion with your learners since they did not create this variable - you did.

Some answers that may be insightful are below.

• https://www.mathworks.com/matlabcentral/answers/458425-help-with-matlab-error?s_tid=srchtitle
• https://www.mathworks.com/matlabcentral/answers/634764-is-it-possible-to-suppress-the-default-error-message-of-the-function-assessvariableequal
• https://www.mathworks.com/matlabcentral/answers/519580-matlab-grader-complex-and-conditioned-assessments-for-alternative-solutions-differentiable-error-m
• https://www.mathworks.com/matlabcentral/answers/590032-matlab-grader-feedback-for-common-errors

I try to avoid modifying the code to accomodate grading. The idea is they are learning how to do code up some task, so I don't want them to get confused by code added purely for assessment purposes. Also, this can lead to over-constraining a problem, forcing students to solve it the same way I did rather than letting them come up with their own solution. Not saying you are, but just some things to take into consideration as you design your problems.

Some options I might explore would be

1. Converting the function-type problems to a script-type problem so that intermediate variables can be assessed.
2. Giving students a way to visualize the output, and including an image of the correct output in the feedback.

[Mark Thompson]

Cris,

Thanks again for the feedback. I'd still like to stick with the function-type problem as I'd like students to walk away with a function they can later use.

I'll keep the special code for assessment they need to a minimum (just assigning variables) and call it out that it is for assessment only and how they can use the feature for debug. I hope that works.

Good comments. I have put some figures in the additional feedback that show the desired output at certain stages and put some code in there to show how to create the plots.

I have noticed that you can't copy code from the additional feedback to use back in the "Code to Call the function" section, so I'll have to keep it to a minimum.

The links you provided are helpful and I'll review them again.

Thanks,

Mark

[Cris LaPierre]

I would consider not being able to copy from the additional feedback section at least an enhancement request, if not a bug. Can you report that here?

[Mark Thompson]

Done. I submitted a bug report.

[Cris LaPierre]

The current workaround might be to add the feedback code to the learner template (or in the 'Code to call your function' section of function type problems) and comment it out. Students could then uncomment it if they want to use it. Your feedback could tell students to uncomment parts of it.

Another workaround option is to put the code in a script that you save and attach to the problem. Then, all students have to do is call the script by name at the correct place. This way, you could still give them the script name in the feedback. It'd be easier to type a file name than several lines of code.

These are, of course, just temporary workarounds until copy/paste does work.

[Mark Thompson]

Cris,

Thanks for the feedback. I think one of those will work.

I'm considering putting the code in a separate file(s) and attaching it to the problem. Then in the feedback still including the code snip so they can see it along with other feedback. Then I'll direct students where they can copy from or what script to call. I think that should work for now.

If the script files are attached to the problem are they essentially on the "path" so that a script can be run by referencing it in the "Code to call your function" section. I think that is what you are implying above.

I'm also considering adding one very simple assignment that will allow them to "cut their teeth" on writing a function and using Grader as a tool to help them understand the approach. Something that exercises the functionality of Grader and gets them used to it.

This is the second semester I have used the tool. The first time I didn't do a good job in providing helpful assessments and feedback so spent a good deal of time working with students directly to help them debug their code. I'm hoping by incorporating your suggestions that they will be able to debug their problems on their own and have a better learning experience.

I was however, encouraged when a number of students took their finished working functions and continued to use them over the semester as part of their "toolbox".

Mark

[Cris LaPierre]

Yes, attached files are in the current folder, so can be called by name without needing to include the path. They will appear in the outupt of dir if you want a sanity check.

A very simple function problem (My first FUNCTION problem) is included in the Getting Started with MATLAB Grader collection you see when on the Add Problem screen.

All problems in this collection were created by MathWorks to demonstrate features of Grader. Feel free to modify and adapt them to fit your needs. The 'Get started with' line tries to convey what each problem is demonstrating.

You can also find a table with information on what each problem is demonstrating at the end of the Teaching with MATLAB self-paced course.

There are also additional catalogs you should be able to access that contain starter problems in specific areas.

[Mark Thompson]

I changed my mind and I tried putting the code to help debug the function in the "Code to call your function" section and commenting it out. I left those code pieces in the feedback so students could see them (but they can't copy them yet) and instructed them to uncomment the code in the "calling" section. That section has became large and a bit unwieldy, however.

I found one problem however. While you can use CTRL^R to comment code lines you cannot use CTRL^T to uncomment code in that section as in the normal editor. At least in the Chrome browser. That section also doesn't have a button to comment or uncomment code. I think that's a big drawback to that approach, making it difficult to make code executable (deleting all the % symbols).

I'm going to punt and put all the tests into a separate file that students can download and copy from to the "calling" section when the get the assignment. It's clunky but the only approach that I can see right now that allows students to see, use and modify the test code easily.

Mark

[Cris LaPierre]

I've inquired about the ctrl+T before, but I believe that is not a shortcut we could repurpose like with ctrl+R. I think the feedback that the 'Code to call your funciton' should have an uncomment button is a good enhancement request.

[Mark Thompson]

Ok. Thanks. I already submitted it as a bug. Hopefully that will suffice.