Make an machine vision simulatie using simulink 3D surrounding

I have a challenging assignment, but I can't figure it out. I hope someone can help me. I want to simulate a machine vision system in Matlab. An onion lies on a conveyor belt and falls off according to a pre-calculated path. I used formulas to calculate the x and y distance during the fall of the onion. I now want to drop the onion using the 3D built-in functions of simulink. That is the first problem: how can I drop an onion according to 2 variables (x and y that depend on time t) and simulate this in the new simulation environment of simulink (so not with vr sink).

The second thing I want: position a camera under the conveyor belt and take a photo of the onion as it falls. I would like to make a calculation on this photo. I want to calculate what percentage of the onion is visible with the camera. I would prefer to achieve this with a built-in function, but I think it will ultimately have to be done with a Matlab script. So what I want: position a camera and when it sees the onion falling by, it takes a photo. From the photo I want to determine what the visible surface of the onion is. A challenging job, right? I really hope someone can help me with this!

Answer 1

[Amish]

Hi Sjoerd,

This indeed seems like a challenging problem, however you can utilize the "Simulink 3D Animation toolbox" and "Image Processing Toolbox" to try to accomplish it. You can look at the task in the form on different manageable parts:

Onion Fall's Modelling: This can be done using the "integrator" block, assuming you already have the equations for the trajectory (x(t) and y(t)).

3D modelling: You can use the basic shapes provided by the toolbox to represent the onion and the conveyor belt. Alternatively, you can also import them from some other modelling software.

Fall Animation: Assuming that you have the necessary equations (for x(t), y(t) and z(t)), you will need to need a Simulink model to implement the equations and then connect its output to a MATLAB Function block that updates the position of the onion in the virtual scene. Since you prefer not using VR Sink, you might need to directly manipulate the 3D object properties through MATLAB scripts triggered by Simulink events.

For accomplishing the second part of the problem of camera positioning, and image analysis, you will need to setup a "virtual Camera" using the "3D Animation toolbox". The image capturing can be done by programming the Simulink model's script to trigger a snapshot when the Onion object is in the camera's field of view.

The image analysis can be done using the "Image Processing Toolbox" by converting the images into grayscale and then using techniques like edge detection and segmentation (built-in), to find the visible surface area of the onion.

(This might involve converting pixel measurements to real-world measurements based on the scale of your simulation)

The accuracy will lie in iteratively refining your models and scripts based on the results you obtain at each step.

Here are some references you can use:

Modelling equations: https://www.mathworks.com/help/simulink/ug/model-a-differential-algebraic-equation.html

Simulink Event trigger: https://www.mathworks.com/help/simulink/slref/simulink.event.inputwrite.html

3D environments in Simulink: https://www.mathworks.com/help/sl3d/3d-environment.html

3D animations in Simulink: https://www.mathworks.com/help/sl3d/

Hope this helps!