Choose optimum neighbor from an end point after applying hough transform

Hi everyone, let's say i have the undermentioned picture:

Ι 've applied Ηough transform in the image of fishing nets. I 've already found the lines, the starting and ending points, the peaks etc.

Starting points (SP) of the main line appear as red 'o' and endpoints (EP) as red 'x'.

Every time i create a square around the endpoint and detect the neighbor starting points (SP), which appear as green 'o', in the rectangle.

The lines which start from the green starting point are pictured by magenta color .

I have a matrix where i store the distance of each starting point from the endpoint and i pick the one with the minimum value. I pick the closest.

The problem is, that some lines are long and others are short, it would be better to find a way to chose a line that would be as nearer as it can to the endpoint and also be as long as possible.

Any ideas on how can i do this?

I am begging you, to find me a solution. Cheers