Without seeing what your matrices are, it is difficult to help you too much.
How do you know what the "right" V is? Yes, you want the residuals to be positive, but people want lots of things they cannot have.
There is no constraint in the use of pinv that this difference will be positive. You will get whatever your matrix indicates is appropriate to solve the problem, in a least squares sense. That means you will have lots of positive and negative values!
You can use lsqlin. Apply inequality constraints. Note that you will still get tiny negative results, due to floating point arithmetic. Again, people want lots of things they cannot have. You can always add a tiny amount to the constraint, constraining
X + N*V >= delta
for small delta.
