The answer is easy. It depends!
What does it depend on? How sparse are your matrices? A lot of people think if half the entries in a matrix are zero, then using sparse form is right. WRONG.You don't really gain much there. In fact, it probably runs MUCH more slowly. As a test on a relatvely small pair of matrices...
As = sprand(1000,1000,0.5);
Bs = sprand(1000,1000,0.5);
And that was only for a 1000x1000 pair of matrices.
A significant problem is that sparse matrix multiplication does not use multiple processors. On my own computer, I have 8 real cores. But that sparse multiply uses only ONE of them, whereas the full matrix multiply uses all 8 in parallel.
Now, let me try it again, but for larger matrices that are truly sparse.
As = sprand(5000,5000,0.001);
Bs = sprand(5000,5000,0.001);
As you see in the second test, the sparse multiply whizzes past the full multiply. We are talking tortoise and hare here.
So the real problem is you most likely do not have matrices that are even remotely sparse. Hey, I have a lot of zeros. So sparse must be good. Just having a lot of zeros is not enough. A matrix needs to be seriously sparse for you to see a gain. But when that is the case, sparse is a HUGE benefit.
You may want to do some reading about sparse matrices and the concept of fill-in to understand why all of this works as it does.
For example, consider the matrices:
A1 = sprand(1000,1000,0.5);
B1 = sprand(1000,1000,0.5);
A2 = sprand(5000,5000,0.001);
B2 = sprand(5000,5000,0.001);
Do you see that the product A1*B1 is a completely full matrix? So even though A1 and B1 were sparse in theory, the product A1*B1 is completely full. It is still stored in sparse form of course. And if you then multiply that matrix by ANOTHER sparse matrix, the product will be even slower to compute.