Why does this special case for associativity of a matrix product with a hadamard product hold true?

I found this very useful mathematical property to hold true for a special case. Can someone show me why?

Let A = m x n matrix.

Let B = n x p matrix.

Let C = 1 x p vector.

When the hadamard product B .* C is performed, Matlab duplicates the rows of C n times to form a n x p matrix C1 with n identical rows.

Then the following expression holds true:

A x (B .* C1) = (A x B) .* C2 where Matlab again duplicates the rows of C, but this time the 1 x p row vector is duplicated m times to form m x p matrix C2.

Example:

A = rand(2,5);
B = rand(5,7);
C = rand(1,7);
D1 = A * (B .* C)

D1 = 2×7

1.6311 0.2174 1.4102 1.6453 0.1953 1.0507 0.6540 1.2156 0.2325 1.3234 1.1411 0.1932 0.9036 0.4533

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

D2 = (A * B) .* C

D2 = 2×7

1.6311 0.2174 1.4102 1.6453 0.1953 1.0507 0.6540 1.2156 0.2325 1.3234 1.1411 0.1932 0.9036 0.4533

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

D1 and D2 are identical.

The reason this is so useful for my application is that A and B are system matrices that are calculated only once, with m << n. C changes thousands of times, so the operation (A x B) .* C is many times faster than A x (B .* C). I need to show why this works mathematically so that I can write it up.

Thank you.

Answer 1

[Matt J]

It is clear why this happens when n=1, i.e, when A*B is an outer product. But now for arbitrary n, the matrix product can be written as a sum of outer products, so the result follows because Hadamard product distributes into addition.

[Clay Fulcher]

Thank you Matt. This is very helpful. I can use it for my writeup. However, the C on the left-hand side of the equation is not identical to the C on the right-hand side. They have the same values for each row, but different number of rows. How can that be expressed precisely mathematically?

[Matt J]

You are quite welcome, but if this resolves your question, please Accept-click the answer.

[Clay Fulcher]

I edited my response...still have one unresolved issue...

[Matt J]

However, the C on the left-hand side of the equation is not identical to the C on the right-hand side. They have the same values for each row, but different number of rows. How can that be expressed precisely mathematically?

I don't follow. You said that C is a row vector. I intended C to be a row vector in all my expressions.

[Clay Fulcher]

Yes C is a row vector. However, a Hadamard product requires the two matrices be identical in size. Matlab conveniently recognizes that it must duplicate the rows of a vector in order to perform the Hadamard product, and it does that. On the left-hand side of your equation, C must have the same dimensions as A*B, or [n x p]. On the right-hand side of the equation C must have the same dimension as B, or [m x p]. In other words, if Matlab didn't automatically size the C matrix as needed, the equation can't be exactly correct. What I'm searching for is a mathematical expression, independent of Matlab, that proves this special case.

In general, an associative property for a matrix product with a Hadamard product doesn't exist. i.e., in general,

A * (B .* C) not equal to (A * B) .* C.

[Matt J]

Maybe this is what you are looking for:

where denotes a column vector of k ones.

[Clay Fulcher]

Yes this works! Thank you!

Answer 2

[Paul]

Hi Clay,

If you're willing to accept (or show, which I think would be straight forward) that, in Matlab, matrix .* row is equal to matrix*diag(row), then it follows that the two cases are equal to each other after converting both to ordinary matrix multiplication.

rng(100);
A = rand(2,5);
B = rand(5,7);
C = rand(1,7);
D1 = A * (B .* C);
D2 = (A * B) .* C;
norm(D1 - D2)

ans = 2.4991e-16

D1prime = A*(B*diag(C));
norm(D1-D1prime)

ans = 0

D2prime = (A*B)*diag(C);
norm(D2-D2prime)

ans = 0

[Clay Fulcher]

Thank you Paul. I didn't realize matrix .* row = matrix * diag(row) . This is a excellent way to express it. Much appreciated.