The simplest answer is to use my blktridiag utility from the file exchange.
By default, it produces a sparse matrix. Since you did not ask for that, I put a call to full around it.
Of course, if this is homework, then my answer is useless. If it is not, then my answer is the perfect solution. Find blktridiag on the File Exchange at this link:
Could you solve it using blkdiag? Well, yes, if this is your homework, with some effort. In that case, you you create the main diagonal of matrices from A? How might you do that? Perhaps that is a sufficient hint to get you started, IF this is your homework. Better is to just use code designed to solve your problem, efficient, already written and debugged.