if A and B are square matrices of same order such that A*=A and B* = B, where A* denotes the conjugate transpose of A, then `(AB-BA)* is equal to

Options

A. null matrix
B. `AB-BA`
C. BA-AB
D. none of these

Answer

To solve the problem, we need to find the expression \((AB - BA)^*\), where \(A^*\) denotes the conjugate transpose of matrix \(A\). Given that both \(A\) and \(B\) are Hermitian matrices (i.e., \(A^* = A\) and \(B^* = B\)), we can proceed as follows:

### Step-by-step Solution:

1. **Start with the expression**:
\[ (AB - BA)^* \]

2. **Use the property of conjugate transpose**:
The property of the conjugate transpose states that \((XY)^* = Y^*X^*\) for any two matrices \(X\) and \(Y\). Therefore, we can apply this property to our expression:
\[ (AB - BA)^* = (AB)^* - (BA)^* \]

3. **Calculate \((AB)^*\)**:
Using the property mentioned:
\[ (AB)^* = B^*A^* \]
Since \(A^* = A\) and \(B^* = B\), we have:
\[ (AB)^* = BA \]

4. **Calculate \((BA)^*\)**:
Similarly, we find:
\[ (BA)^* = A^*B^* = AB \]

5. **Substitute back into the expression**:
Now we substitute back into our expression:
\[ (AB - BA)^* = BA - AB \]

6. **Final result**:
Therefore, we conclude that:
\[ (AB - BA)^* = BA - AB \]

### Final Answer:
\[ (AB - BA)^* = BA - AB \]