Let `alpha epsilon R` and the three vectors `veca=alpha hati+hatj+3hatk, vecb=2hati+hatj-alpha hatk` and `vecc=alpha hati-2hatj+3hatk`. Then the set `S={alpha: veca, vecb` and `vecc` are coplanar}:

Options

A. Is singleton
B. Is empty
C. Contains exaclty two positive numbers
D. Contains exactlly two numbers only one of which is positive.

Answer

To determine the set \( S \) such that the vectors \( \vec{a}, \vec{b}, \) and \( \vec{c} \) are coplanar, we follow these steps:

### Step 1: Write down the vectors
The vectors are given as:
\[ \vec{a} = \alpha \hat{i} + \hat{j} + 3 \hat{k} \]
\[ \vec{b} = 2 \hat{i} + \hat{j} - \alpha \hat{k} \]
\[ \vec{c} = \alpha \hat{i} - 2 \hat{j} + 3 \hat{k} \]

### Step 2: Set up the condition for coplanarity
The vectors \( \vec{a}, \vec{b}, \) and \( \vec{c} \) are coplanar if their scalar triple product is zero. The scalar triple product can be computed using the determinant of a matrix formed by the components of the vectors:
\[ \begin{vmatrix} \alpha & 1 & 3 \\ 2 & 1 & -\alpha \\ \alpha & -2 & 3 \end{vmatrix} = 0 \]

### Step 3: Calculate the determinant
We can compute the determinant:
\[ \begin{vmatrix} \alpha & 1 & 3 \\ 2 & 1 & -\alpha \\ \alpha & -2 & 3 \end{vmatrix} \]

Expanding this determinant:
1. The first row gives:
\[ \alpha \begin{vmatrix} 1 & -\alpha \\ -2 & 3 \end{vmatrix} - 1 \begin{vmatrix} 2 & -\alpha \\ \alpha & 3 \end{vmatrix} + 3 \begin{vmatrix} 2 & 1 \\ \alpha & -2 \end{vmatrix} \]

Calculating each of the 2x2 determinants:
- For \( \begin{vmatrix} 1 & -\alpha \\ -2 & 3 \end{vmatrix} = (1)(3) - (-2)(-\alpha) = 3 - 2\alpha \)
- For \( \begin{vmatrix} 2 & -\alpha \\ \alpha & 3 \end{vmatrix} = (2)(3) - (-\alpha)(\alpha) = 6 + \alpha^2 \)
- For \( \begin{vmatrix} 2 & 1 \\ \alpha & -2 \end{vmatrix} = (2)(-2) - (1)(\alpha) = -4 - \alpha \)

Putting it all together:
\[ \alpha(3 - 2\alpha) - (6 + \alpha^2) + 3(-4 - \alpha) = 0 \]
This simplifies to:
\[ 3\alpha - 2\alpha^2 - 6 - \alpha^2 - 12 - 3\alpha = 0 \]
Combining like terms:
\[ -3\alpha^2 - 18 = 0 \]
This leads to:
\[ 3\alpha^2 + 18 = 0 \]

### Step 4: Solve for \( \alpha \)
Rearranging gives:
\[ \alpha^2 = -6 \]
Since \( \alpha^2 \) cannot be negative for real numbers, there are no real solutions for \( \alpha \).

### Conclusion
Thus, the set \( S \) is empty, meaning there are no values of \( \alpha \) for which the vectors \( \vec{a}, \vec{b}, \) and \( \vec{c} \) are coplanar.

### Final Answer
\[ S = \emptyset \]