The unit vector which is orthogonal to the vector `3hati+2hatj+6hatk` and is coplanar with the vectors `2hati+hatj+hatk` and `hati-hatj+hatk` is (A) `(2hati-6hatj+hatk)/sqrt(41)` (B) `(2hati-3hatj)/sqrt(3)` (C) `3hatj-hatk)/sqrt(10)` (D) `(4hati+3hatj-3hatk)/sqrt(34)`

Options

A. `(2hati-6hatj+hatk)/(sqrt(41))`
B. `(2hati-3hatj)/(sqrt(13))`
C. `(3hatj-hatk)/(sqrt(10))`
D. `(4hati+3hatj-3hatk)/(sqrt(34))`.

Answer

To solve the problem, we need to find a unit vector that is orthogonal to the vector \( \mathbf{b} = 3\hat{i} + 2\hat{j} + 6\hat{k} \) and is coplanar with the vectors \( \mathbf{u} = 2\hat{i} + \hat{j} + \hat{k} \) and \( \mathbf{v} = \hat{i} - \hat{j} + \hat{k} \).

### Step 1: Define the unit vector
Let the unit vector be represented as \( \mathbf{a} = \lambda_1 \hat{i} + \lambda_2 \hat{j} + \lambda_3 \hat{k} \).

### Step 2: Establish the coplanarity condition
Since \( \mathbf{a} \) is coplanar with \( \mathbf{u} \) and \( \mathbf{v} \), we can express \( \mathbf{a} \) as a linear combination of \( \mathbf{u} \) and \( \mathbf{v} \):
\[ \mathbf{a} = \alpha \mathbf{u} + \beta \mathbf{v} \]
Expanding this gives:
\[ \mathbf{a} = \alpha (2\hat{i} + \hat{j} + \hat{k}) + \beta (\hat{i} - \hat{j} + \hat{k}) = (2\alpha + \beta)\hat{i} + (\alpha - \beta)\hat{j} + (\alpha + \beta)\hat{k} \]
Let:
\[ \lambda_1 = 2\alpha + \beta, \quad \lambda_2 = \alpha - \beta, \quad \lambda_3 = \alpha + \beta \]

### Step 3: Apply the orthogonality condition
The vector \( \mathbf{a} \) must be orthogonal to \( \mathbf{b} \). This gives us the condition:
\[ \mathbf{a} \cdot \mathbf{b} = 0 \]
Calculating the dot product:
\[ (2\alpha + \beta) \cdot 3 + (\alpha - \beta) \cdot 2 + (\alpha + \beta) \cdot 6 = 0 \]
Expanding this:
\[ 3(2\alpha + \beta) + 2(\alpha - \beta) + 6(\alpha + \beta) = 0 \]
This simplifies to:
\[ 6\alpha + 3\beta + 2\alpha - 2\beta + 6\alpha + 6\beta = 0 \]
Combining like terms:
\[ (6\alpha + 2\alpha + 6\alpha) + (3\beta - 2\beta + 6\beta) = 0 \]
This results in:
\[ 14\alpha + 7\beta = 0 \]
From here, we can express \( \beta \) in terms of \( \alpha \):
\[ \beta = -2\alpha \]

### Step 4: Substitute back into the expressions for \( \lambda_1, \lambda_2, \lambda_3 \)
Substituting \( \beta = -2\alpha \) into the expressions for \( \lambda_1, \lambda_2, \lambda_3 \):
\[ \lambda_1 = 2\alpha - 2\alpha = 0 \]
\[ \lambda_2 = \alpha - (-2\alpha) = 3\alpha \]
\[ \lambda_3 = \alpha + (-2\alpha) = -\alpha \]
Thus, we have:
\[ \mathbf{a} = 0\hat{i} + 3\alpha\hat{j} - \alpha\hat{k} = \alpha(0\hat{i} + 3\hat{j} - \hat{k}) \]

### Step 5: Normalize the vector to find the unit vector
The magnitude of \( \mathbf{a} \) is:
\[ \|\mathbf{a}\| = \sqrt{0^2 + (3\alpha)^2 + (-\alpha)^2} = \sqrt{9\alpha^2 + \alpha^2} = \sqrt{10\alpha^2} = \sqrt{10}|\alpha| \]
To find the unit vector:
\[ \hat{a} = \frac{\mathbf{a}}{\|\mathbf{a}\|} = \frac{3\hat{j} - \hat{k}}{\sqrt{10}} \]

### Final Answer
Thus, the required unit vector is:
\[ \hat{a} = \frac{3\hat{j} - \hat{k}}{\sqrt{10}} \]