Solution set of the inequality
`( cot^(-1) x)^(2) - ( 5 cot^(-1) x) + 6 gt 0` is

Options

A. `(cot 3, cot 2)`
B. `( - infty, cot 2) cup ( cot 2, infty)`
C. `( cot 2, infty)`
D. None of the above

Answer

To solve the inequality \( (\cot^{-1} x)^2 - 5 \cot^{-1} x + 6 > 0 \), we can follow these steps:

### Step 1: Substitute \( \cot^{-1} x \) with \( t \)
Let \( t = \cot^{-1} x \). The inequality then becomes:
\[ t^2 - 5t + 6 > 0 \]

### Step 2: Factor the quadratic expression
Next, we can factor the quadratic expression:
\[ t^2 - 5t + 6 = (t - 2)(t - 3) \]
Thus, the inequality can be rewritten as:
\[ (t - 2)(t - 3) > 0 \]

### Step 3: Determine the critical points
The critical points from the factors are \( t = 2 \) and \( t = 3 \). These points divide the number line into intervals. We will test the sign of the product \( (t - 2)(t - 3) \) in each interval:
- Interval 1: \( (-\infty, 2) \)
- Interval 2: \( (2, 3) \)
- Interval 3: \( (3, \infty) \)

### Step 4: Test the intervals
1. **For \( t < 2 \)** (e.g., \( t = 0 \)):
\[ (0 - 2)(0 - 3) = ( -2)( -3) = 6 > 0 \quad \text{(True)} \]

2. **For \( 2 < t < 3 \)** (e.g., \( t = 2.5 \)):
\[ (2.5 - 2)(2.5 - 3) = (0.5)( -0.5) = -0.25 < 0 \quad \text{(False)} \]

3. **For \( t > 3 \)** (e.g., \( t = 4 \)):
\[ (4 - 2)(4 - 3) = (2)(1) = 2 > 0 \quad \text{(True)} \]

### Step 5: Combine the results
The inequality \( (t - 2)(t - 3) > 0 \) holds true in the intervals:
\[ (-\infty, 2) \cup (3, \infty) \]

### Step 6: Convert back to \( x \)
Since \( t = \cot^{-1} x \), we need to find the corresponding values of \( x \):
- For \( t < 2 \):
\[ \cot^{-1} x < 2 \implies x > \cot(2) \]
- For \( t > 3 \):
\[ \cot^{-1} x > 3 \implies x < \cot(3) \]

### Final Solution Set
Thus, the solution set for the inequality is:
\[ (-\infty, \cot(3)) \cup (\cot(2), \infty) \]