If a variable straight line `x cos alpha+y sin alpha=p` which is a chord of hyperbola `(x^(2))/(a^(2))=(y^(2))/(b^(2))=1 (b gt a)` subtends a right angle at the centre of the hyperbola, then it always touches a fixed circle whose radius, is (a) `(sqrt(a^2+b^2))/(ab)` (b) `(2ab)/(sqrt(a^2+b^2))` (c) `(ab)/(sqrt(b^2-a^2))` (d) `(sqrt(a^2+b^2))/(2ab)`

Options

A. `(sqrt(a^2+b^2))/(ab)`
B. `(2ab)/(sqrt(a^2+b^2))`
C. `(ab)/(sqrt(a^2+b^2))`
D. `(sqrt(a^2+b^2))/(2ab)`

Answer

To solve the problem, we need to analyze the given equation of the variable straight line and the hyperbola, and then derive the radius of the fixed circle that the line touches.

### Step 1: Understand the given equations
The equation of the variable straight line is given as:
\[ x \cos \alpha + y \sin \alpha = p \]
The equation of the hyperbola is:
\[ \frac{x^2}{a^2} - \frac{y^2}{b^2} = 1 \]
where \( b > a \).

### Step 2: Rearranging the line equation
We can rearrange the line equation to express it in a different form:
\[ \frac{x \cos \alpha}{p} + \frac{y \sin \alpha}{p} = 1 \]
This suggests a relationship between the coordinates \( x \) and \( y \) along with the parameters \( \alpha \) and \( p \).

### Step 3: Substitute into the hyperbola equation
To find the relationship between \( p \), \( a \), and \( b \), we can substitute the expressions for \( x \) and \( y \) from the line equation into the hyperbola equation. This gives us:
\[ \frac{x^2}{a^2} - \frac{y^2}{b^2} = 1 \]
Substituting \( x = p \cos \alpha \) and \( y = p \sin \alpha \):
\[ \frac{(p \cos \alpha)^2}{a^2} - \frac{(p \sin \alpha)^2}{b^2} = 1 \]

### Step 4: Simplifying the equation
Expanding this, we have:
\[ \frac{p^2 \cos^2 \alpha}{a^2} - \frac{p^2 \sin^2 \alpha}{b^2} = 1 \]
Factoring out \( p^2 \):
\[ p^2 \left( \frac{\cos^2 \alpha}{a^2} - \frac{\sin^2 \alpha}{b^2} \right) = 1 \]

### Step 5: Setting up the equation for \( p^2 \)
Rearranging gives us:
\[ p^2 = \frac{1}{\frac{\cos^2 \alpha}{a^2} - \frac{\sin^2 \alpha}{b^2}} \]
This can be rewritten as:
\[ p^2 = \frac{a^2 b^2}{b^2 \cos^2 \alpha - a^2 \sin^2 \alpha} \]

### Step 6: Finding the radius of the fixed circle
Since the line subtends a right angle at the center of the hyperbola, we can derive that the radius \( R \) of the fixed circle that the line touches is given by:
\[ R = \frac{ab}{\sqrt{b^2 - a^2}} \]

### Conclusion
Thus, the radius of the fixed circle is:
\[ R = \frac{ab}{\sqrt{b^2 - a^2}} \]

### Final Answer
The correct option is (c) \( \frac{ab}{\sqrt{b^2 - a^2}} \).

---