Find the shortest distance between the lines:
`vec(r) = hat(i) + 2 hat(j) - 3 hat(k) + lambda (3 hat(i) - 4 hat(j) - hat(k))`
and `vec(r) = 2 hat(i) - hat(j) + hat(k) + mu (hat(i) + hat(j) + 5 hat(k))`.

Answer

To find the shortest distance between the two given lines, we can use the formula for the distance \( D \) between two skew lines represented in vector form. The formula is:

\[ D = \frac{|(\vec{C} - \vec{A}) \cdot (\vec{B} \times \vec{D})|}{|\vec{B} \times \vec{D}|} \]

where:
- \( \vec{A} \) and \( \vec{B} \) are points and direction vectors of the first line,
- \( \vec{C} \) and \( \vec{D} \) are points and direction vectors of the second line.

### Step 1: Identify the points and direction vectors

From the given lines:

1. Line 1:
\[ \vec{r_1} = \hat{i} + 2\hat{j} - 3\hat{k} + \lambda(3\hat{i} - 4\hat{j} - \hat{k}) \]
Here, \( \vec{A} = \hat{i} + 2\hat{j} - 3\hat{k} \) and \( \vec{B} = 3\hat{i} - 4\hat{j} - \hat{k} \).

2. Line 2:
\[ \vec{r_2} = 2\hat{i} - \hat{j} + \hat{k} + \mu(\hat{i} + \hat{j} + 5\hat{k}) \]
Here, \( \vec{C} = 2\hat{i} - \hat{j} + \hat{k} \) and \( \vec{D} = \hat{i} + \hat{j} + 5\hat{k} \).

### Step 2: Calculate \( \vec{C} - \vec{A} \)

\[ \vec{C} - \vec{A} = (2\hat{i} - \hat{j} + \hat{k}) - (\hat{i} + 2\hat{j} - 3\hat{k}) \]
\[ = (2 - 1)\hat{i} + (-1 - 2)\hat{j} + (1 + 3)\hat{k} \]
\[ = \hat{i} - 3\hat{j} + 4\hat{k} \]

### Step 3: Calculate \( \vec{B} \times \vec{D} \)

To find \( \vec{B} \times \vec{D} \):
\[ \vec{B} = 3\hat{i} - 4\hat{j} - \hat{k}, \quad \vec{D} = \hat{i} + \hat{j} + 5\hat{k} \]

Using the determinant to calculate the cross product:
\[ \vec{B} \times \vec{D} = \begin{vmatrix} \hat{i} & \hat{j} & \hat{k} \\ 3 & -4 & -1 \\ 1 & 1 & 5 \end{vmatrix} \]

Calculating the determinant:
\[ = \hat{i} \begin{vmatrix} -4 & -1 \\ 1 & 5 \end{vmatrix} - \hat{j} \begin{vmatrix} 3 & -1 \\ 1 & 5 \end{vmatrix} + \hat{k} \begin{vmatrix} 3 & -4 \\ 1 & 1 \end{vmatrix} \]
\[ = \hat{i}((-4)(5) - (-1)(1)) - \hat{j}((3)(5) - (-1)(1)) + \hat{k}((3)(1) - (-4)(1)) \]
\[ = \hat{i}(-20 + 1) - \hat{j}(15 + 1) + \hat{k}(3 + 4) \]
\[ = -19\hat{i} - 16\hat{j} + 7\hat{k} \]

### Step 4: Calculate the magnitude \( |\vec{B} \times \vec{D}| \)

\[ |\vec{B} \times \vec{D}| = \sqrt{(-19)^2 + (-16)^2 + 7^2} \]
\[ = \sqrt{361 + 256 + 49} = \sqrt{666} \]

### Step 5: Calculate the dot product \( (\vec{C} - \vec{A}) \cdot (\vec{B} \times \vec{D}) \)

\[ (\vec{C} - \vec{A}) \cdot (\vec{B} \times \vec{D}) = (\hat{i} - 3\hat{j} + 4\hat{k}) \cdot (-19\hat{i} - 16\hat{j} + 7\hat{k}) \]
\[ = (1)(-19) + (-3)(-16) + (4)(7) \]
\[ = -19 + 48 + 28 = 57 \]

### Step 6: Calculate the shortest distance \( D \)

\[ D = \frac{|(\vec{C} - \vec{A}) \cdot (\vec{B} \times \vec{D})|}{|\vec{B} \times \vec{D}|} \]
\[ = \frac{|57|}{\sqrt{666}} = \frac{57}{\sqrt{666}} \]

Thus, the shortest distance between the two lines is:

\[ D = \frac{57}{\sqrt{666}} \]