Let `|veca|=1, |vecb|=1 and |veca+vecb|=sqrt(3)`. If `vec c` be a vector such that `vec c=veca+2vecb-3(veca xx vecb)` and `p=|(veca xx vecb) xx vec c|`, then find `[p^(2)]`. (where [ ] represents greatest integer function).

Answer

To solve the problem step by step, we will analyze the given information and apply vector algebra principles.

### Step 1: Understand the given conditions
We are given:
- \(|\vec{a}| = 1\)
- \(|\vec{b}| = 1\)
- \(|\vec{a} + \vec{b}| = \sqrt{3}\)

### Step 2: Use the property of magnitudes
From the property of magnitudes, we can square the third condition:
\[ |\vec{a} + \vec{b}|^2 = (\vec{a} + \vec{b}) \cdot (\vec{a} + \vec{b}) = 3 \]
Expanding the dot product:
\[ |\vec{a}|^2 + |\vec{b}|^2 + 2(\vec{a} \cdot \vec{b}) = 3 \]
Substituting the known magnitudes:
\[ 1^2 + 1^2 + 2(\vec{a} \cdot \vec{b}) = 3 \]
\[ 2 + 2(\vec{a} \cdot \vec{b}) = 3 \]
This simplifies to:
\[ 2(\vec{a} \cdot \vec{b}) = 1 \implies \vec{a} \cdot \vec{b} = \frac{1}{2} \]

**Hint:** Use the property of magnitudes and the dot product to relate the vectors.

### Step 3: Define vector \(\vec{c}\)
We are given:
\[ \vec{c} = \vec{a} + 2\vec{b} - 3(\vec{a} \times \vec{b}) \]

### Step 4: Calculate \(\vec{a} \times \vec{b}\)
The magnitude of \(\vec{a} \times \vec{b}\) can be calculated using:
\[ |\vec{a} \times \vec{b}| = |\vec{a}||\vec{b}|\sin \theta \]
Where \(\theta\) is the angle between \(\vec{a}\) and \(\vec{b}\). Since \(\vec{a} \cdot \vec{b} = |\vec{a}||\vec{b}|\cos \theta\):
\[ \frac{1}{2} = 1 \cdot 1 \cdot \cos \theta \implies \cos \theta = \frac{1}{2} \implies \theta = 60^\circ \]
Thus:
\[ |\vec{a} \times \vec{b}| = 1 \cdot 1 \cdot \sin 60^\circ = \frac{\sqrt{3}}{2} \]

### Step 5: Calculate \(\vec{c}\)
Now substituting back into \(\vec{c}\):
\[ \vec{c} = \vec{a} + 2\vec{b} - 3\left(\frac{\sqrt{3}}{2}\right) \hat{n} \]
Where \(\hat{n}\) is the unit vector in the direction of \(\vec{a} \times \vec{b}\).

### Step 6: Calculate \(p = |(\vec{a} \times \vec{b}) \times \vec{c}|\)
Using the vector triple product identity:
\[ \vec{u} \times (\vec{v} \times \vec{w}) = (\vec{u} \cdot \vec{w})\vec{v} - (\vec{u} \cdot \vec{v})\vec{w} \]
Let \(\vec{u} = \vec{a} \times \vec{b}\), \(\vec{v} = \vec{c}\). We need to compute:
\[ p = |(\vec{a} \times \vec{b}) \times \vec{c}| \]

### Step 7: Calculate \(p^2\)
We will compute:
\[ p^2 = |(\vec{a} \times \vec{b})|^2 |\vec{c}|^2 - (\vec{a} \times \vec{b} \cdot \vec{c})^2 \]
Substituting known values, we can find \(p^2\).

### Step 8: Final calculation
After performing the calculations, we find:
\[ p^2 = \frac{21}{4} \]
The greatest integer function \([p^2]\) is:
\[ \lfloor \frac{21}{4} \rfloor = 5 \]

### Final Answer
Thus, the answer is:
\[ \boxed{5} \]