\[\begin{array}{l} A = \left[ {\begin{array}{*{20}{c}} 5&6&1\\ 2&{ - 1}&5 \end{array}} \right]\,\,{\rm{and}}\,{\rm{let}}\,\,B = \left[ {\begin{array}{*{20}{c}} 5\\ 1\\ 4 \end{array}\,\,\,\,\,\begin{array}{*{20}{c}} 2\\ 6\\ 3 \end{array}} \right]\\ \therefore \,AB = \left[ {\begin{array}{*{20}{c}} 5&6&1\\ 2&{ - 1}&5 \end{array}} \right]\,\left[ {\begin{array}{*{20}{c}} 5\\ 1\\ 4 \end{array}\,\,\,\,\,\begin{array}{*{20}{c}} 2\\ 6\\ 3 \end{array}} \right]\\ = \,\left[ {\begin{array}{*{20}{c}} {25 + 6 + 4}&{10 + 36 + 3}\\ {10 - 1 + 20}&{4 - 6 + 15} \end{array}} \right]\\ = \,\left[ {\begin{array}{*{20}{c}} {35}&{49}\\ {29}&{13} \end{array}} \right] \end{array}\]

As the determinant is of the third degree in $$a, b, c,$$  we get
\[\left| {\begin{array}{*{20}{c}} { - 2a}&{a + b}&{a + c} \\ {b + a}&{ - 2b}&{b + c} \\ {c + a}&{c + b}&{ - 2c} \end{array}} \right| = \lambda \left( {b + c} \right)\left( {c + a} \right)\left( {a + b} \right),\]
where $${\lambda}$$ is independent of $$a, b, c.$$
Putting $$a = 0, b = 1, c = 2,$$
\[\left| {\begin{array}{*{20}{c}} 0&1&2 \\ 1&{ - 2}&3 \\ 2&3&{ - 4} \end{array}} \right| = \lambda \cdot 3 \cdot 2 \cdot 1.\,\,\,\,\,{\text{Now find }}\lambda .\]

Use $${C_3} \to {C_3} - {C_2},{C_2} \to {C_2} - {C_1}.$$

\[\vartriangle = \left| {\begin{array}{*{20}{c}} 5&4&3 \\ {100x + 50 + 1}&{100y + 40 + 1}&{100z + 30 + 1} \\ x&y&z \end{array}} \right| = \left| {\begin{array}{*{20}{c}} 5&4&3 \\ 1&1&1 \\ x&y&z \end{array}} \right|\]
$$\eqalign{ & \left( {{R_2} \to {R_2} - 100{R_3} - 10{R_1}} \right) \cr & = x - 2y + z = 0\,\,\left( {\because \,\,x,y,z\,\,{\text{are in A}}{\text{.P}}{\text{.}}} \right) \cr} $$

\[\begin{array}{l} \left| \begin{array}{l} a + x\,\,\,\,\,a\,\,\,\,\,\,\,\,\,x\\ a - x\,\,\,\,\,a\,\,\,\,\,\,\,\,\,x\\ a - x\,\,\,\,\,a\,\,\,\, - x \end{array} \right| = 0\\ \Rightarrow \left( {a + x} \right)\left( \begin{array}{l} a\,\,\,\,\,\,\,\,\,x\\ a\,\,\,\, - x \end{array} \right) - a\left( \begin{array}{l} a - x\,\,\,\,\,\,\,\,\,x\\ a - x\,\,\,\, - x \end{array} \right) + x\left( \begin{array}{l} a - x\,\,\,\,\,a\\ a - x\,\,\,\,\,a \end{array} \right) = 0\\ \Rightarrow \left( {a + x} \right)\left( { - 2ax} \right) - a\left( { - 2x\left( {a - x} \right)} \right) + x.0 = 0\\ \Rightarrow - 4a{x^2} = 0\\ \Rightarrow x = 0 \end{array}\]

\[{\vartriangle _2} = \frac{1}{{abc}}\left| {\begin{array}{*{20}{c}} a&{abc}&{{a^2}} \\ b&{cab}&{{b^2}} \\ c&{abc}&{{c^2}} \end{array}} \right|\]
\[{\vartriangle _2} = \left| {\begin{array}{*{20}{c}} a&1&{{a^2}} \\ b&1&{{b^2}} \\ c&1&{{c^2}} \end{array}} \right| = - \left| {\begin{array}{*{20}{c}} 1&a&{{a^2}} \\ 1&b&{{b^2}} \\ 1&c&{{c^2}} \end{array}} \right| = - {\vartriangle _1}.\]

\[A = \left[ \begin{array}{l} \,\,0\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\, - 1\\ \,\,0\,\,\,\,\,\, - 1\,\,\,\,\,\,\,\,\,\,0\\ - 1\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,0 \end{array} \right]\]
$${\text{clearly }}A \ne 0.\,{\text{Also }}\left| A \right| = - 1 \ne 0$$
\[\therefore \,\,{A^{ - 1}}\,{\rm{exists,}}\,\,{\rm{further }}\left( { - 1} \right)I = \left[ \begin{array}{l} - 1\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,0\\ \,\,\,0\,\,\,\,\, - 1\,\,\,\,\,\,\,\,\,0\\ \,\,\,0\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\, - 1 \end{array} \right] \ne A\]
\[{\rm{Also }}\,\,{A^2} = \left[ \begin{array}{l} \,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\, - 1\\ \,\,0\,\,\,\,\,\,\, - 1\,\,\,\,\,\,\,\,\,\,0\\ - 1\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,0 \end{array} \right]\left[ \begin{array}{l} \,\,0\,\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\, - 1\\ \,\,0\,\,\,\,\,\,\, - 1\,\,\,\,\,\,\,\,\,\,0\\ - 1\,\,\,\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,0 \end{array} \right]\]
\[\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, = \left[ \begin{array}{l} \,1\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,0\\ 0\,\,\,\,\,\,\,\,\,1\,\,\,\,\,\,\,\,0\\ 0\,\,\,\,\,\,\,\,0\,\,\,\,\,\,\,\,\,1 \end{array} \right] = I\]

$$\eqalign{ & \because {B^n} - A = I \cr & \therefore {B^n} = I + A \cr} $$
\[\begin{array}{l} {B^n} = \left[ {\begin{array}{*{20}{c}} 1&0&0\\ 0&1&0\\ 0&0&1 \end{array}} \right] + \left[ {\begin{array}{*{20}{c}} {26}&{26}&{18}\\ {25}&{37}&{17}\\ {52}&{39}&{50} \end{array}} \right]\\ {B^n} = \left[ {\begin{array}{*{20}{c}} {27}&{26}&{18}\\ {25}&{38}&{17}\\ {52}&{39}&{51} \end{array}} \right]\\ {\rm{or, }}{\left[ {\begin{array}{*{20}{c}} 1&4&2\\ 3&5&1\\ 7&1&6 \end{array}} \right]^n} = \left[ {\begin{array}{*{20}{c}} {27}&{26}&{18}\\ {25}&{38}&{17}\\ {52}&{39}&{51} \end{array}} \right]\,\,\,\,\,.....\left( {\rm{i}} \right) \end{array}\]
$$\therefore n \ne 1$$
Now put $$n = 2,$$  then
\[\begin{array}{l} {B^2} = {\left[ {\begin{array}{*{20}{c}} 1&4&2\\ 3&5&1\\ 7&1&6 \end{array}} \right]^2} = \left[ {\begin{array}{*{20}{c}} 1&4&2\\ 3&5&1\\ 7&1&6 \end{array}} \right]\left[ {\begin{array}{*{20}{c}} 1&4&2\\ 3&5&1\\ 7&1&6 \end{array}} \right]\\ = \left[ {\begin{array}{*{20}{c}} {27}&{26}&{18}\\ {25}&{38}&{17}\\ {52}&{39}&{51} \end{array}} \right] \end{array}\]
Which is equal to R.H.S. of eq. (i).
∴ $$n = 2$$

We have $${P^T} = 2P + I$$
$$\eqalign{ & \Rightarrow \,\,P = 2{P^T} + I \cr & \Rightarrow \,\,P = 2\left( {2P + I} \right) + I \cr & \Rightarrow \,\,P = 4P + 3I \cr & \Rightarrow \,\,P + I = 0 \cr & \Rightarrow \,\,PX + X = 0 \cr & \Rightarrow \,\,PX = - X \cr} $$

Use $${R_1} \to {R_1} - {R_2},{R_2} \to {R_2} - {R_3},{R_3} \to {R_3} - {R_4}.$$         Then
$$\left( {x - \lambda } \right)\left( {x - \mu } \right)\left( {x - \nu } \right) = 0.$$