For a gas sample with $${N_0}$$ number of molecules, function $$N\left( V \right)$$  is given by:
$$N\left( V \right) = \frac{{dN}}{{dV}} = \left[ {\frac{{3{N_0}}}{{V_0^3}}} \right]{V^2}$$      for $$0 \leqslant V \leqslant {V_0}$$   and $$N\left( V \right) = 0$$   for
$$V > {V_0}$$   where $$dN$$  is number of molecules in speed range $$V$$ to $$V + dV.$$   The $$rms$$  speed of the gas molecule is -

Solution :
$$\eqalign{ & V_{rms}^2 = < {V^2} > = \frac{{V_1^2 + V_2^2 + V_3^2 + .......}}{N} \cr & = \frac{{\int {{V^2}} dN}}{{\int d N}}\,\,{\text{here}}\,\,\frac{{dN}}{{dV}} = N\left( V \right) \cr & V_{rms}^2 = \frac{1}{N}\int\limits_0^\infty N \left( V \right){V^2}dV \cr & = \frac{1}{N}\int\limits_0^{{V_0}} {\left[ {\frac{{3N}}{{V_0^3}}{V^2}} \right]} {V^2}dV = \frac{3}{5}V_0^2 \cr & \Rightarrow {V_{rms}} = \sqrt {\frac{3}{5}} {V_0} \cr} $$