J=∫x2tan-1(x)dx.
Let dv=x2dx, then v=⅓x3; let u=tan-1(x), du=dx/(1+x2).
J=⅓x3tan-1(x)-⅓∫x3dx/(1+x2),
J=⅓x3tan-1(x)-⅓∫(x-x/(1+x2))dx,
J=⅓x3tan-1(x)-⅙x2+⅓∫x/(1+x2))dx,
J=⅓x3tan-1(x)-⅙x2+⅙∫2x/(1+x2))dx,
J=⅓x3tan-1(x)-⅙x2+⅙ln(1+x2)+C, where C is a constant.