(x^2-1)y"+3xy'+xy=0 power series solution
Let y = Sigma[n=0...infty] a_n.x^n, y' = Sigma[n=1...infty] n.a_n.x^(n-1), y'' = Sigma[n=2...infty] n(n-1).a_n.x^(n-2)
Substituting for y, y' and y'' into the DE,
Sigma[n=2...infty] n(n-1).a_n.x^n - Sigma[n=2...infty] n(n-1).a_n.x^(n-2) + Sigma[n=1...infty] 3n.a_n.x^n + Sigma[n=0...infty] a_n.x^(n+1) = 0
Modifying the indices such that all the Sigmas have x to the same power,
Sigma[n=2...infty] n(n-1).a_n.x^n - Sigma[k=0...infty] (k+2)(k+1).a_(k+2).x^k + Sigma[n=1...infty] 3n.a_n.x^n + Sigma[k=1...infty] a_(k-1).x^k = 0
expand those summations for k/n = 0 and 1,
Sigma[n=2...infty] {n(n-1)a_n - (n+2)(n+1)a_(n+2) + 3n.a_n + a_(n-1)}x^n - 2a_2 - 6a_3.x + 3a_1.x + a_0.x = 0
Sigma[n=2...infty] {(n^2-n+3n)a_n - (n+2)(n+1)a_(n+2) + a_(n-1)}x^n - 2a_2 + (a_0 + 3a_1 - 6a_3)x = 0
Equating the coefficients of x to zero,
a2 = 0, a_0 + 3a_1 - 6a_3 = 0, -> a_3 = (1/6)(a_0 + 3a_1)
(n+2)(n+1)a_(n+2) = n(n+2)a_n + a_(n-1), n = 2,3,4,...
n=2, 12a_4 = 8a_2 + a_1 = a_1 -> a_4 = (1/12)a_1
n=3, 20a_5 = 15a_3 + a_2 = 15a_3 -> a_5 = (3/4)a_3 = (1/8)(a_0 + 3a_1)
The general series solution is then:
y(x) = a_0 + a_1.x + a_2.x^2 + a_3.x^3 + a_4.x^4 + a_5.x^5 + O(x^6)
y(x) = a_0 + a_1.x + (1/6)(a_0 + 3a_1).x^3 + (1/12)a_1.x^4 + (1/8)(a_0 + 3a_1).x^5 + O(x^6)