Let the 3 numbers in GP is : a,ar, ar²

Given that

       a+ar+ar² = 56

=> a(1+r+r² ) = 56

=> a = 56/(1+r+r² ) ...............1

Again

a-1, ar-7,ar² - 21  is in AP

So 

     (ar – 7) – (a – 1) = (ar² – 21) – (ar – 7)

=> ar - 7 - a + 1 = ar² -21 - ar + 7

=> ar – a – 6 = ar² – ar – 14

=> ar² – 2ar + a = 8

=> ar² – ar – ar + a = 8

=> a(r² + 1 – 2r) = 8

=> a (r – 1)² = 8 ..............2

Now put value of a from equation 1 in equation 2, we get

     {56/(1+r+r² )}*(r – 1)² = 8

=> 56*(r – 1)² = 8 *(1+r+r² )

=> 7*(r -1)² = (1+r+r² )

=> 7(r² – 2r + 1) = 1 + r + r²

=> 7r² – 14 r + 7 – 1 – r – r² = 0

=> 6r² – 15r + 6 = 0

=> 6r² – 12r – 3r + 6 = 0

=> 6r (r – 2) – 3 (r – 2) = 0

=> (6r – 3) (r – 2) = 0

=> r = 3/6, 3

=> r = 1/2, 3

Put the value of r in equation we get

Now when r = 1/2 , a = 32

       when r = 2 , a = 8

Case1: when a = 32 and r =1/2 then series is:

          32, 16, 8

Case2: when a = 8 and r =2 then series is:

          8,16,32