Let r is the radius of the circle.

From the figure,

OP = OQ = OR = r

In triangle ABC,

From Pythagorus Theorem,

     AC² = AB² + BC²

=> AC² = 5² + (12)²

=> AC² = 25 + 144

=> AC² = 169

=> AC = √169

=> AC = 13

Now, area(Δ AOB) +area(Δ BOC) + area(Δ AOC) = area(Δ AOB)

Now, s = (5 + 12 + 13)/2 = 30/2 = 15

So, area(Δ AOB) +area(Δ BOC) + area(Δ AOC) = area(Δ AOB)

=> (OP*AB)/2 + (OP*AB)/2 + (OP*AB)/2 = √{s*(s-a)*(s-b)*(s-c)}

=> {(OP*AB) + (OP*AB) + (OP*AB)}/2 = √{15*(15-5)*(15-13)*(15-13)}

=> {5r + 12r + 13r}/2 = √{15*10*3*2}

=> 30r/2 = √900

=> 15r = 30

=> r = 30/15

=> r = 2

So, the radius of the circle is 2 cm.