When pentavalent Impurity is added then four electrons from impurity forms covalent bond with four electrons from pure semiconductor material by sharing electrons to form octet to gain stability.

And hole is formed in bonding because of absence of electrons as in P type semiconductor.  

Extra electron from pentavalent impurity is free to move within the valence band itself. Even though electron goes to conduction band from valence band after gaining band gap energy , because this free electron is not involved in any bonding so that absence of this free electron does not change anything in valence band. So absence of this electron does not form hole in its original place in valence band.