As per sources, the incident comes before he was supposed to be produced before court to give statement in Rangdari case.
The sources also tell that the gun was fired by another prisoner Sunil Rathi.
The 51-year-old criminal was shot ten times and then his body was dumped into a gutter inside the prison premises.
Considering the major loophole in security arrangements, the jailer and Deputy Jailer were suspended soon after the incident.
Speaking on the incident, Chief Minister Yogi Adityanath said "Have ordered a judicial inquiry and suspension of the jailor. Such an incident occurring inside jail premises is a serious matter. Will conduct an in-depth investigation and strict action".
ALSO READ: Who was Don Munna Bajrangi? Gangster shot dead inside UP jail
Bhandari or Prem Prakash was brought to Baghpat from Jhansi jail on late Sunday evening.
The incident comes after his wife had held a press conference and claimed a threat to his life; stating that plans are being chalked out to kill him an a 'fake encounter'.
Following the incident, the opposition is slamming the government, questioning the law and order, saying “Even inside jails, the law and order is not place”. The opposition is also questioning how did the weapon go inside the jail.