He was administered the oath of office by Jammu and Kashmir Chief Justice Gita Mittal at a ceremony held at Raj Bhawan at the foothills of Zeberwan range. His warrant of appointment was read out by Chief Secretary B V R Subrahmanyam.
Murmu, who will turn 60 next month, is a a 1985 batch IAS officer from Gujarat.
BJP leader Jugal Kishore and Rajya Sabha MP and PDP member Nazir Laway were among the over 250 guests present the function.
Also read: RK Mathur Sworn In As First Lt Governor Of Ladakh, Umang Narula To Be His Advisor
Murmu was principal secretary to Narendra Modi during his tenure as Chief Minister of Gujarat.
He is one of the youngest governors in the country.
Meanwhile, the President's rule imposed in undivided Jammu and Kashmir was on Thursday revoked following bifurcation of the state into two Union Territories.
The Home Ministry in a notification on Wednesday replaced the state of J-K with the "union territory of Jammu and Kashmir" and announced omission of "permanent residents or hereditary state subjects".
In the late night notification, the ministry's Jammu and Kashmir division announced a slew of measures, including application of central laws to the state.
Read: Union Territories Jammu & Kashmir, Ladakh Come Into Existence; J&K To Be UT with Legislature