The man allegedly entered the Aap Shambu temple in Roopnagar area of Janipur this evening and broke some items and the windowpanes, a police officer said.
Following the incident, people in large numbers came onto the roads and held a protest demanding the immediate arrest of the accused, he said.
The protestors also resorted to stone-pelting and set three vehicles on fire in the area.
"A case has been registered and the accused has been arrested. He is mentally disturbed and had come to the area to see a psychiatrist," the officer said.
He said the police was investigating the entire incident.