Addressing a rally here, Modi, in an apparent dig at UPA chairperson Sonia Gandhi, said 125 crore people of the country were the "high-command" of his government, which was "not remote-controlled by a madam".
Hitting out at the Congress on Babbar's remarks, he said, "When one doesn't have issues to talk about, he resorts to abusing somebody else's mother."
The Congress leader had kicked up a row in Indore Thursday when he compared the falling value of rupee against US dollar with the age of the prime minister's mother.
Attacking Gandhi, Modi alleged that "coffers of banks were emptied for the rich during the madam's government". "However, our government opened doors of banks for the needy youth," Modi said.
The prime minister asked why the Congress was worried about Madhya Pradesh Chief Minister Shivraj Singh Chouhan being called "mama".
"They are worried Shivraj is called mama...Why, don't you remember (Ottavio) Quattrocchi mama and Warren Anderson (then chairman of the Union Carbide) mama," he said, referring to the Bofors scandal and the 1984 Bhopal Gas tragedy.
He said the Congress was voted out of power in Madhya Pradesh 15 years ago because it indulged in the politics of divisiveness.