Nice's mayor, Christian Estrosi, who described the attack as terrorism, said on Twitter it had happened in or near the city's Notre Dame church.
Also Read|Twitter Deletes Ex-Malaysian PM's Controversial 'Muslims Have Right To Kill French’ Tweet
Estrosi said the attacker had repeatedly shouted the phrase "Allahu Akbar", or God is greatest, even after he had been detained by police.
One of the people killed inside the church was believed to be the churchwarden, Estrosi said, adding that a woman had tried to escape from inside the church and had fled into a bar opposite the building.
"The suspected knife attacker was shot by police while being detained, he is on his way to the hospital, he is alive," Estrosi told reporters.
'The suspect carried the Holy Quran in his hands'
According to reports by French news agency AFP, the man suspected of the attack is a 21-year-old Tunisian who arrived in Europe a few weeks ago. He has been named as Brahim Aouissaoui by French and Tunisian law enforcement sources.
News agency AP has claimed that the suspect was carrying a copy of the Holy Quran when he attacked worshippers in a French church.
The police who shot Aouissaoui had "without any doubt prevented an even higher toll," said chief prosecutor Jean-Francois Ricard, adding investigators had found two unused knives in a bag at the scene, as quoted by Reuters.
The suspect landed in late September on the Italian island of Lampedusa, where he was placed in virus quarantine by authorities before being released with an order to leave Italian territory.
France’s chief anti-terrorism prosecutor Jean-Francois Ricard said that the attacker arrived in Nice by train at 6.47am (05:57 GMT) and spent nearly half an hour in the station before he headed to the church just 400 metres away.
Also Read|WATCH: Protest Against Emmanuel Macron In Mumbai, Bhendi Bazar Road Plastered With French President’s Pictures
The Victims
A 60-year-old woman died inside the church and the body of a man, a 55-year-old church employee, was found nearby, his throat also slit.
Another woman, a 44-year-old Brazilian who had fled the church to a nearby restaurant, died shortly afterward from multiple knife wounds.
"Tell my children I love them," she managed to say before her death, according to French cable channel BFM TV.
Previous such attacks
The attack in the Mediterranean city of Nice was the third in less than two months that French authorities have attributed to Muslim extremists, including the beheading of a teacher who had shown caricatures of the Prophet Muhammad in class after the images were re-published by a satirical newspaper targeted in a 2015 attack.
France on high alert
France has been on high alert since the January 2015 massacre at the satirical weekly magazine Charlie Hebdo marked the beginning of a wave of jihadist attacks that have killed more than 250 people.
Tensions have heightened since last month when the trial opened for 14 suspected accomplices in that attack.
The paper marked the start of the court proceedings by republishing cartoons of the Prophet Mohammed that infuriated millions of Muslims worldwide -- the same caricatures that teacher Samuel Paty used as lesson material.
President Emmanuel Macron visited Nice after the attack calling it an “Islamist terrorist attack”.
(With inputs from agencies)