The 24-year-old is yet to be named officially but has been identified locally as Mohammed Nadeemuddin from Hyderabad.
UK media reports state the gruesome incident took place at about 12:30pm local time.
Thames Valley Police said he was rushed to the hospital from the car park of Tesco supermarket in Wellington Street in Slough, on the western suburbs of London, but died from his stab wounds on Wednesday.
"The victim's kin have now been made aware and are being supported by specially trained officers, Detective Superintendent Ian Hunter," Head of the Thames Valley Police Major Crime Unit, said.
"I understand that this would have caused concern in the community, but I would like to offer some reassurance to the public that we do not believe this incident to be a random attack, and that the people involved in the incident are believed to be known to one another, he said.
The police said its officers were still in the early stages of the investigation and are working hard to establish the full circumstances and establish witnesses.
"We are still continuing in our attempts to locate and bring the offender or offenders to justice. There will be a continued police presence in the area conducting reassurance patrols," Hunter said.
He appealed for anyone with information relating to the incident to come forward and speak to police.
The police also urged the public not to share images of the incident on social media, but instead to report these to the police.
The scene watch which was in place at the scene of the attack has also now been lifted, and the covered car park at Tesco has been reopened.
Tesco in a statement said it was assisting police with their investigation into the "tragic incident".
"We are deeply shocked and saddened by the fatal stabbing in our Slough Extra car park today and our thoughts are with the victim's family at this very difficult time," it added.
A 21-year-old witness told BBC that he was in the car park when he "heard shouting" and saw "lots of blood on the floor".
"I got closer and saw a man on the floor," he added.
According to media reports from India, the victim's India-based family members are looking into travelling to the UK and have sought the help of External Affairs Minister Sushma Swaraj.
(With additional information from PTI)