Meanwhile, the Centre has ordered a probe into the matter.
"You can wear anything, whether it is yellow, red or green. There is no restrictions with regard to clothing. There is no guideline from our side. We have set up an inquiry into the matter," said Tourism and Culture Minister Mahesh Sharma, reported PTI.
Sharma asserted that there are no restrictions with regard to the clothing for visiting Taj Mahal, the world famous 17th-century monument.
However, minister said that a tourist guide or a policeman could have asked them to remove the saffron scarves, the truth will emerge after the inquiry.
When 34 models representing their country recently visited Taj Mahal, some of them covered their head with saffron scarves to protect themselves from the heat.
However, when they were about to enter the monument, they were reportedly asked to remove the scarves.
To participate in the Supermodel International Contest, 34 models representing their country have been on a 11-day visit to India, starting April 12.
The incident provoked protest from some Hindu outfits including the Bharatiya Janata Yuva Morcha (BJYM) which said it hurt their religious sentiment.
About Taj Mahal
The Taj Mahal is an ivory-white marble mausoleum on the south bank of the Yamuna river in the city of Agra, UP. It was commissioned in 1632 by the Mughal emperor, Shah Jahan, to house the tomb of his favourite wife, Mumtaz Mahal.
The tomb is the centrepiece of a 17-hectare (42-acre) complex.
It includes a mosque and a guest house, and is set in formal gardens bounded on three sides by a crenellated wall.