In a series of tweets, he said, "George Sahab" represented the best of India's political leadership.
"Frank and fearless, forthright and farsighted, he made a valuable contribution to our country. He was among the most effective voices for the rights of the poor and marginalised," he said on Twitter.
The 88-year-old former defence minister died here after suffering from a prolonged illness.
READ: All about the former Defence Minister who led Pokharan-II, Kargil war
Modi said, "When we think of Fernandes, we remember most notably the fiery trade union leader who fought for justice, the leader who could humble the mightiest of politicians at the hustings".
"A visionary railway minister and a great defence minister who made India safe and strong. During his long years in public life, George Sahab never deviated from his political ideology. He resisted the Emergency tooth and nail. His simplicity and humility were noteworthy," the PM said.
Defence Minister Nirmala Sitharaman had visited the All India Institute of Medical Sciences (AIIMS) in New Delhi to pay tributes to Fernandes.