Its Secretary General Bheki Ntshalintshali said he is no longer the “right person” to lead the country.
Mr Zuma has been under growing pressure following a major cabinet reshuffle which included the sacking of trusted Finance Minister Pravin Gordhan.
That led to South Africa’s credit rating being cut to junk status putting more pressure on a troubled economy.
The Congress of South African Trade Unions (Cosatu), a key part of the governing alliance, says it has 1.8 million members.
It forms part of what is called the Tripartite Alliance along with the the African National Congress (ANC) and the South African Communist Party (SACP).
The SACP has also called on Mr Zuma to go.
Cosatu General Secretary of Bheki Ntshalintshali told a media briefing that the union’s decision was driven by Mr Zuma’s failure to consult it before making changes to his cabinet.
He termed the president’s leadership as “inattentive, negligent… and disruptive”.
He added that the organisation was not concerned about Mr Gordhan’s sacking because he was, as his predecessors, “not a friend of the workers”.
“We will support the new minister where necessary and fight with him where necessary,” he added.
Mr Ntshalintshali also criticised ratings agency S&P’s decision to downgrade South Africa to junk status saying the union views it as political interference.
‘A crushing blow for Zuma’ by Milton Nkosi, BBC Africa
President Jacob Zuma is certainly politically weaker today than he was over the weekend. Cosatu’s call for him to step down is a crushing blow to the beleaguered head of state.
The workers’ union federation had been a reliable backer of Mr Zuma against unrelenting calls for his removal.
Focus now shifts to the National Working Committee (NWC) of the ANC which is currently meeting.
However, regardless of the outcome of that meeting, deep divisions within the body will entrench positions between the pro and anti-Zuma factions.
Mr Zuma is due to step down in 2019 at the end of his second five-year term as president.
Last week, Deputy President Cyril Ramaphosa called the sacking of Mr Gordhan “totally unacceptable”.
Later, in a speech at the weekend, which has been interpreted as a public broadside against Mr Zuma, he called for a renewal of the country and criticised “greedy and corrupt people”.
But President Zuma’s obituary has been written many times before only for him to rise from the ashes, says the BBC’s Milton Nkosi in Johannesburg..
Despite the reported scandals that has dogged his administration, several attempts to remove Mr Zuma have floundered.
Junk status
Meanwhile, new Finance Minsiter Malusi Gigaba has been working to reassure South Africans about the state of the economy.
Mr Gigaba told a media briefing that Monday’s downgrade to junk status by ratings agency S&P was a setback, but said that people should not be despondent.
“I’m not saying it’s easy to get out of a rating downgrade, yet I remain confident,” he added.
He said he would lead a meeting with ratings agencies Fitch and Moody’s.
.BBC