The decision to "recall" him "urgently" followed marathon talks of the ANC's top leadership body, the BBC reported.
Zuma, 75, had agreed to step down if he were granted a transition period of three to six months, ANC Secretary General Ace Magashule said at a press conference in Johannesburg on Tuesday.
But the ANC rejected Zuma's proposal and the President's current stance is unclear.
"South Africa's going through a period of uncertainty and anxiety as a result of the unresolved matter of transition," Magashule said, adding that the decision was a difficult one "taken after exhaustive discussions".
Magashule did not give a deadline or transition period for Zuma to resign. The President's term had been due to end next year.
Zuma has served as South Africa's President since 2009, but his tenure in office has been beleaguered by corruption allegations and a growing rift with ANC party members. He has resisted increasing pressure to quit since December, when Cyril Ramaphosa replaced him as leader of the ANC.
The party's National Executive Committee (NEC) decided the removal should be "treated with urgency", said Magashule.
"It is obvious we want Comrade Ramaphosa to come in as the President of South Africa," he added.
It will be very difficult for Zuma to resist a formal request to step down but he would not be legally obliged to do so and could technically carry on as President despite losing the faith of his party, the BBC said.
However, he would then be expected to face a confidence vote in Parliament. The vote has been scheduled for February 22, but it could be held earlier.
Despite Magashule saying that "the decision is now final", he revealed that Zuma told him that he "did not believe the NEC has the right" to make such a ruling.
Magashule said Zuma was expected to respond to the NEC's decision on Wednesday, although they had given him no deadline.
(This story has not been edited by Social News XYZ staff and is auto-generated from a syndicated feed.)
