The deadly attack took place around 3.30 p.m. when the ministry employees were coming out after a day's work, media reports said.
Xinhua news agency reported that several people were killed and injured. The Afghan media reported six deaths.
Ambulances raced the injured from near the gate of the defence ministry to hospitals, witnesses said.
The first explosion was reportedly small. When people gathered to find out what was happening, a second but powerful blast took place, according to a tweet by Afghan journalist Bilal Sarwary.
The Defence Ministry is located next to the Finance Ministry, the Presidential Palace as well as other key government buildings in the heart of Kabul.
About two weeks ago, 13 people were killed in a suicide attack on the American University in Kabul.
