The Dow Jones Industrial Average on Tuesday fell 58.63 points, or 0.32 per cent, at 18,444.36, Xinhua news agency reported.
The S&P 500 lost 4.51 points, or 0. 21 per cent, to 2,175.87. The Nasdaq Composite Index was down 12.12 points, or 0.23 per cent, to 5,220.21.
Fed Vice Chairman Stanley Fischer said early on Tuesday that it is impossible to say whether the next interest rate hike would be "one and done," according to media reports.
Fischer's comments were in line with remarks he and Fed Chairperson Janet Yellen made on Friday at a conference in Jackson Hole, Wyoming, indicating that the time to lift rates again was nearing.
"In light of the continued solid performance of the labor market and our outlook for economic activity and inflation, I believe the case for an increase in the federal funds rate has strengthened in recent months," Yellen said Friday.
Analysts said it is possible for the Fed to hike interest rates as soon as September. But about 71 per cent of 62 economists surveyed by the Wall Street Journal this month believed that the Fed will wait until December to raise rates.
On the economic front, the Conference Board Consumer Confidence Index came in at 101.1 in August, up from 96.7 in July.
