The men -- aged 17, 18 and 26 -- were detained after a series of pre-dawn raids in the states of Schleswig-Holstein and Lower Saxony, BBC reported.
The police have seized "extensive material", the prosecutors added.
But no concrete missions or orders have so far been found, BBC reported.
In a statement, the Federal Public Prosecutor's office said investigations so far suggested the three had come to Germany in November 2015 with the intention of "carrying out a previously determined order (from IS) or to await further instructions".
They were said to have travelled through Turkey and Greece on false passports.
