I think Illiers-Combray is more on the cycle route, but it's totally doable, just adds some kms, and probably more road-walking. At the far side of Ermenonville, just follow the chemin à velo markings.
ETA The recommended footpath goes to Luplante and La Bourdiniere St. Loup, but its only...