A couple of books you might want to read - The Great Donkey Walk:
From Spain to Greece by Pilgrim Way and Tracks by
Susan Chitty and Thomas Hinde (3 kids, 2 parents, 2 donkeys, mayhem and a wonderful education/adventure)
The Seven Year Hitch: a family odyssey by David R Grant ( 5 kids, mum went home, they went round the world despite civil war, crime, disease - turned out well-adjusted competent adults...)
Travel at any age broadens your mind and horizons - as a kid it gives you skills to better face life with. I'm sure there will be days you regret it and days you wish you could go on forever, but i am sure you will all benefit from the experience.
And on albergue noise - I don't think toddlers have anything on olympic-snoring, plastic-bag scrunching, light on/off/on/off, door-slamming adults, who leave at 4am!!!
Just make sure you let them out of their strollers for a good run each day and never run out of portable food..... Good luck!