Be sure not to miss the section from Le Puy to Conques, the old town of Cahors, the town of Moissac (abbey and river walk), the the Basque section from Navvarenx through to SJPP (route agnostic). There are missable sections from Figeac to Moissac, as one crosses the Karst limestone section. Later on there is the Big Agriculture section as someone else has mentioned.
It's not that these sections are difficult, or ugly, or unsafe, as are some other route sections in other places. It's just that they weren't much worth the effort. Dull, no views, very long stages, poorly marked route (or intentionally mis-marked, thanks to local teen-agers, the year I passed through), limited availability of grocery resupply (either because stores were distant from the route, or inopportune closing days).
(I realize that we each have different travel experiences, and many thoroughly enjoyed the entire Le Puy route. Plus, there are the other aspects of a pilgrimage walk, such as pilgrim friendships, meditations and prayer retreats. But the OP did ask the question, and I have tried to respond.)