If you stop at Orisson there is a good chance that you will meet many people who you meet many times over the course of your walk, it's a good place. They have a very nice terrace with great mountain views if the weather is good. The first 8km from St Jean are the steepest, which takes you to Orisson.
I've walked it twice this year. The first time in snow as the first leg of the
Camino Frances, 2 hours to Orisson, stayed the night, 5 hours to Roncesvalles.
The second in fog and rain just last week. I walked it for fun as I finished my walking in St Jean, and had a day to spare before travelling home. St Jean to Roncesvalles in around 6 hours. Including a coffee at Orisson and a snack in a refuge hut.
I was feeling fit after walking the Le Puy route, up for the challenge and raring to go.
The route is not as difficult as many people (including myself) worry about before doing it for the first time. That said I would stay at Orisson again. No need to leave St Jean early. Meet lots of new friends. An easier next day as you've already done 8km.