I can only give you my own experience. Three years ago I walked the same time you are walking, from Sept through Oct/Nov. I took a compressible but warm polyfill jacket and only wore it twice.
What I did end up purchasing in Burgos was two cotton longsleeved women's undershirts for cold nights and mornings.
This time, for cold/rainy weather I'm taking two pair of cuddle duds long underwear, a silk/cashmere lightweight pullover to layer over that, a fleece shirt to go over THAT, and a lightweight goretex rainjacket/windjacket. This way, I can take things off as I warm up.
The only place it was really cold for me was in Galicia and when it was raining and the wind was blowing, so I also am taking lightweight rain pants. I took a poncho last time and dumped it and purchased a rainsuit along the way. I got sick after getting chilled and being exposed to the flu in a crowded refugio, so this time I'm going prepared to stay warm.
Also, regarding the pack, I have an Arcteryx 30 and I love it. Just the right size. It weighs 14 pounds loaded and I'll dump some things along the way. Better to take less than more in my opinion. The FREE BOXES in the refugios are full of stuff people brought and decided they did not need. You can raid those boxes or buy what you need in Spain.
Buen Camino!