I don't know the solution but I think the whole question comes back to the old gnarly debate of 'Who is a real Pilgrim' and should those who are not 'Real Pilgrims' always stay in the albergues. Many medieval pilgrims walked to Santiago more than once and just like us, some decided not to stay in the dedicated pilgrim hospices and opted for inns and monasteries instead.
Many of us who keep going back love the Camino for itself, and for the journey - not for the destination. That is why so many people go back time and again and walk the many different routes - for the sheer enjoyment of the routes themselves, not to arrive at the cathedral and revere the relics of Sant Iago . The cathedral website clearly states that walking the Camino is not the goal. The goal is to arrive at the cathedral and revere the remains of the saint in the crypt.
So, if our goal of returning to the Camino is to relive the journey, are we entitled to always use the shelters that were established for 'pilgrims' - not for hikers, wanderers, fans of the Camino? Should we leave the albergues for first time pilgrims?
All my research on the establishment of the albergue system shows that they were never meant to cater for all pilgrims. Refugios were established in remote places where there wasn't sufficient accommodation and these were meant for all pilgrims. But, AMIGOS decided to only establish one albergue (for the less affluent pilgrims) in towns and cities where alternative accommodation was plentiful.
This is from the Roncesvalles website:
"Ninguna localidad del Camino de Santiago tiene la obligación de disponer de plazas de albergue para todos los peregrinos y más si hay hoteles en la misma localidad.
"No town in the Camino de Santiago is required to have hostel places for all pilgrims if there are hotels in the same location."
I walked the Camino 4 times between 2002 and 2009 and understood that 'to be a real pilgrim' I had to stay in the albergues. By 2009 the bed-race had become ridiculous and I realised that, as I wasn't a 'real pilgrim' in the religious sense (I'm not a Christian, not Catholic, do not revere old bones), there was no reason for me to stay in albergues if I didn't have to. There are a few traditional albergues that I really love and will always try to get a bed in them if possible. They need a few paying guests to keep going otherwise they won't be able to pay for electricity, water, toilet paper, gas for the stove and will all have to close.
Perhaps those with financial limitations could plan shorter walks to fit their budget? Although this could also exclude you from staying in the albergues after FICS decided that long-haul walkers should have first choice in staying in the albergues. When it was suggested that some people with very good reasons and motives for walking the pilgrimage to Santiago can't do long distances the reply was :
"Running for a role with just effort? It's your problem. We are not interested. The only thing we want is not to break the pilgrimage by the absurd massification of the last 100 km. Nothing else. But nothing less. Is it so hard to understand? "
As I said, I'm not sure of the solution but to answer Johnny's suggestion, I don't think the private albergues should have to bear the responsibility of housing poor pilgrims when the original plan was not to have more than one albergue in each city.