So... you have at least 3 options:
1. Catch the one Pesa bus per day from Bilbao to Bayonne (as mentioned by biarritzdon). It leaves at 7am and costs around €20 (according to this year's winter timetable, but in my experience the timetables seldom change from year to year). A 7am departure will mean quite an early start, especially if you don't know the city that well, and you might need to queue for bus tickets, so that will also need to be factored into your plans.
2. Catch an Alsa bus from Bilbao to Irun (there are about 10 buses per day, costing around €8). Not entirely sure where the bus stops in Irun, but shouldn't be too far from the Hendaye train station. From Hendaye, it's a 41-minute journey to Bayonne, there are trains every hour (SNCF, costs €7.90)
3. If you've been to Bilbao before and aren't that fussed about visiting it again, you could also catch a bus directly from the airport eastwards to San Sebastian (it's bus number DO50B with Pesa/Lerraldebus). Buses leave every hour, the journey only takes 1h15. San Sebastian a great city if you maybe wanted to spend your first night there. Possibly also easier to navigate - and, as biarritzdon says, more transport connections. On the 12th you could either catch a direct bus to Bayonne, or else, my personal favourite (I love trains!) catch the local Euskotren which drops you off just outside the SNCF station in Hendaye.