I`d buy the ticket in advance, online, but I am a confessed overplanner. There are 5 or 6 daily services from Paris to Bayonne, but take note that not all of them are direct. Don´t forget to "compost" or validate your ticket before boarding the train (you punch them in the little yellow devices -just do as locals do- or you may risk a fine. If you choose to buy tickets on the spot, you can do that in any "gare" or central station (there are 4 in Paris) and some metro stations, like Chatelet (close to Notre Dame).
I love trains, but another option is by plane -actually it may be cheaper, and it is obviously faster. In this case, the airport is Biarritz (BIQ), from there you take a local bus or a taxi to Bayonne, it is just a short trip.
When in Paris, don´t forget to visit the Saint Jacques square -this is where pilgrims met to start the long journey to Compostelle.
Buen camino!