From Figeac, which is worth exploring a bit itself, you can hop a train and easily visit Rocamadour. A great day at a great monument!
I wouldn’t spend extra time in Conque, which has a great church but it’s really a three-hour town. Perhaps Cahor, just because it’s big enough to have different...