If it doesn't make a regular clicking sound when turning a corner (with the wheel all the way left or right) then it's less likely to be a CV joint issue.
I believe the click you hear is more likely to be "driveshaft ping". On these cars the end of the driveshaft is splined and fits into a spline on the wheel hub. A bolt in the end of the driveshaft retains it and stops it from sliding out of the splined hub. You can see this bolt if you remove the plastic "volvo" trim cap in the very centre of the wheel.
Whatever magic volvo do with their cars in the factory, they don't really make this clicking noise. Mine's made a click ever since I replaced a broken drive shaft, and it even does it since a pro garage replaced that same driveshaft some 5 years/50k miles later, post me breaking the CV joint and them not being able to find a replacement that fit the non-oem driveshaft I fitted years earlier. For the first couple of weeks after the repair, the tick noise was absent, then it returned..
The cause is a very slight rotational movement of the shaft within the splines of the hub, resulting in a ticking noise. If you put the car in reverse and lift the clutch with a reasonable "bang" it will spin the driveshaft backwards against the hub splines and make a tick, then if you select a forward gear and lift the clutch again with some vigour you'll hear the tick again. It shouldn't really do it again in the forwards direction, until you apply some action that rotates the driveshaft the other way, such as reversing. If it is driveshaft ping you'll also be able to produce the ticking sound regularly if you "kangaroo" (cycle on and off the acelerator, causing a jerking back and forth motion) the car in a forward gear as you drive along. The tick should be heard every time the car lurches forwards or backwards
The noise is relatively harmless; mine's been doing it for the last 50k or so miles without getting noticeably worse. If it ever did get so back that the splines in the hub or shaft wore completely away, it's a simple enough repair