I believe the wrist strap is more important than the anti static mat, but that said I only use the wrist strap and not the mat. If you don't have a grounded strap (either on your wrist, ankle, or shoe), then remember that you can build up a charge moving around so you have to touch your anti static mat or whatever it is electrically connected to so you don't zap your components.
Instead I ground my case while working on PCs, and I always have an anti static bag ready for a component if I'm taking it out of the case. Essentially my computer components only ever touch the inside of the anti-static bag, my fingers (while I'm wearing a wrist strap), and the computer case / components inside the computer case which is grounded either by having the PC's PSU plugged in but turned off, or through the odd setup through a butchered AT power supply as I pictured below.

Figure: Oddball grounding scheme that goes wall -> cheap PSU that's turned off -> alligator clip -> case of computer I'm working on