With rare and specific exceptions (usually related to the condition of the driving surface such as snow) correctly functioning ABS is not dangerous and occurances of circumstances where it is a help rather than a hinderance far outweigh those where the reverse is true. Modern ABS systems (pretty much anything from the last 20 years) are far better at maintaining the level of braking at an optimal level for balancing both minimising stopping distances and maintaing control and manuevering ability of the vehicle.
That said like any electrical or mechanical system ABS can go wrong and ensuring it is in proper working order is as important and ensuring that the rest of the braking system is functioning correctly.
If you are encountering ABS kicking in "often" then there is something wrong somewhere - I've been driving ABS equipped cars for over 15 years and hundreds of thousands of miles. And given that I'm just as dozy as everyone else I've had my fair share of hard braking moments yet I've only had ABS cut in less than a dozen times.
Hitting the pothole and ABS kicking in could be a momentary confusion of the system resulting from the differing wheelspeeds resulting from the pothole or it could indicate a malfunctioning sensor somewhere. Especially given you are seeing it rather alot in normal use. I'd be getting it checked for the peace of mind if nothing else.
Other things to consider that could be causing excessive occurances of ABS operating would be tires and driving style.
If your tires are of poor quality or excessively worn their grip levels could be significantly below what the car needs and it wouldn't take much for the brakes to overpower them and lock the wheels relatively easily under braking causing the ABS to try and compensate. As for driving style, I'm not trying to be offensive here but do you have a tendency to leave braking quite late? Are you frequently surprised by events ahead of you causing you to brake sharply?