For me (native speaker from the NYC metro area), the three are mostly interchangeable. However, I wouldn't expect to hear "handbag" coming out of the mouth of anyone under sixty. I also wouldn't call my giant corduroy sack a handbag - I would only use that to describe a smaller, nicer bag, if I used it at all - but I wouldn't be surprised if my over-70 aunt called my giant corduroy sack a handbag. I generally use the word "purse" to describe my daily bag, but pocketbook is equally familiar to me.
please respond with whether or not you're a native speaker and where you're from!