The alert I got was:
FREE MSG: Chase Fraud-Did you use card ending xxxx for $xx.xx at INGLES MARKETS on 07/13? If YES reply 1, NO reply 2
In cybersecurity, getting these kinds of alerts is a pretty common kind of scam. Attackers will send out lots of these kinds of SMS and email and try to get you to verify your account, essentially tricking you into sharing sensitive information.
If you ever get one of these kinds of alerts, you should try to verify it independently. So I logged into my credit card account and saw that there were several purchases that morning. Looking up the name of the store, it appears to be a chain of grocery stores in North Carolina. Ok so definitely fraud.
So I responded with a "1" to the SMS message, and it said that Chase would call when a specialist is available, or call the number on the card.
There's a minor risk here with the first option, which is that getting a phone call from an unknown number doesn't mean that it's legitimate. In computer security, this is the mutual authentication problem, which is that while your credit card company can verify if it's you or not, you don't have any easy ways of verifying if it really is your credit card company calling you.
The safe thing to do here is the second option, which is to call the phone number on the back of the credit card.
Now, as someone who does research in cybersecurity, even all of this is not guaranteed. It's possible that a hacker could have intercepted my web browser request to Chase's web site, knew the last 4 digits of my credit card, knew my mobile phone number for SMS (SMS can be spoofed), and intercepted Chase's 1-800 number, but the combination of all of these is pretty low. Plus, if a hacker were skilled enough to do all of the above, they would chase after bigger fish than me.
So a new credit card is on the way, and the damage is limited, both for me and for Chase. I should also say good on Chase for having an excellent fraud detection department too. This is actually the first time Chase has warned me about possible fraud on my credit card, despite all of my travels around the world, and they got it right.