There’s no need to stress, though, since the hit on your credit score is temporary and you should see it bounce back within a few months.
Refinancing affects your credit score is because the lender conducts a hard inquiry on your credit report, which will decrease credit score about 5-10 points. Again, this is temporary. If you can save hundreds of dollars in the long run, a slight dip to your credit score isn’t a huge deal.