You can close the card with the fee, just make sure you do it before the annual fee hits the bill. You might be able to just switch to the cheaper card, I have had that option with other cards. At worst they won’t let you and you can close the other card and open some other card. Or, better is to open a new card before closing that one because your credit will be higher before closing it.