Some did, some didn't. If you weren't a Calvinist, you weren't really welcome in early Massachusetts. Maryland was founded as a Catholic colony. Rhode Island was the first colony established with religious liberty as a founding principal, but even there, it took a while before they decided that non-Christians were OK. The Dutch governor of New Amsterdam tried to expel the first Jews to settle there, but the Dutch East India Company overruled him and et the Jews stay, so when England took control and changed the name to New York, they inherited a bit of tolerance.