Being vegan is never going to be easy, but you can do it without nuts. Lots of soy and legumes should help. Have you considered going vegetarian instead? It’s also difficult, but you can find ethically produced dairy products if that’s your main concern, and eating moderate quantities of cheese, eggs, yogurt, etc. would be a good, healthy supplement to your diet.