The main part of the brain that is responsible for memory is the amygdala, the hippocampus, the cerebellum and the prefrontal cortex. The amygdala involves in fear and fear related memory. The Hippocampus is associated with declarative and episodic memory as well as recognition memory.