Technically, if your maternal line is Jewish, you are Jewish and have no need for conversion, while if your Jewish ancestry is paternal at any link in the chain, you need to convert. Practically, however, the person who is technically a Jew but has grown up cut off from Jewish culture and traditions will have a lot of learning to do before they fit into the community. In contrast, someone who is technically a non-Jew but has grown up with significant Jewish education may have an easier time. If, however, there is merely a family memory that grandpa was a Jew, with no significant knowledge of Judaism, the most this can provide to the potential convert is motivation to learn.