Wearing boxer shorts actually increases sperm count, but not permanently. Wearing briefs decreases sperm count, but as soon as the man switches to boxers, his sperm count returns to normal. There have been studies done that prove the sperm count increases if you wear boxer shorts as opposed to wearing briefs. So, no wearing boxer shorts do not decrease sperm count.