You probably have male flowers and few if any female flowers. Female flowers have what appears to be the beginnings of the fruit back behind the flower. Male flowers have pointy protrusion in the center of each flower. Male flowers come before female flowers. Once you have both male and female you should have fruit, but you can always help out. Using a small craft paint brush take the sticky pollen from the male flower and deposit it in the center of the female flower.