Indeed, they are bathing. Birds have oil glands near their tail section. When you see them preening (cleaning and smoothing their feathers), it is to remove dirt and insects, plus spread the oil. A water source helps to remove any debris, which is why we witness them splashing about with wings extended.