You can wear trunks, ie. a swim suit with legs, not a speedo. As long as you have a modest appearance, the gym won’t mind. But they aren’t designed for exercising and may be very uncomfortable. They aren’t designed for handling your sweat. And they may cause very painful chafing.