The battery cannot be removed quickly! Inductance of the wiring stores energy and keeps the current flowing and decaying smoothly. There will be an opposite polarity, higher voltage pulse on the battery terminals, limited by the circuit capacitance only.