Yes, batteries + too much heat = BOOM!
Chewy’s right about keeping the tip clean.
Use a high wattage iron like 65 or 100. Use rosin core solder or buy some flux (pase or liquid is fine). Let the iron heat up to full temp. Melt some solder on the tip. If you have flux, put a little on the battery where you’re going to solder. If you don’t have flux, don’t worry about it. Now, touch the iron to the batteryand push some solder between the iron and the battery while moving the iron around in a small circular pattern. Keep melting solder on until you have a big bead of solder then stop melting solder but keep the iron on the battery. Keep moving the iron around in a circle for a few seconds. You’ll know when the solder sticks properly because the bead of solder will no longer follow the iron around, it will stick to the battery instead. If it doesn’t stick after about 10 seconds you should probably stop and let it cool down before trying again. If you pull the iron away and the solder is stuck to the battery but looks like a round bead then it didn’t stick, it’s just the crystalized flux holding it on, you should be able to break it off by hand. Try again!
-Peter