Like for instance, standard alkaline batteries work badly in digital cameras. I would think a battery is a battery. Why is this??

Cameras need, for the flash, a battery that can deliver a high current for a brief period as the caps charge up for the next flash.

That means the battery has to have a low internal resistance, and not all batteries are good in this area.

There are not many other distinguishing features of batteries. The only ones that come to mind is the ability to hold charge for a long period, or the ability to hold a lot of energy.

.