One of the biggest decisions you will ever make in your law enforcement career is which backup firearm to carry. I say this because there is some merit to the theory that it makes sense to carry a backup gun that is chambered in the same caliber as your primary service handgun.
In my case I used both methods. I carried a backup gun that was chambered in the same ammunition that I carried in my service handgun. I also carried a Walther PPK or a Walther PP in .380 caliber when I carried a .38 Special Smith & Wesson service revolver as well as a 9mm or .45 ACP pistol.
After having done both, I can say I never felt like I had an advantage when I went with the more "common sense" route and carried a backup gun that was chambered in the same caliber as my primary service handgun.
Even today with so many primary full-size pistols that also have companion secondary pistols, in the end you may best be served by carrying a more easily concealable five-shot .38 Special revolver or a well made .380 pistol like the new Smith & Wesson Bodyguard.
Two of the most critical considerations when you select a backup gun are whether or not it is flawlessly reliable and if you can easily pass a firearms qualification course with it.
But your backup gun must also be reasonably effective. This means that I would probably draw the line and limit the authorized backup guns in my department to being no lower in scale than .380 and .38 Special. I don't care how good of a shot you may be with a two-shot derringer or some .32 ACP mini pistol that fits in the palm of your hand; you must have a cut-off when it comes to authorizing backup guns.
My personal basic requirement is that any handgun I select as a backup gun must also be a weapon I would be confident carrying off duty and may be possibly compelled to take enforcement action with.
So, if you don't like the idea of taking down a bank robber or a knife-wielding violent emotionally disturbed person (EDP) with a five-shot .38 Special revolver with a snubby short barrel, then you need to kick things up a notch. Carry a Glock 26, a Glock 27, or some other suitably sized handgun that you could carry on duty if you had to.
The other day I met two deputy sheriffs who were not carrying backup guns because they felt confident enough being armed with a patrol rifle in 5.56mm caliber that came with three 30-round magazines. My problem with this concept is that law enforcement officers do not walk around with an M4 carbine or some other military style semi-automatic rifle in a substantial caliber with extra magazines.
A true backup gun must be carried on your person at all times, so suck up the slight inconvenience and pack heavy, especially when you work without a partner. Trust me when I tell you that should your primary pistol malfunction for any reason or get lost or damaged, carrying a second handgun — even one that is highly concealable — will make you feel as if you've won the lottery. For you will be able to remain in the fight even if you're only armed with a backup gun as your temporary primary weapon.
This is why it is imperative that you qualify with your backup gun so you will know that your second handgun is capable of achieving a passing score on a LE firearms qualification course. Once you have this kind of faith in your backup gun you can then carry it effectively in harm's way with tremendous confidence.