I am a heathen in this regard, I guess. To me, all pistols are carry pistols, and therefor all pistols are defensive pistols, and therefor no pistols shall have optics, certainly not optics that cannot be detached immediately in case of problems.
BTW, the most common "problem" that I can see is condensation. I am outside in the cold. I come inside, and end up having to draw my gun. If the optic is covered in condensation, which probably will happen, I am in a metric crap-ton of trouble. Fancy coatings notwithstanding, at least one author experienced this issue when doing a writeup on RDSes for handguns in one of the gun mags...SWAT, I think? He went on to gush about the speed and utility of the setup, despite freely admitting that his entire sighting system was 100% useless due to the weather during a drill. I call this a predictable problem, and one that I will avoid by keeping the RDS off of my gun.
I know that Gabe Suarez & Cmpany, along with many others, are all ate up with the idea of RDSes on pistols. I concede that they are faster. I refuse to believe that they are reliable enough, given the issues with batteries, glass, switches, mud, blood, condensation, water, impact, etc etc etc.
Yes, the same can be said of rifle optics. The difference is that the rifle optic can be ejected, and the rifle is carried in your hand when at the ready, and you can check the status of the optic at any time, so your chances of being surprised with a dead or occluded optic is smaller. Handguns ride in holsters until needed, and will not be checked as often, given their role as a defensive weapon. If you have a rifle in hand, you anticipate using it, and you probably have an opportunity before setting out to check status. This is not the case with carry handguns. Sure, you can check them when you holster them, but how many hours ago was that?
Now, for range and competition guns? Have at it.