Twenty-odd years ago, I was involved in a Product Data Management system implementation. This is just part of a much larger story, but the salient point from the epic saga is that I worked for a psychopath, and he tried hard at making my life difficult. I never figured out why. I think it was because he blamed me for something my previous boss did to his project. Anyway, we’ll get back to him later.
I was operating as a sysadmin, tasked with ingratiating the main admin from France to install an application on our servers, here in the US. At the time, corporate IT had just made it policy that no one but them could have root on machines hosted in their data center. On Unix (as opposed to Windows), I didn’t mind. That works just fine. However, the other admin had made getting root his #1 requirement. I told him of the policy. He didn’t relent. So I tried to elevate the coming train wreck with my management and everyone in corporate IT, hoping that something could be worked out before he arrived.
The guy shows up, shakes my hand, and asks me for the root password. I get on the phone with the main Unix admin. They finally relent, and allow me (because I’ve known them for 6 years by that point) to sudo to root to setup all the prerequisites.
The other admin is furious, tells us he can’t do anything until he gets root, and goes back to his hotel. Next day. Big meeting. Everyone on the phone. Group in one office, corporate IT in theirs, admin from the hotel, boss in the UK. I ask: “Michael, what specific commands do you need to run as root?” He says — get this — “You get in your car, and you turn the key, and it starts up. You don’t know how; it just works.”
In our room, we all just looked at each other in disbelief. First of all, he was talking to a bunch of mechanical engineers who happened to fall into implementing a PDM project. We all understood exactly how cars work. Second of all, everyone on the call would expect “the expert” at installing the application stack to be able to answer the question.
It was clear there was no arguing about it further, and the project had to get done so that he could shuffle off back to France, so they gave him root, and he did his thing from the hotel, and never spoke to me again.
After all the nonsense, you know what the problem was? The application server was configured to run on port 80, out of the box. That’s it! It assumed it would be running on the standard, privileged port. We could just as easily have configured it to run on port 8000, or port XYZPDQ. It didn’t matter! We had a load balancer running on port 80 in front of it. It could have been any port we wanted! Our “expert” admin couldn’t understand that, and my fearless management wouldn’t hold him accountable for such an elementary understanding of what he was doing.
In the weeks after, I realized that my boss had made me the scapegoat with upper management for the situation, because I was the one that tried to head this disaster off at the pass. Since I had sent emails, and talked about it, apparently I was the one who was causing the problem. This was just one of the many conflicts with my psychopathic boss. I had to learn a lot of hard lessons about politics over the 3 years on that project, but this one backfired in the most unexpected way.
Unfortunately, I had basically the same sort of thing happen again a few years ago. I tried to warn my management that IT was telling me something really, really stupid, and that it was going to come to a head in a spectacular way. But they couldn’t understand anything I was telling them, and trusted that IT knew better than I did. The problem is that IT didn’t want me to be working on the project. They felt they should have been the ones to “get the business” to develop it, and were actively trying to slow me down. Unfortunately, I didn’t learn what else to do in this situation except continue to try to educate the people who are looking at me like I’m crazy. Anyway, maybe I’ll blog that one 20 years from now.