One of the most interesting weeks ever at work. Newish guy in the networking group accidentally pulled out all the fiber connecting servers to our SAN. Did I say that he did this in the middle of the day on Tuesday?
Lessons learned
It took us 2 hours to find the right connections and plug everything back in. A lot of the documentation was out of date and wrong.
Redundancy is awesome. A 2 node cluster with 4 HBA’s only needs one for both instances to come back up. Out of 30 or so cables we needed to find the connections for only 3-4 to get everything back up and did the rest with less stress.
And Microsoft is awesome as well. Everything came back with no corruption as soon as the fiber found its home again