Thursday, May 5, 2011

To Cool Or Not Cool...

I don't know about you guys, but I'm super happy that spring has finally arrived. This winter was just cold, snowy and not a lot of fun. Luckily it looks like nothing but sunshine and butterflies from here on out. It'll be time to start busting out the bathing suits before you know it. With the spring comes the rain, it's a necessary evil that brings with it trees and shade which we'll need in the summer. Much to my surprise though, it's been raining indoors. How is that possible, you ask? It's a very easy explanation.

For as long as I can remember, one of the two air conditioning units in my server closet (I can't even call it a datacenter) has been malfunctioning. The room is not very big, but unfortunately the architects didn't take into consideration what was going into the room and didn't plan it out too well. There isn't a lot of space and there are at least a dozen servers and a dozen network switches in there pumping out heat. The AC units we have are barely enough to get by as it is. If one of them goes down, we're in trouble. Since my boss has only been in his new position for a year, he's still in the "SAVE MONEY, LOOK GOOD FOR THE SUITS, KISS AS MUCH ASS AS POSSIBLE" phase of his employment. It's completely understandable. He's thinking about his career and anyone else would do the same thing. The issue I have is where he's trying to save said money.

I have been bringing up the issue of this faulty AC unit almost since the start of my employment. I've worked at companies that had similar AC issues and it never ends well. In fact this particular issue was the reason that I worked 14 straight weekends last summer. Being that I am the only support guy here, 14 straight weekends can really take its toll on you. Both physically and mentally. Especially when it happens during the summer. Instead of being in the car heading to the beach, I'm on a hot subway heading to work.

Fast forward to this week...

The AC has failed for 72 straight days. When I say fail, I mean it starts blowing hot air instead of cold. Or in this case it starts dripping water, which is blown throughout the room. Normally, turning it on and off will resolve the issue. The problem occurs when it happens overnight. The room can get pretty hot. I lost count of how many times I've come in and the server closet has been 90 degrees plus (which is a great operating environment for servers). Now that my new boss is here and he's looking to make an impact I figure he'll want to nip this issue in the bud. <Charlie Murphy voice> Wrong, WRONG! I even went so far as to get a quote from our AC service company on how much it would cost to replace the unit. Their answer, $10k. $10k? That's it? Really? With labor included? WTF?!??! LET'S DO THIS!! Alas, the head honcho said that price is a little too steep. I went on to explain to him that we easily have $750k-1M worth of equipment in that room. Spending $10k to prevent failures is definitely worth it. I also went on to explain that a server outage will cost us money on missed orders, user downtime, etc. Guess what? It all comes down to the money.

So what's your opinion? Do you think it's worth the incurred cost to insure that your network can function properly? I sure as hell do. Unfortunately for me, I'm not the guy who makes the call on this. I'm just the one who has to come in on the weekend to get everything back up and running.

Wednesday, May 4, 2011

Thanks For Trying


I had to leave the office early the other day. I took off around 2pm. I didn't know it at the time but there was a major issue being addressed. 

I come in the next morning and there is a brand new Windows 7 tablet on my desk. (See THIS POST for the reason we have a Windows tablet in the first place.) There is a note on it from my boss. He needed me to get it connected to the wireless network and that if I have any questions I should call him. My first question would be: How do you still have a job? You can't do this? Really, my girlfriend who is not technical at all could handle this in about 5 minutes, easily. My boss, his rock star comrade and the new desktop guy spent "the whole afternoon" (his words) trying to get it connected to no avail. Wait. What? The three of you spent a whole afternoon on this? Who was curing cancer or searching for intelligent life in the stars then? I wouldn't normally be so surprised but i built a wiki just for things like this that is only available to the people in IT. There is literally a document with step by step instructions on how to connect to our wireless network. Did I mention it has pictures too? Yep, I even took screenshots. 

It's really quite disturbing. The levels of ineptitude that I'm surrounded by is enough to break most people's spirits. I don't know what bothers me more. The fact that they were too dumb to figure it out, the fact that my boss and his comrade each make at least 40k more than me, or the fact that they were all too lazy to spend two minutes reading the instructions. Either way I had to waste a full three minutes of my day fixing this major issue for them. Glad i was there to save the day. It's just too bad there were no 4th graders around to do it instead.