The call forwarded to me seemed simple enough. A vendor's technician was at a client's site to install a card access system, and he was having issues connecting to the application running on a Windows Server 2003 over the corporate LAN. He requested that the network be checked to ensure there were no network issues. I proceeded to the site with my tool kit and a laptop with a sniffer program installed.
Physical connectivity didn't seem to be a problem, as both the server network interface and the port on the switch to which it was physically connected displayed a link light. To be sure, I used a cable analyzer to ensure the integrity of the Cat 5E line. When troubleshooting possible network issues, it's often best to start from the physical layer and work up the OSI model.
The switch registered the server's media access control address on the correct port, and pings from switch to server were successful. The server was running two network services: Terminal Services and a proprietary card access application. I could Telnet to TCP Port 3389 from my laptop plugged into the switch and configured with the correct IP network information, but not to the access system's TCP port.
At this point, I was certain the issue was with the server and not network-related, but the card access system technician held a differing view. After a brief dialogue exchange, I realized he was quite serious when he said that my straight-through Cat 5E patch cable was incorrect because Ethernet relied on "twisted pair" and therefore needed a "turnaround" cable.
Keeping a straight face
Obviously, I had to keep my initial reaction to myself, even though this was one of the more humorous -- and incorrect -- networking statements I'd heard in quite a while. I realized I was dealing with a technician who was not very knowledgeable in Ethernet networking basics. Up until recently, the system that he had installed for years had relied on dedicated serial lines to the card reader controllers, and Ethernet was a relatively new concept for him. I explained that I was completely sure network connectivity was fine and that we should check the server.
Unfortunately, this sort of exchange happens more often nowadays. As more nontraditional equipment migrates to IP network connectivity, technicians used to dealing with older connectivity methods are forced to learn an entirely new transport mechanism.
As network administrators, it's our responsibility to recognize that the experts installing such systems may not share our networking skills and therefore may require a more basic approach to solving a connectivity problem than, say, a Unix system administrator. The following are a few tips I've learned to help smooth the process when the installation doesn't go as planned.
Respect the technician's abilities
In the above example, my initial reaction was more incredulous than anything else. I found it hard to believe that the card access company would send a technician so lacking in basic network knowledge to install a system that relied on TCP/IP networking to work.
It would have been easy to discount the technician by immediately requesting to speak with a high-level engineer at the company's support site. However, while that may have resulted in the solution, undermining the on-site technician would have been hurtful to him both personally and professionally. Here was someone who had worked on similar systems for many years and was most probably doing his first on-site installation of the new setup.
Instead of escalating, I used this opportunity to help educate. I took a trace of both the successful connection with the server Terminal Services port and the failed connection to the proprietary application. I showed him the TCP three-way handshake and explained what it was. I detailed my troubleshooting methodology without getting too deep into "techno-babble," discussing what the link lights and successful pings meant.
Following this, the technician checked the server and discovered the card access service wasn't running. He contacted his engineering support group, and in quick order, they determined the problem and implemented a solution. Afterward, he asked if I could send him the trace so that he may refer to it in the future as a learning aid, and he thanked me for the explanation.
This is not to say that it's never appropriate to escalate the problem to the vendor's engineering level. The on-site technician often is not an expert system administrator for the system that he is installing. If you aren't getting the technical answers you need, escalation may be the only alternative.
While we all take pride in our extensive network knowledge, it's important to realize that others who may not share in the knowledge often have years of experience in other areas. I firmly believe that as network professionals we have an obligation to be ambassadors of the profession by being positive and helpful at all times.