<ember 31, 2010, the fiber-optic cable that connected Northfield, Minnesota to the outside world was severed. For nearly thirteen hours, from the early morning to 6 p.m., the telephone and Internet services of both Carleton College and St. Olaf went down. ITS workers and phone company employees were called in from their holiday break and after a frantic day of work, which involved digging a hole in the Arb to access the fiber-optic cable and repair it, Internet service was restored. No students were present on campus at the time, and the only knowledge we had of the incident was a press release issued by Carleton after the Internet connection was restored.
However, when students arrived the next day and unpacked their belongings, they found themselves unable to log into the school’s wireless network. Like most of the students here, I assumed that this was just an aftereffect of the same problem that had taken down the Internet the day before. The truth, however, is stranger and much more complicated.
“The wireless and the internet problem were two separate events,” said Chris Dlugosz, the network manager at ITS. To find out the true story behind the problem, he explained, you need to go back to the beginning of this year’s fall term, when the wireless network at Carleton suddenly became sluggish and unreliable, sometimes refusing to let users connect or disconnecting them while they were online. The problem, Dlugosz explained, was a sudden increase in the number of wireless devices Carleton students were using. “During fall term, wireless use doubled because of new devices like smartphones and iPhones,” Dlugosz explained. The software that handles Carleton’s wireless network is out of date and had trouble handling so many connections at once. In addition, the software placed an upper limit on the number of devices which could use the network at any given time. “The software was only licensed for 1024 simultaneous users,” Dlugosz told the Carletonian. “Until Fall 2010, we never exceeded that limit.” Smartphones and other wifi-enabled devices will sometimes log into the network when they are switched on but not in use, increasing the number of devices on the network – and once there are 1024 devices linked to the network, it will refuse to allow any more devices to connect. “We were hitting the license limit” – the 1024-device limit- “with only 700 active users,” because of inactive devices like iPhones, Dlugosz explained.
To combat the problem, ITS enacted a temporary fix in the late fall by re-licensing the system for 8192 users. In addition, after most of the students moved out at the end of Fall term, they undertook a project to create a separate network for the Carleton Library. Splitting the network in two would reduce the number of connections each network’s software had to handle, increasing the network’s speed. To understand what happened next, Dlugosz said, you need to understand how a wireless network operates.
In order to distinguish separate networks, each wireless receiver broadcasts an SSID (“Service Set Identifier”) – a code that identifies the specific wireless network to wireless devices like computers and iPhones. The library network would be completely separate from the main campus network, and broadcast a different SSID. However, when ITS attempted to set the library network’s SSID on Thursday, December 30, they accidentally changed the main campus network’s SSID by mistake. As a result, computers and other wireless devices were no longer able to recognize and connect to the network. “The problem wasn’t noticed immediately,” Dlugosz said, “because it was break, and no one was using the network.” The next day, the fiber-optic cable which contained Carleton’s Internet connection broke, and ITS suddenly had much more significant problems to worry about. “In the days after we lost the Internet connection,” Dlugosz said, “we were very carefully monitoring the Internet traffic in and out of our school, and couldn’t spare any time to check the wireless and notice that something was wrong.” Even when students started to arrive, ITS’ monitoring software didn’t show any problem with the wi-fi. “Only about 20% of campus internet traffic goes through the wi-fi network,” Dlugosz said. The other eighty percent goes through wired connections, such as the Ethernet ports which can be found in most classrooms and dorms. The monitoring software was registering a reasonable amount of internet traffic as students moved in, and it didn’t display the fact that none of it was taking place over the wireless network. In addition, the network equipment never showed any problems. “All of the wireless equipment on campus was still operating,” Dlugosz explained. “It was simply inaccessible to the students” because it wasn’t broadcasting the proper SSID. On the first day of classes, when the help desk opened, it was immediately flooded with calls from students and staff who couldn’t access the wireless network. After confirming that all of the wireless receivers on campus still worked, ITS checked the wi-fi software settings and re-set the Carleton network to broadcast the proper SSID. “The problem was fixed in fifteen minutes, and twenty seconds later there were over four hundred devices on the network,” Dlogosz said.
As of this time both the faulty fiber-optic cable and the wi-fi connection problem have been completely resolved, but ITS is still working to improve the system. This winter, Dlogosz told the Carletonian, we will be installing new wi-fi software which doesn’t limit the number of users that can log on. According to Dlogosz the installation will take down the network for about 30 minutes, and no fixed date has been set for when it will occur.
Carleton’s disconnection from the Web on December 31st and the loss of wi-fi at the start of the term had no real connection to one another; they were separate problems which happened to co-incide. If there is any connection between them, it is in the competence and dedication of the ITS workers, who not only resolved both problems in remarkably short order but sacrificed their time off to do so.