Customers frustrated with Fibrant outages

Published 12:00 am Tuesday, July 24, 2012

By Emily Ford
eford@salisburypost.com
SALISBURY — Rick Anderson McCombs said he lost more money due to Fibrant’s unexpected outages last week than he has saved by switching to the city’s new broadband network.
Forty-two customers walked out of the Sidewalk Deli Thursday because they couldn’t pay with a credit card while Fibrant was down, said Anderson McCombs, who owns the downtown restaurant.
The deli lost more than $500 in one day, he said, including a customer who wanted to buy 30 sandwiches for an office meeting.
“That’s the one we felt most painfully,” Anderson McCombs said.
Like Fibrant’s 2,000-plus other customers, Anderson McCombs received several notifications from the city warning customers they could experience some downtime between midnight and 6 a.m. for three weeks while Fibrant added a second Internet circuit to help lure more commercial subscribers.
But when the network came back up early Thursday morning after the planned midnight maintenance, things went awry.
“Obviously, that was a bad day,” said Mike Jury, Fibrant’s new general manager.
Fibrant, which sells Internet, phone and cable TV services in Salisbury, was down for hour after hour for some customers. Sporadic outages continued Friday.
The city has temporarily suspended the three-week upgrade of Fibrant to avoid more widespread outages.
Jury said he will not proceed with adding a backup Internet circuit until he’s confident the maintenance can proceed smoothly.
He said crews discovered equipment that could not handle the load as Fibrant was brought back up early Thursday morning.
“We stumbled across equipment that had issues that we were not aware of,” Jury said.
The equipment has not properly supported the network for some time and could have been an underlying issue during previous outages, he said.
Jury would not name the vendor that provides the equipment but said the company is working diligently to come up with a solution. He said the equipment connects the fiber-optic cable in the headend before Fibrant sends the signal out to homes and businesses.
The headend, located in the city’s Customer Service Center on Martin Luther King Jr. Avenue, is the master facility for receiving, processing and distributing signals over Fibrant.
Jury said he has been conferencing with all Fibrant vendors to make sure equipment throughout the network works well together.
“Right now, we have it stabilized,” he said.
If redundancy is crucial to lure commercial subscribers like hospitals and colleges, customer Brad Farrah said he doesn’t understand why Fibrant didn’t offer a back-up Internet circuit in the first place.
The city justified borrowing more than $33 million to build the network because it would be an economic development tool, Farrah said.
“If we’re doing this to attract business, why wasn’t it set up for businesses right from the start?”he said.
The answer wasn’t clear Monday. Key officials who launched Fibrant — broadband director Mike Crowell, City Manager David Treme, marketing director Len Clark — no longer work for the city.
Jury, who took the helm at Fibrant in March, said he doesn’t know why the network has no redundancy. New City Manager Doug Paris said he would find out.
Subscriber Daniel Michael said the outages were frustrating.
Michael has a scanner that monitors emergency communications for Rowan County and Salisbury, which he then broadcasts through www.radioreference.com.
“Every time Fibrant went up and down, it caused the software to freak out, and then I start getting emails from people listening wanting to know why the feed was constantly going up and down,” Michael said in an e-mail.
According to his agreement with the website, Michael said he must provide the live feed for a certain amount of time.
In just two days, Michael said he nearly exhausted the total downtime he’s allotted each month before they terminate his feed.
“I seriously hope that when they get ready to do the next (upgrade), they have a better plan in place, and their equipment is in better shape to support all aspects of Fibrant,” Michael said.
Skip Wood said he’s having second thoughts about installing Fibrant in the Kress building downtown. Six companies under the Sharp Capital Group umbrella will begin operating in the building this fall.
“We need to feel really good that we will be up and running all the time,” said Wood, a partner with Sharp Capital. “Based on recent events, I’m struggling.”
Wood said he plans to learn more about the upgrade, which Jury said will give the network the reliability commercial customers demand.
While the outages were frustrating, subscriber Greg Shields said he could never go back to Time Warner Cable after experiencing Fibrant’s symmetrical upload and download speeds.
“I cannot get comparable service from any other provider,” Shields said.
He said he expects some downtime with a residential telecommunications service, as opposed to a business-grade service. Customer service could have performed better during the outages, said Shields, who got a busy signal or was placed on hold for more time than he thought was acceptable.
Subscriber Marc Hoffman also could not get a Fibrant employee on the phone until 11:30 p.m. Thursday. Hoffman said his service was down for nearly 15 hours total.
A composer, Hoffman uploads large files at night and was surprised when Fibrant went down for the entire scheduled maintenance period both Tuesday and Wednesday nights, before the major outage on Thursday.
“The letter from the city didn’t imply that the service would be down for long periods,” he said. “I thought it would be sporadic.”
Hoffman said the city should pro-rate bills to account for lost hours.
But city officials said they’re not sure how many customers lost service and for how long. The entire network never went down at once.
“That’s a hard number to quantify,” Jury said.
Assistant City Manager John Sofley said no one has canceled a subscription due to the outages, and most customers have been understanding.
City officials have not discussed pro-rating bills or issuing credits, Sofley said.
Mayor Paul Woodson said he is not discouraged but apologized for the “glitch.”
“It’s one of those things where, like anything a year or two old, you’re just trying to get all the bugs out,” Woodson said.
Fibrant offers the best technology available, he said, which means it’s a complicated network.
To avoid a repeat of last week’s outages, Fibrant engineers are working with the vendor to update the code and prevent the equipment from becoming overloaded when the network comes up from an outage, Jury said.
“Updating the code will solve the problem,” he said. “I feel very confident.”
Jury said he will test the updated equipment in Fibrant’s onsite lab before putting it into production. Then he will move forward with the scheduled maintenance, which could take place several weeks from now.
“We have a full-court press on all vendors to get this resolved,” he said.
While outages are unacceptable, Jury said he’s glad the problem came to light “when we had all hands on deck.”
Because of the scheduled upgrade, all Fibrant vendors were accessible and on heightened alert, he said.
Jury attributed the problem to Fibrant’s cutting edge, fiber-to-the-home technology.
“When you are on the bleeding edge of technology, sometimes you have bleeding-edge problems,” he said.
Despite losing more than $500 during the outages, Anderson McCombs said he will stick with Fibrant. He switched 10 months ago from AT&T, saving $38 a month.
“Love it or hate it, it’s best for the city to get it going,” he said.
The city has invested too much money in Fibrant to turn back now, Anderson McCombs said.
“I want to see this survive,” he said. “I sure want them to get it right.”
Contact reporter Emily Ford at 704-797-4264.