April 25, 2015

Fibrant vendor defends equipment

Published 12:00 am Friday, December 7, 2012

SALISBURY — The company that sold Fibrant the equipment city leaders say has been unreliable is defending its access gear as top-of-the-line.
Salisbury’s new high-speed broadband utility has struggled with repeated outages due in part, the city says, to Zhone’s access gear not performing as promised.
The access gear transmits video, Internet and phone signals from Fibrant’s technical center on Martin Luther King Jr. Avenue to homes and businesses in Salisbury.
Brian Caskey, chief marketing officer for Zhone, said Fibrant is using Zhone’s flagship MXK platform, which has been a “tremendous success” worldwide.
Oakland, Calif.-based Zhone has deployed more than 4,000 of the same units around the globe, Caskey said.
Zhone’s fiber-optic business has increased 700 percent since 2009, Caskey said. Fibrant is a fiber-optic network that competes with private providers like Time Warner Cable.
Caskey said built-from-scratch networks often encounter difficulties, but Fibrant has had more than usual.
“It’s been a situation where we have all been challenged with more troubles than we normally see,” he said.
Zhone and other vendors who sold equipment to Fibrant are working with city staff to diagnose and solve problems that are causing the network’s instability.
Caskey said Zhone has gone above and beyond its contract to help Fibrant be successful.
Under state law, the city is required to award contracts to the lowest bidder. Zhone won the Fibrant contract in 2009 with a low bid of $2 million for equipment, engineering and construction of a passive optical network, or GPON, that brings fiber-optic cabling and signals to 5,000 homes and businesses.
Zhone’s bid was nearly $300,000 lower than the next lowest bid.
To date, Zhone has billed the city $1.2 million, including $186,929 for access gear and about $1 million for other equipment such as the optical network terminal, or ONT, boxes on customers’ homes and businesses.
Under the contract, Zhone is also responsible for installation of access gear and providing maintenance, including software upgrades for five years. 
Sometimes, new networks launch with few problems, Caskey said. But Fibrant has experienced growing pains.
No amount of planning on the city’s part could have prevented some of the issues, he said.
“Nobody could have foreseen some of the problems that we’ve encountered,” he said. “When you start a network, you try to plan the best you can, but challenges always happen.”
With complicated networks like Fibrant, which uses state-of-the-art technology and has few peers, finding and correcting problems is often trial and error, Caskey said.
“There are many moving parts,” he said.
The access gear is a relatively small part of the overall network, Caskey said.
“There could be a lot of other things going on that could be providing instability for the network,” he said.
While not denying the access gear may play a role in Fibrant’s unreliability, Caskey said “you don’t get anywhere with a finger-pointing contest with your customer.”
Fibrant General Manager Mike Jury, a 24-year veteran of the cable industry, said the access gear has been unreliable since the city launched the $33 million network in December 2010 and has failed in every outage the network has experienced.
Because of access gear’s interface with the network, the equipment is always involved in an outage, said Mike Fisher, Zhone’s vice president of sales for North America. That doesn’t mean the gear actually caused the outage, he said.
The equipment is not underperforming, Fisher said, and Zhone will continue to work as a partner with Fibrant.
“We are looking at their entire network right now and reviewing it with the city to see how we can improve it,” he said.
Jury said Thursday the city is sticking with Zhone. Fibrant tested access gear made by Zhone’s competitor, Calix, after a 13-hour outage on Nov. 9.
Zhone recently provided a new code to load on the gear, called a firmware update, which has run successfully in tests, Jury said.
“We are going to keep testing and when I feel comfortable, we will move small blocks of customers over,” he said.
Fibrant initially will move customers who have agreed to participate in the test, such as city employees, Jury said.
If it works, the firmware update will give Fibrant the stability to avoid incidents like a two-hour outage on Dec. 3 that affected some customers.
In that instance, a malfunctioning DVR in someone’s home sent bad information back through the network and caused the outage, Jury said.
The updated access gear would prevent that from happening, he said.
“These are the lessons we are learning as we are growing,” Jury said.
Through a process of elimination, Fibrant crews determined the faulty DVR was causing the problem, disconnected that customer and restored service to others, he said.
Jury said while Fibrant will probably have a four-year adjustment period including equipment trials and errors, he disagreed with Zhone’s assessment that Fibrant has had more problems than other start-up networks.
Overall, Fibrant and Zhone “have a great relationship,” Jury said. “They understand the challenges.”
Jury said he is still diagnosing the cause of the lengthy Nov. 9 outage, when Fibrant experienced some kind of surge that caused equipment failures and brought the network down for all 2,200 customers.
Electricians have been at the technical center to look at the network’s grounding and will make recommendations.
“We know we had a surge,” Jury said. “The question is, why we weren’t protected from that surge.”
Contact reporter Emily Ford at 704-797-4264.