The volume of offshore software development for U.S. companies has been growing for some time now. So at first blush, it's not surprising that Hyperactive Technologies, a company founded by Carnegie Mellon University robotics scientist R. Craig Coulter, would also look abroad for help in developing its specialized restaurant management systems.  After all, perfectly serviceable models for directing teams of overseas software engineers to crank out code for every sort of IT application have already been evolved through countless instances of technology outsourcing. 

For essentially all of them, the key requirement is that the company have a product in mind which it can clearly specify upfront.  So, for example, if you're building a database, you would tell the developers that you have X number of fields, of Y different types, that Z are the sorts of inputs that will be received, and such-and-such is the type of output you expect to generate.  In effect, it requires you to detail your specifications and then provide a proven engineering methodology that allows the offshore group to understand exactly what they have to build, how to build it, and then how to validate what they've built.

But what if you can't specify exactly what you expect to see at the end of the process or even a clear path for getting there?  For Coulter's company, which makes a robotic production management system for fast food restaurants that looks at incoming traffic through a video camera and then tells the kitchen staff which foods to prepare and when to prepare them, a certain amount of ambiguity is inevitable.  Dealing with that uncertainty is fundamental to the emerging science of robotics, especially when it comes to recognizing visual images.  But overcoming it not only required Hyperactive to re-invent the offshore management process, it forced the company to completely rethink and reorganize its own engineering function.

Iteration

"This is a new type of software development for offshore," Coulter acknowledged.  "When we talk about the robotic system's core capabilities - sensing and planning - we're talking about a very different class of software from the kinds that are typically offshored.  So we have been working with a group in India to develop an engineering methodology that allows these sorts of robotics software applications to be built outside the United States."

The reason a new sort of engineering methodology is needed, according to Coulter, is because algorithms for robotics cannot be specified in the absence of data.  "The only way to build a robotic vision system is to go out and collect video data and build a vision system that, in our case, can spot a car.  The designer uses that video data as input to the design process," he said.  "Computer vision, as a field, has not yet advanced to the point where a closed-form engineering solution can be calculated." 

As a result, engineering for computer vision is an area where the notion of doing it right the first time simply doesn't apply.  Instead, solutions are iterated - a process in which designers look through hours of video and then try one approach or another to interpret the data.  Along the way, they create filters and gradually improve on the system's performance as they go along.  Then they collect more data and go through the cycle again and again.  "You might go through that process ten times, or even a hundred times, as you go out and collect more video data," he said.  "That's because the phenomena you see in the first four hours of video may be very different from the phenomena that you see in the hundredth hour.  You'll never know all the input cases until you've collected an exorbitant amount of video data and processed it.  We're talking about tens of terabytes of data.  So you have to try out the vision program on the first terabyte of data, see how it works, iterate on the design and keep on doing that until you get to a point of stability." 

Science v. Engineering

"There's no such thing as a computer Vision Engineer - there's no book an engineer can go to and say ‘I want to be able to detect a car; here are all the methods that have been developed to date which are known to be able to detect cars.'  They don't exist.  So computer vision development still requires the scientific approach, which is basically iterative development and testing," he said.  "As a general rule, this is a community where you still have to iteratively try your solution and refine it as opposed to being able to plan it out mathematically and then build it." 

Industrial robots - the kind of assembly line devices that Coulter refers to as pick-and-place manipulators - operate in a controlled environment and only need to recognize a finite number of items and shapes in order to perform their tasks.  For Hyperactive Technologies, the challenge is that no two drive-thru restaurants look just alike.  On top of that, there are distinctive differences in weather and illumination patterns, as well as driving habits and other regional factors, that change the visual image and can throw a robot off track from one day to the next and one fast food outlet to another. 

That means taking a whole different approach.  "We've had to create an engineering methodology since there isn't one right now," Coulter noted.  "We've been working with our offshore partner, in Hyderabad, to implement that methodology and try it out." 

Here's how it works: "We're going through steps where we learn how to do the engineering process and then document it so that at each step we can offshore the next piece.  We already understood how to build filters in a way we could explain to the offshore group," Coulter said.  "Then as we learn to do that and how to document and explain it properly, the next piece gets sent over.  So we're in this iterative process of defining the engineering process, trying it out ourselves, and then once it works, shipping it to the offshore partner."

Inverse development

That process eventually led the company to turn its development process upside down.  "One of the major changes we've had to make is moving toward an architecture-central development process where you have a very well defined, top-down, structured design.  Historically, that has not been the case with vision systems.  Vision systems have almost always been built bottom-up.  So it's really taking the normal vision system development process and standing it on its head," he said.  "The normal process is the one that's followed in the university - that's how prototype systems are built.  But when you're building a production system, the importance of specification can't be overstated." 

Perhaps surprisingly, the huge time shift between the U.S. and Hyderabad, where there is essentially no overlap in daylight work hours, never turned into a problem at all.  "They're working when we're not," Coulter noted, "so they typically work through some part of the problem and then, at the end of the day, they have a list of questions that they email to us.  And our guys have all day to figure out the answers to their questions before the Indian team gets back to work.  So for both sides, the delay becomes imperceptible since it occurs in their off hours.  That's actually quite an advantage." 

Of course, the company also has help nearby.  "We have project managers on each side," he pointed out.  "There's a project manager who leads the team on the India side and then there are project managers on the U.S. side; the two managers communicate back and forth.  And then if there are technical questions, you'll get the appropriate technical people in on a conference call.  So having a project manager on each side has really been the best way for us to go." 

Exposing weaknesses

What has Hyperactive Technology learned, as a result?  "We probably learned the same lesson in a different context that I've heard from many other guys who have done offshoring, and that is the importance of being able to write good specifications," Coulter reflected.  "The fact that we're developing systems that have to be iterated has exacerbated that problem.  But to the degree that it has, it's been helpful to us because it has helped to expose the weaknesses in writing specifications for vision-type systems in a way that is going to be very healthy for us in the long term. 

"Every time I talk to anybody who's done offshoring, they always are very adamant about the need to have well-developed specifications and a well-developed testing process.  I couldn't agree more; the advantage in our case is that as we're trying to develop an engineering methodology for a computer vision system and the needs of the offshore team have helped drive us to develop an appropriate specification process." 

---