News
Events
Video
The Inventor's Eye
How Professor Eric Fossum invented the digital camera-on-a-chip—and is helping today’s students create their own technological innovations.
By Michael Blanding
Photograph by John Sherman
A mere decade ago, cameras were special-occasion objects. We brought them to birthday parties and beach vacations, capturing only the biggest moments of our lives and downloading them for safekeeping. A decade before that, digital cameras were virtually unknown; taking a picture meant loading film into a camera, determining whether a shot was worth using up a precious 24th of a roll, and hoping the image lived up to our memory when we sent it out for developing.
Now with smartphones, we carry cameras in our pockets virtually all the time—spontaneously catching photos of family gatherings, nights out with friends, or just pretty scenes in the park, and emailing or uploading them to Facebook instantaneously. We use these smartphone cameras to video-call friends halfway across the world, record our children’s school concerts, and apply makeup without a mirror. So much have these cameras become a part of our lives that we barely think about how much they have deepened our ability to capture and share the precious moments of our days.
The origins of the cell phone camera, however, come from as far away from our everyday experience as it’s possible to get—outer space. Solid-state device engineer Eric Fossum was working at NASA’s Jet Propulsion Laboratory (JPL) trying to make space camera technology lighter and smaller when he came up with a camera-on-a-chip breakthrough that made modern-day digital cameras possible. “I was born in October 1957, which was the same year Sputnik was launched,” says Fossum. “The space program was part of my life growing up, and so when the opportunity came to work for the space program it was irresistible.”
After that camera breakthrough, however, Fossum had to fight to get anyone to take it seriously. Finally, he struck out on his own as an entrepreneur to produce the first digital camera sensors, which took years to finally catch on. Now Fossum makes the most of his experience: As a professor at Thayer School, he not only teaches classes on engineering, but also runs the Ph.D. Innovation Program to encourage the next generation of entrepreneurs to learn how to commercialize new technologies.
“There’s a real need for engineers to understand the basics of business and realize it’s not rocket science,” he says, pun intended. And along the way, Fossum and his students are currently working on the next generation of digital camera technology, which could be as revolutionary today as his initial breakthrough was 25 years ago.
Growing up in Connecticut, Fossum thought that starting his own business was the last thing he’d ever do. He watched his father, a mechanical engineer, struggle with his company, which made electromagnetic clutches and brakes. “I’d spend Saturday in his facility installing light fixtures or cleaning toilets or something,” remembers Fossum. “He was always under a lot of stress. It was always financially difficult.”
Fossum himself swore he’d never run his own business. But he was interested in computers and studied computer engineering and physics at Trinity College and Yale, where he earned his doctorate in electrical engineering. Through a doctoral fellowship, he spent summers at Hughes Aircraft Co. in Canoga Park, Calif., working on visual devices to help guide missiles. “The whole process of making these things at the semiconductor level was exciting,” he says. “I realized that was where I wanted to be.” After graduating in 1984, he became an engineering professor at Columbia, where he researched state-of-the-art image sensing technology based on the “charge-coupled device” or CCD. But by 1990 he had gotten the call to work at NASA’s JPL at CalTech and jumped at the chance. “I’m a country person at heart, and didn’t really enjoy living in New York City,” he says. “But the space program was the real lure for moving out west.”
Now instead of missiles, he was making image-sensing technology for spacecraft journeying far into the solar system. Originating from Japan, the current consumer CCD cameras then in use provided good image quality, but they were also power hungry, requiring large batteries to operate. In addition, scientific CCDs were very susceptible to damage caused by radiation. That meant that spacecraft would need to carry large solar panels, as well as metal radiation shields, all of which added excessive weight to precious payloads.
Fossum’s job was to try and shrink the cameras down in size and reduce the overall weight of the spacecraft. To do that, he came up with a new idea. Instead of using CCD technology, he proposed using complementary metal-oxide-semiconductor (CMOS) technology, the standard for constructing integrated circuits on computer chips. Using that technology, combined with a new technique called “active pixel sensing,” Fossum hypothesized that he could dramatically shrink the size of the camera and the power it consumed and make it less susceptible to radiation damage.
He describes the difference this way: Imagine a football field full of people whose job it is to measure rainfall, using buckets that completely cover the surface area. Once people’s buckets get filled with water, they carefully pour them into the buckets of the people next to them, and so on until the water gets to the sideline, where it is measured to get the total amount.
That’s what CCD technology is like. Individual pixels sense the amount and color of light that hits them, and then circuits transfer the information across the chip to where it is recorded. That’s why they required so much energy to operate. “If you imagine that football field and people pouring buckets in a bucket brigade, you can imagine you’d have to pour pretty carefully to get anywhere near what you started with,” says Fossum. “With the CCD technology, you have to transfer the charge packet that represents the accumulated photoelectrons step by step across the chip without losing many electrons or else the image-quality deteriorates rapidly.” The greater the number of pixels, the faster that charge has to travel and the more power that is required to transfer it.
With active pixel technology, however, each pixel could individually sense and record the photons that hit it. It’s as if, says Fossum, all of the people on the football field had their own individual measuring sticks and cell phones and could call in their measurements without having to do the bucket brigade.
Engineers had dismissed CMOS technology, however, because it produced too much noise that drove down image quality compared to the CCD cameras. Fossum and his team at JPL figured out a way to reduce the noise through a process called intrapixel charge transfer, which measured the charge before and after the pixel was hit by photoelectrons. Fossum compares the process to the way that cold cuts are weighed at the deli, first by weighing the wrapping and then weighing the wrapping with the lunchmeat to figure out the total weight of your purchase.
“I spent a lot of time figuring out why nobody had ever thought of this before,” says Fossum, surprised at how easy the ultimate solution proved to be. “I thought there must be something wrong—what am I missing?” Once he had proven to himself that this technique could work, however, he gained confidence in the new technology and began proselytizing to others in the field. At the time, NASA was encouraging engineers to partner with American businesses to commercialize new technology, and Fossum hit the road pitching it to Bell Labs, Kodak, National Semiconductor and other companies, trying to sell them on the idea of “portable videophones” that consumers might carry with them to take pictures and videos.
He got a few takers, but businesses were skeptical of the new technology, which seemed too different from what they were used to. “It seemed there was this window of opportunity to introduce this new technology, and I felt like people were going to miss the boat,” says Fossum. At the same time, he started receiving some custom requests from customers looking for his revolutionary camera-on-a-chip, such as an Israeli medical company that wanted to create a pill camera that could be swallowed by patients.
In 1995, with his then-wife, Sabrina Kemeny, he took it upon himself to found Photobit Corp., continuing his day job at JPL while Kemeny stayed at home and ran the company with a small team of engineers. In short time, it became clear that the company had grown too large for Fossum to work on it only part-time, and he made the decision to leave JPL to commit to it. “There were two paths—one was working as a salary man for someone else, the other was to grow this company,” he says. “I felt the CMOS image sensor was my baby, and I wanted to give it every opportunity to succeed.”
Fossum headed up the engineering side of the company, while Kemeny ran the business side. They planned strategy over the kitchen table. “My kids were always complaining at dinner—would you stop talking about this?” Fossum laughs. Despite his vow as a child never to follow in his father’s footsteps as an entrepreneur, Fossum found himself running a small business after all. Unlike his father, however, Fossum began seeing faster success in his industry. Little by little, Photobit began establishing the technology as a way to create digital cameras that took up a fraction of the space of those then being used.
Although Fossum took out patents on his ideas—eventually he would hold more than 150—other companies began adopting CMOS technology and competing against his upstart firm to produce digital cameras and web cameras for laptops. “The real swing happened when Toshiba started to produce sensors based on our technology,” says Fossum. “That was scary to us, but it gave us instant credibility, so everyone started paying attention and getting interested.”
Photobit eventually grew to a corporation with more than 125 employees and $20 million in annual revenues. With such success, it attracted the attention of other firms, and in 2001 the board decided to sell to semiconductor company Micron Technology. With the advent of cell-phone cameras a few years later, the technology took off. Fossum stayed on as an individual contributor for a year, but the thrill had gone. “Going from being the big cheese to working for somebody else who doesn’t understand the business as well as you do and then feels like they have to establish themselves as the boss is just a rotten situation, and I was glad to exit finally,” says Fossum.
Retired for a few years, he was asked to lead another startup company, called Siimpel, which used microelectromechanical systems (MEMS) technology to produce smaller auto-focus systems for cell-phone cameras. “I got the chance to try out all of the things I’d learned about business,” says Fossum. “It was so much easier the second time around.” Apropos of the new business venture, he says the number one thing he learned is the importance of focus. “It’s important to start with a few strong customers and make a particular product for a particular application rather than trying to make something for everybody,” he says. “They call it the bowling pin strategy. You take out the center pin first, and then it knocks down all of the others.”
Another lesson was driven home to him, appropriately, at an actual bowling alley. “They always tell entrepreneurs how important it is to make your investors and customers happy, but it’s also important to treat your employees well, because they are really the key to your success,” says Fossum. “I remember at Siimpel we held an event for employees and their families at a bowling alley with pizza and everything and my wife, Susan, turned around and whispered in my ear, ‘You better not screw this up.’ And I did my best not to.”
After several years of growing the firm as CEO, Fossum this time left the company before it was sold and became a consultant for Samsung and other companies. By 2010 he was restless again. “I felt I wasn’t done intellectually yet, and maybe I had something more to contribute,” he says. Living in New Hampshire, he contacted Dartmouth, asking if he might teach a course at Thayer. Dean Joseph Helble not only offered him a full slate of classes, but also asked if he would be interested in directing a new innovation program to teach entrepreneurial skills to Ph.D. students.
Fossum says he couldn’t imagine a better match for his skills. “I was thinking of my academic credentials, and I found my entrepreneurial credentials were also appreciated. It was an opportunity to use the entirety of my life experience.” In addition to counseling Ph.D. Innovation Program students on how to turn their engineering ideas into potential businesses, Fossum is pursuing a next-generation image sensor that is as much a step ahead of the CMOS sensor as that one was ahead of the CCD sensor.
“With the CMOS sensor, if you recall, we were last out on the football field with a bucket, but that meant having to measure each bucket accurately,” says Fossum. “But what if we counted every raindrop that came in.” That is the principle behind the so-called quanta image sensor, which would use pixels on the order of 500 nanometers, so small that they could detect individual photons of light. “It would be like using a thimble instead of a bucket,” says Fossum. Each time a photon hits, theoretically, it would knock loose an individual electron that could be counted with high sensitivity. “It is harder to achieve, but it would also be faster because we are just detecting a 1 or 0 for the presence or absence of light. It becomes a binary decision.”
Fossum has set up a lab at Dartmouth to research and design the technology, organizing a half-dozen graduate students into a team resembling a working research group at a company. Assistant Professor of Engineering Kofi Odame is impressed with the focus Fossum’s lab shows. “Academic labs are typically run in order to extend the body of knowledge, and the structure and discipline you’d find in industry is not there,” says Odame, who is collaborating with Fossum to design the integrated circuits that would translate Fossum’s analog signals into digital readouts. “The way he goes about things, he clearly defines who is the client, what are their goals, what are the deliverables, so there is a lot less fuzziness.”
Rather than having students meet individually to discuss progress, Fossum has them come together in weekly meetings in which each student discusses his or her part of the project with the others. That not only keeps everyone apprised of the project as a whole, says doctoral student J.J. Ma, but also holds everyone accountable. “All of the students are working very hard, in part because of our own personalities, but also because of the way Eric manages the team,” says Ma. “It’s a very competitive environment—everyone wants to show they are making progress.”
Along the way, students are getting practical experience in the hard work of turning theory into practice. Last summer, Ma interned at Rambus, a design and development company that is partially funding Fossum’s research and working to commercialize his quanta imaging technology. So far, Rambus imaging division head Michael Ching says that the company has been able to produce a binary pixel architecture on several test chips. “It’s achieved many of the benefits that Eric hypothesized,” he says. “It’s an exciting first step.”
Fossum’s experience running his own companies has made him uniquely qualified to collaborate with business in helping to make his latest ideas a success. “It’s very refreshing,” says Ching. “Oftentimes when you work with people in academia, concepts sounds good but only a small number of them are actually practical even in a 10-year timeframe,” he says. “Eric thinks about these advanced concepts but also understands the practical aspects of these technologies. His approach spans the entire range.”
If that approach is successful, Fossum may within the next decade have another accolade to add to an already spectacular career of innovation—which includes membership in the National Academy of Engineers and a place in the National Inventors Hall of Fame. Whatever happens with his latest research designing the next generation of imaging sensors for consumer devices, however, he still treats his contribution to the current state of the art in imaging sensors with an air of disbelief.
“It’s pretty overwhelming to think about,” he says. Even after all these years, he is amazed at how his idea has taken on a life of its own. “There is a sense of surprise about that—it really works, it does what I thought it was going to do.” From the germ of an idea to make a better camera for spaceships, his technology is now carried around in the pockets of millions of people worldwide. “As an engineer,” he says, “you never know where your inventions are going to end up.”
Categories: Features
give