Developer interview - What can you do with the IR Motion Camera?

Developer interview - Part 2

What can you do with the IR Motion Camera?

Ogasawara-san, I’d like to ask a bit about the hardware side of things now. Can you tell me how the hardware team came to be involved with Nintendo Labo?

Mr. Ogasawara: We started working together pretty much when the software team came and showed us the “Carry-Con” prototype. What got us to that point was, well, I mentioned in the last interview that I was in charge of the IR Motion Camera.

You proposed the idea for the IR Motion Camera to Kawamoto-san, right?

Mr. Ogasawara: That’s right. So over in the hardware department we were thinking of ways that we could put the IR Motion Camera to use. We were actually making prototypes of our own, completely unrelated to Kawamoto-san and Sakaguchi-san’s “prototype party.” This was happening separately and completely within the hardware department.

Another prototype party?!

Mr. Ogasawara: We were calling it “usage research” in my department. Basically we were making a lot of different prototypes, one of which was something we called an “optical attachment.” It looked like this.

Optical Attachment

Mr. Ogasawara: In the back we’ve affixed reflective tape to serve as a marker. The Joy-Con emits a beam of invisible infrared light that is reflected back by these markers, enabling the camera to read their movement. By making it like this we can get data when things move without needing any additional electronic components.

Using the camera to figure out if something is rotating… this is starting to sound familiar!

Mr. Ogasawara: Isn’t it? The first prototype we made with it was this…

Fishing Rod 3D printed sample

The fishing rod!

Mr. Kawamoto: That’s right, but this one operates on completely different principals than the final Toy-Con Fishing Rod.

Mr. Ogasawara: Next, we made this thing. We thought that if we used the optical attachment, we could do all sorts of things with it.

Large Controller

That looks like cardboard.

Mr. Ogasawara: It is. We made a giant controller out of cardboard and called it the “Big-Con.” The way it worked was pretty simple—you could push the big buttons, and that would move a reflective marker up and down inside the box. Then the IR camera would read the motion of the markers.

This is the first prototype produced with an eye on the building process. I’ve always wanted to show the customer how fun hardware can be. The team got together and thought about how to do that, and we realized that we wanted to actually show the customer how interesting it is to make hardware. We started to think that cardboard was an ideal material for an idea like this.

You mean that you were all arriving at these ideas independently of one another?

Mr. Ogasawara: Yeah, we didn’t find out until later, but as it turns out we were all thinking about the same things.

Sakaguchi-san was thinking of “bundles of sensors” while Ogasawara-san was thinking about “optical attachments”? That’s amazing. How did you end up working together?

Mr. Sakaguchi: I was looking for people who were willing to help us with our hardware prototyping, and was introduced to Ogasawara-san. One of the first things we talked about was the Nose Picker-Con.

This was during the prototype party, right?

Mr. Ogasawara: Yes. And like I said, my team was already working on prototypes, so we decided to join forces and assist Sakaguchi-san’s team. That was when they showed me their robot prototype, and I was totally blown away. The way they’d used our optical attachment was ingenious, and the software they built around it was really neat.

It’s like you were destined to work together!

Mr. Ogasawara: It really does feel that way! It was a great coincidence that we were both working independently on similar prototypes, but truthfully the prototypes my team had been working on didn’t have the potential to actually become products on their own. But because I was able to present the IR Motion Camera and the optical attachment that makes use of it to Sakaguchi-san at just the right time, I feel like I was able to make a genuine contribution to Nintendo Labo—I’m very grateful for the way things worked out.

Mr. Kawamoto: I can explain it with a cooking metaphor. Ogasawara-san’s team found new ingredients to work with, and Sakaguchi-san’s team prepared them and plated them up. Together they were able to make a new menu, with dishes no one has ever seen before.