Pranav Mistry sixth sense what happened

Pattie Maes + Pranav Mistry: Meet the SixthSense interaction

ABOUT THE SPEAKERS
Pattie Maes - Researcher
As head of the MIT Media Lab's Fluid Interfaces Group, Pattie Maes researches the tools we use to work with information and connect with one another.

Why you should listen

Pattie Maes was the key architect behind what was once called "collaborative filtering" and has become a key to Web 2.0: the immense engine of recommendations - or "things like this" - fueled by other users. In the 1990s, Maes' Software Agents program at MIT created Firefly, a technology (and then a startup sold to Microsoft) that let users choose songs they liked, and find similar songs they'd never heard of, by taking cues from others with similar button. This brought a sea change in the way we interact with software, with culture and with one another.

Now Maes is working on a similarly boundary-breaking initiative. She founded Fluid Interfaces Group, also part of the MIT Media Lab, to rethink the ways in which humans and computers interact, partially by redefining both human and computer. In Maes' world (and really, in all of ours), the computer is no longer a distinct object, but a source of intelligence that's embedded in our environment. By outfitting ourselves with digital accessories, we can continually learn from (and teach) our surroundings. The uses of this tech - from healthcare to home furnishings, warfare to supermarkets - are powerful and increasingly real.

More profile about the speaker
Pattie Maes | Speaker | TED.com
Pranav Mistry - Director of research, Samsung Research America
As an MIT grad student, Pranav Mistry invented SixthSense, a wearable device that enables new interactions between the real world and the world of data.

Why you should listen

When Pranav Mistry was a PhD student in the Fluid Interfaces Group at MIT's Media Lab, he worked with lab director Pattie Maes to create some of the most entertaining and thought-provoking interfaces the world had ever seen. And not just computer interfaces, mind you - these are ways to help the digital and the actual worlds interface. Imagine: intelligent sticky notes, Quickies, that can be searched and can send reminders; a pen that draws in 3D; and TaPuMa, a tangible public map that can act as Google of physical world. And of course the legendary Sixth Sense, which is now openly sourced

Before his studies at MIT, he worked with Microsoft as a UX researcher; he's a graduate of IIT. Now, as director of research at Samsung Research America, Mistry heads the Think Tank Team, an interdisciplinary group of researchers that hunts for new ways to mix digital informational with real-world interactions. As an example, Mistry launched the company's smartwatch, the Galaxy Gear, in 2013.

More profile about the speaker
Pranav Mistry | Speaker | TED.com

RECOMMENDED CONTETENTS

That demonstration - a product from Pattie Maes' lab at MIT, led by Pranav Mistry - was THE talking point at TED. A device with a projector that is worn on the body and paves the way for comprehensive interaction with our environment. Think of "Minority Report" and a lot more.
Pattie Maes - Researcher
As head of the MIT Media Lab's Fluid Interfaces Group, Pattie Maes researches the tools we use to work with information and connect with one another. Full bioPranav Mistry - Director of research, Samsung Research America
As an MIT grad student, Pranav Mistry invented SixthSense, a wearable device that enables new interactions between the real world and the world of data. Full bio

Double-click the English transcript below to play the video.

I've been by this
I have long been fascinated by the question
of we could or a -
whether we can develop or create a sixth sense -
a that would give us
a sense that gives us a seamless
and to
and provides easy access to meta-information,
or that
or to information that may be available somewhere,
that be to help us make the right
and who can help us
about it is that we're.
making right decisions whatever we encounter.
And some of you,
And some of you may say:
well, don't do that?
Well, aren't today's cell phones already doing that?
But I would say no.
When you here at -
When you meet someone at TED -
and this is the, of, of the -
and this is probably the main networking place of the year -
you don't
You're not shaking hands with anyone
and then say, "Can you on for a
and then say "One moment please!
while I take out my and you? "
I'll take out my phone and google you. "
Or when you go to the
Or when you go to the supermarket
and you're there in that
and stand in the big corridor,
of of,
with different types of toilet paper,
you don't take out your, and a,
do not take your phone out to open a browser
and go to a to try to
and go to a website so that you can decide
of these
which of these different types of toilet paper
is the to make.
would be the most ecologically sound purchase?
So we don't really have
Well, we actually don't have easy access
to all this
to all this relevant information,
that can just help us make
who might be able to help us with this
about what to do and what to take.
To make optimal decisions about our further course of action.
And so my at the
And that's why my working group at the Media Lab has been involved for some time
has been a of
with the development of a number of inventions
to give us to this
to give us this information
in a way,
to make accessible in a simple way,
that the any of.
without the requirement of behavior modification on the part of the user.
And I'm here to
So I'm here revealing
our,
our youngest and so far
and so,
is still very much a work in.
which is in the middle of development.
I'm the right now
Actually, I am currently wearing the device on my body
and of it
and we patched it up somehow
with that are off the -
from regular bulk goods -
and that, by the way, only 350
which, by the way, only cost $ 350
at this in time.
I'm a, just a,
I carry a camera, so a simple webcam,
a, with a little.
a portable battery operated projector with a small mirror.
These to my in my
These components transmit information to my cell phone, which is in my pocket,
as the and.
and that functions as a transmission as well as a computation device.
And in the here we see my,
And in the video here we see my student, Pranav Mistry,
really the been
who is actually the brilliant head who is responsible for the implementation
and this.
and is responsible for developing this whole system.
And we see how this
And we see how this system
him up to any
enables him to approach any surface
and his to with the
and start using his hands to deal with the
that is in of him.
to interact with information projected in front of it.
The the.
The system follows the four important fingers.
In this, he's
In this case he was wearing a simple marker cap,
that you.
that you may already recognize.
But if you want a more
But if you want an even more stylish version,
you could your in.
you can also paint your fingernails in different colors.
And the these
And the camera basically follows those four fingers
and any that he's
and recognizes all of their gestures.
so he can just go to, for, a of Long,
So he can, for example, go to the map of Long Beach,
in and out,.
zoom in or out, etc.
The
The system also recognizes the pictorial gestures
as the "take a",
such as the "take picture" gesture,
and then takes a of is in of you.
and it photographs whatever is in front of you.
And when he then back to the,
And when he returns to the Media Lab, ¥
he can just go up to any
he can just walk towards any wall
and all the that he's,
and project all photos taken by him,
them and them,
they sort and organize,
and them,.,
again all.
and again using normal gestures.
So, some of you were here two
Some of you were very likely here the year before last
and saw the by
and saw the demonstration by Jeff Han.
or some of you think, "Well, doesn't this look like the?"
And you might be thinking, "Well, doesn't that look like the Microsoft Surface Table?"
And yes, you,
And yes, you also interact using normal gestures,
, .
But the here is that you can use any,
But the difference here is that you can use any area
you can to up to any,
can approach any surface,
your if nothing is
including your hands if nothing else is available,
and with this.
and interact with this projected data.
The is,
The device is completely portable,
So one is that it's.
So an important difference is that it is completely mobile.
even more is that in
An even more important difference is that in mass production
this would not more than
in the future it will not cost more than today's cell phones
and would not of be a -
and would not have such a large packaging -
could look a lot more
could look a lot more stylish
than this that I'm around my.
than this version that I'm wearing around my neck.
But other than some of you out yours
But besides being able to let some of you act out your imagination,
of looking as as in ","
that you look as cool as Tom Cruise in "Minority Report"
the why we're really about this
we are really excited about this device,
is that it really can as one of these
because it can actually work like one of those "sixth sense" devices,
that you
giving you the relevant information
about is in of you.
about anything that stands in front of you.
So we see here going into the
So we are now looking at Pranav when he visits the supermarket
and he's for some.
and he buys some paper towels.
And, as he up a the can
And when he lifts a product, the system can
the that he's up,
the lifted product through image recognition or
or,
Recognize marker technology
and give him the or an.
and give it green or red light.
He can for.
He can also request further information.
So this here
is a good, his.
is a particularly good choice according to his personal criteria.
Some of you want the with the in it
Some of you may be more likely to use toilet paper with most bleach
than the.
Prefer the most environmentally sound option.
If he up a in the,
When he looks at a book in a bookstore
he can get an -
can he get an Amazon rating.
it right on the of the.
It will be projected onto the book cover.
This is, our,
The author of the book is Juan, our previous speaker.
a great, by the way, at.
Incidentally, it has received excellent ratings from Amazon.
And so, the of the
And now Pranav turns the page
and can then see about the -
and can see more information about the book -
, maybe of by his,.
Comments from readers, perhaps some information from his favorite critic, etc.
If he to a
When he turns to a certain page,
he an by maybe an of a of
may he find a note from one of our friends who is an expert
that him a little of
and thus receives additional information
about is on that.
to any content on this page.
the -
While reading a newspaper -
it never has to be.
There are no longer any old camels.
You can get of the that you're about
You can get video comments about the event you are reading.
You can get the.
You can get information about the latest scores, etc.
This is a more one.
This is rather controversial.
As you with at,
When you interact with someone here at TED,
maybe you can see a of the,
maybe you see a cloud of catchphrases
the that are with that
i.e. the terms associated with this person,
in and.
that are on their blog or personal website.
In this, the is in,.
In this case the student is interested in cameras, etc.
On your way to the,
On the way to the airport,
if you up your, it can tell you that your is,
when you collect your boarding pass, it can inform you if there is a flight delay,
that the has,.
whether you have to go to the other gate, etc.
And, if you need to know what the time is
And when you need the time
it's as as a watch -
just draw a clock -
So that's where we're at so
So that's where we are so far
in this
with the development of this sixth sense,
that would give us to all this
who can grant us seamless access to all this relevant information
about the things that we come.
to things we encounter.
My, really, like I said this.
My student Pranav, who, as I have said, is the brilliant mind behind this idea.
() ()
He does a lot of
He really deserves a lot of applause
because I don't think he's much in the last,.
because he hasn't slept much in the last three months I guess.
And his is not very about him.
And his girlfriend probably doesn't enjoy him very much either.
But it's not, it's very much a work in.
But it's not perfect yet, and it's still developing.
And who, maybe in 10
And who knows, maybe in another 10 years
be here with the.
let's invent the ultimate brain implant for the sixth sense.

RECOMMENDED CONTETENTS

▲ Back to top

ABOUT THE SPEAKERS
Pattie Maes - Researcher
As head of the MIT Media Lab's Fluid Interfaces Group, Pattie Maes researches the tools we use to work with information and connect with one another.

Why you should listen

Pattie Maes was the key architect behind what was once called "collaborative filtering" and has become a key to Web 2.0: the immense engine of recommendations - or "things like this" - fueled by other users. In the 1990s, Maes' Software Agents program at MIT created Firefly, a technology (and then a startup sold to Microsoft) that let users choose songs they liked, and find similar songs they'd never heard of, by taking cues from others with similar button. This brought a sea change in the way we interact with software, with culture and with one another.

Now Maes is working on a similarly boundary-breaking initiative. She founded Fluid Interfaces Group, also part of the MIT Media Lab, to rethink the ways in which humans and computers interact, partially by redefining both human and computer. In Maes' world (and really, in all of ours), the computer is no longer a distinct object, but a source of intelligence that's embedded in our environment. By outfitting ourselves with digital accessories, we can continually learn from (and teach) our surroundings. The uses of this tech - from healthcare to home furnishings, warfare to supermarkets - are powerful and increasingly real.

More profile about the speaker
Pattie Maes | Speaker | TED.com
Pranav Mistry - Director of research, Samsung Research America
As an MIT grad student, Pranav Mistry invented SixthSense, a wearable device that enables new interactions between the real world and the world of data.

Why you should listen

When Pranav Mistry was a PhD student in the Fluid Interfaces Group at MIT's Media Lab, he worked with lab director Pattie Maes to create some of the most entertaining and thought-provoking interfaces the world had ever seen. And not just computer interfaces, mind you - these are ways to help the digital and the actual worlds interface. Imagine: intelligent sticky notes, Quickies, that can be searched and can send reminders; a pen that draws in 3D; and TaPuMa, a tangible public map that can act as Google of physical world. And of course the legendary Sixth Sense, which is now openly sourced

Before his studies at MIT, he worked with Microsoft as a UX researcher; he's a graduate of IIT. Now, as director of research at Samsung Research America, Mistry heads the Think Tank Team, an interdisciplinary group of researchers that hunts for new ways to mix digital informational with real-world interactions. As an example, Mistry launched the company's smartwatch, the Galaxy Gear, in 2013.

More profile about the speaker
Pranav Mistry | Speaker | TED.com

Data provided by TED.

We use Font Awesome icons and the license is here.

"English-Video.net" was created in May 2015.

Last updated: 2020/01/12. With the termination of the "TED API", this site will no longer be updated.

We are currently in the process of creating a new site called "eng.lish.video", so please visit it.

If you have any questions or suggestions, please feel free to write comments in your language on the contact form.

Privacy Policy

Developer's blog