Jean Mary Zarate: 00:05
Hello, and welcome to Tales From the Synapse, a podcast brought to you by Nature Careers in partnership with Nature Neuroscience. I’m Jean Mary Zarate, a senior editor at the journal Nature Neuroscience, and in this series we speak to brain scientists all more than the planet about their life, their investigation, their collaborations, and the influence of their function.
In episode 5, we meet a researcher devoted to understanding the complexities of vision and how to bring eyesight back to the blind.
Pieter Roelfsema: 00:40
So my name is Pieter Roelfsema. I’m the director of the Netherlands Institute for Neuroscience in Amsterdam. And I’m also a professor at the Amsterdam University Health-related Hospital, and at the Absolutely free University of Amsterdam.
I studied the visual brain considering the fact that a lot of years, and I constantly wanted to know what takes place if you truly get started to fully grasp what you see.
And that is far from trivial, simply because in fact, what you see is just a lot of things, a lot of smaller image components that fall on your retina.
But then you have to sort of group that with each other into a representation of objects and the various objects that might surround you. And that is a pretty fascinating topic.
So I studied that for, for a lot of years. And in the final couple of years, we identified ourselves implanting an rising quantity of electrodes (and electrodes is just a different word for wires) into the brain of experimental animals.
And at some point, we realized we are at such a higher quantity, hundreds, let’s ramp it up a small bit.
So we went to 1000, with the concept of stimulating these electrodes and then generating artificial vision.
Pieter Roelfsema: 01:56
So I assume we have various ambitions. 1st, is truly to fully grasp how vision operates. And a single of the subjects that I’m especially interested in is consideration.
So of all the items that surround you, you can spend consideration, choose out a single of these things, what takes place in the brain when you direct your consideration to one thing.
And a different issue is what takes place if one thing enters into your conscious awareness? So what are pretty fascinating inquiries, so some items might get into your awareness and some, some do not. So we are also seeing that.
And a single of the far more applied ambitions of the lab is to develop a visual prosthesis, a visual brain prosthesis, so persons who lost the function in their eyes, the concept will be just to skip these malfunctioning eyes.
And to plug in the visual information and facts from a camera. It really is a single of the central centres, so centres for vision in the brain.
So we strategy that from various angles. So a single is to do modelling. Second is to study vision in humans, just possessing them respond with a button press, understanding what they can and can’t see.
But we also appear at the brain mechanisms in experimental animals. So we appear in mice, and we also appear in monkeys.
Pieter Roelfsema: 03:20
So I began to come to be interested in the brain right after reading a book by Doug Hofstadter. It really is, I assume it is a popular book. It really is named Gödel, Escher, Bach, and was pretty popular when I was beginning to study in the ‘80s.
It was a present from my father. And when I study it, I knew what I wanted to do, I wanted to study consciousness. So then, I in fact began to do some projects just in my hobby time, initial on snails, then on rodents. And at some point, I realized I want to study this in an animal that is closer to humans. And I will truly fully grasp what takes place in the brain when we come to be consciously conscious of one thing.
So we, in my lab, studying cognition and the part of consideration and how it is modulating the activity in the visual brain, we began to implant rising numbers of electrodes. So electrodes are just wires.
And so, at some point, we reached a regular preparation exactly where we implant, say, 200 electrodes, and then we believed, you know, we could multiply this with a smaller aspect. And then we know that from earlier function, that if you stimulate a single electrode electrically, you are artificially activating these brain cells close to the tip of the electrode. And a particular person or an experimental animal (it can even be a particular person who has been blind for far more than ten years), they will see a dot of light. And that is with only a single electrode.
So if you have one hundred or 1000 electrodes, you can develop 1000 of these dots of lights, simply because phosphines. And considering the fact that the area, the visual cortex, exactly where we implant these electrodes, have a map of space, exactly where you stimulate in a map the topic sees at the very same place, the outdoors visual planet, this dot of light.
So if you have 1000, you can fundamentally function with them like a matrix board, they can know from the stadium or from the highway. So if you of course, if you flash up a single bulb, the particular person is going to see, properly, a dot of light. But you can develop patterns. And that is what we set out to do.
So we’re fundamentally writing to this matrix board that is in the brain, and, and see no matter if the animals are capable to recognize them. These, these items that we create as patterns. And we identified certainly that this is the case.
So we have been capable to create, for instance, we educated, we did this in monkeys. We educated them to recognize letters.
And so they knew that if they saw letter A they would have to make an eye movement to the above. If they saw the letter B, to the left, and so on and so forth.
And at some point we educated them visually so these animals have been not blind, they could see.
At some point, we took the visual stimulus away, and we just wrote straight letter A to the brain.
And we have been pretty excited to come across that they have been certainly producing the very same response, as when we would have presented the very same letter visually. We published that in 2020. So a single-and-a-half years ago.
So in our group, we want a lot of unique experience. And some of these varieties of experience are inside our personal group. So we are knowledgeable about how to place wires, electrodes in the brain.
But we also have a lot of collaborations with authorities about the planet, persons who truly know about how to make these electrodes so that they do not harm the brain tissue as well a great deal.
We function with persons in artificial intelligence who aid us to take our camera, to take camera photos and translate them into brain stimulation patterns.
We also collaborate with neurosurgeons who can inform us how to truly make this device and make it one thing that is going to be feasible for a neurosurgeon to truly implant in the brain simply because that is undoubtedly a pretty essential purpose for me, to bring this to a patient.
So the visual procedures will be composed of various elements. The initial is a camera. You can use the camera that you obtain. So there are now various corporations that make these glasses that include a camera.
And these camera photos are sent to a smaller pc. It can be the size of a telephone. And this will take in the camera image and develop from it a pattern to be a post on the matrix boards in the brain.
Then proper now, we nonetheless have truly a physical connection in between a connector that is implanted on the skull of the topic. It can be a monkey. It can be a human. We would like to make this wireless, so it will be a wireless interconnects with a brain chip.
And then from the brain chip, there will be various wires operating into the brain. So these are the ones we contact electrodes. And so, primarily based on the image that the camera captures, there is this brain stimulation pattern.
And that then provides a rudimentary type of vision. So you are not going to see complete colour, complete depth as regular vision would give you. It is going to be pretty rudimentary, like you are walking about with this substantial matrix board in front of you, proper? So it is, it is undoubtedly not going to be excellent, but it is most likely also going to be a great deal far better than practically nothing.
Yes, eyesight operates. It begins, of course, all in the retina, that is at the back of the eye, which is a pretty, pretty sophisticated device.
So there are huge groups of researchers that are studying the retina. And then from there, the information and facts is transported to the brain by way of the optic nerve. And then it begins in the cortex in the initial area, key visual cortex.
And there are cells, brain cells, neurons, that are chosen for relatively easy characteristics of the outdoors planet, say the place and the orientation of an edge of light, no matter if that is a vertical edge or a horizontal edge.
And so they truly do a pretty nearby processing. So you have a lot of of these processors and in parallel. So a single would be straight ahead, a single would be just adjacent, a single would be in the upper left corner for each place in the outdoors visual planet, there is a set of neurons that just care about what is going on there.
And then if you go to greater regions, then this information and facts of these person detectors is combined in far more and far more sophisticated strategies. So fundamentally, what takes place there is that you go from pixels, to ideas. And there are now a lot of persons modelling this.
So there also has been, of course, an artificial intelligence revolution that helped us fully grasp how to go from pixels to ideas. And what these persons in artificial intelligence come across and how they model this course of action in fact, is rather a great approximation of what is going on in the human brain.
So also, in the human brain you have all these stages that are involved in this translation from pixels into ideas.
Now, seeing what the notion in front of you is, no matter if it is a bicycle or a chair, that is only a single of the functions of vision, it is not the only a single. You also can steer your motor behaviour. And there are other brain locations that are involved in that. So they in fact localize the edges. So if you want to choose one thing up, you want to know exactly where your fingers are going to touch the object that you want to choose up.
And you want to know exactly where it is, you want to know how to position all your joints, all your joints. And so all these transformations, they are also in element informed by vision. So that is a different pretty essential part for vision to play.
Pieter Roelfsema: 11:42
So a single issue that is, I assume, fascinating in this domain is the possibility now to sometimes record also neurons from human individuals, and some researchers in my lab are carrying out this.
So these are individuals who have serious types of epilepsy. And the neurologist does not come across the proper cocktail to suppress these epileptic attacks. So then the neurosurgeon comes into the play, and in some circumstances it is apparent what is the problematic area of the brain, but there are some occasional circumstances exactly where the neurosurgeon is not one hundred% confident.
And then these individuals get the set of electrodes, a set of wires in their brain, for about two weeks. And we have ethical approval then to attach to these clinical electrodes, pretty tiny wires.
And by way of these, we can record single neurons. So that was a strategy that was created by Itzhak Fried various years ago. That provides you the exclusive chance to also record from brain cells that are tuned to precise folks.
So if you record from these single neurons, you can also do awesome items. So other persons, but also in our lab, occasionally you can, for instance, make associations in between stimuli.
So suppose that, whilst you associate a popular particular person, say Jennifer Aniston, with a different popular particular person, say, Barack Obama, then we demonstrated that if persons recall these associations…so you give them a image of Jennifer Aniston, and you ask them to recall what was related with them, then you have some neurons that only respond to Barack Obama. And then they will come to be active the moment you speak about Jennifer Aniston, and ask them to recall this association.
So these items I’m also pretty excited about. Due to the fact the neurons that code these ideas are generally also straight away the ideas that are in fact in your consciousness, these are the items you are pondering about.
So that provides you a pretty close hyperlink to what is truly on the subject’s thoughts. And what you can see in the activity of neurons, which I come across fascinating.
Yeah, so we are not recreating the eye. So we’re just skipping it. So I assume that is also why the vision that we’re going to generate is, it is just a great deal significantly less high-quality than the regular vision.
Due to the fact we are implanting electrodes in the person brain, and if we stimulate these, we activate a set of neurons that would commonly under no circumstances be activated in that constellation, That provides you just a dot of lights.
And it does not give us the possibility to develop unique colours, for instance, simply because neurons that are chosen for unique colours are intermingled, and you can’t just selectively only activate the green cells, or the blue cells, or red cells. So that is why it is somewhat rudimentary.
But the challenges if you sort of comprehend that you are under no circumstances going to be as great as regular vision, then are to get a great coverage of the visual fields simply because of the nature of the map of the outdoors planet in the brain.
You have to comprehend that the key visual cortex, which is the initial area exactly where the information and facts comes from, the visual information and facts that is processed in the cortex is substantial. It has a surface location of 25 square centimetres on the left, and a different 25 square centimetres on the proper.
And to get wires everywhere in that area, which is also rather folded, is going to be difficult. So that is a single of the significant challenges that we’re pondering about, how to make confident that we cover the map with electrodes.
If you only cover a smaller element of the map with electrodes, then the topic is only going to be capable to see in a smaller area of the visual field, then they will be blind at all other areas. That is rather undesirable.
Yet another significant challenge is to make an interface with electronics in the brain that has a enough longevity.
So we are now at the moment making use of so-named Juta electrode arrays. So these are arrays of stiff silicone shanks, we contact them. So like, like a bed of nails, is fundamentally what it appears like.
And we know that they function, generally for a year, possibly a small bit longer. But you know, you do not want to implant a patient with a prosthetic device to come across out that right after a single or two years these electrodes are encapsulated by glial cells. So fundamentally the fibrosis, fibrous tissue that encapsulates the electrodes, and you shed the get in touch with with the nerve cells. So in that case, you can’t effectively stimulate any longer.
So that is a different challenge. We have to come across electronic components that have enough longevity. So if you implant them currently, they will nonetheless be functioning, say, in 5 years, or ten years, or even 15 years. I assume these are two main challenges.
Pieter Roelfsema: 16:58
So I get the occasional request. And I have to clarify to these persons who get in touch with me, this is not a clinically authorized device. So it is investigation. And our ambition will be to go to humans in the subsequent say, two years, or possibly a small bit later.
But in that case, it is nonetheless going to be investigation. So do not count on from us in the coming 5 years a therapy. It is just investigation. And of course, the investigation is pretty essential simply because it is going to aid us make the subsequent step, and go towards a device that is clinically authorized.
Ahead of we are there there are all types of regulations, which are there for a great purpose. And we have to show that we comply with all these regulations.
With the technologies we’re making use of now, it is constantly going to be rudimentary. But I would be pretty excited if you are capable to develop a prosthesis that has, say 1000, or even ten,000, or even 50,000 pixels, ought to comprehend that your eye has 1 million pixels.
So if you count the quantity of fibres in the optic nerves, it is about 1 million. So 50,000 is what we may possibly aim for at some point. Sounds ambitious, but it is only five% of the regular, of the regular eye. And it is, that is going to be difficult, but if I would appear back on my profession, and we would have been capable to develop a device that has 50,000 pixels, and various persons are making use of it and it is catching up, I will be tremendously happy about it.
Jean Mary Zarate: 18:51
Now that is it for this episode of Tales From the Synapse. I’m Jean Mary Zarate, a senior editor at Nature Neuroscience. The producer was Don Byrne. Thanks once more to Professor Pieter Roelfsema. And thank you for listening.
