In 2016, at the Frontier Conference of the White House held at the University of Pittsburgh, the then-U.S. President Barack Obama had a fist with Nathan Copeland, a robot arm user, while visiting innovative projects. A mechanical arm that completes direct tactile feedback via a brain-computer interface can allow paralyzed patients to complete the action of pouring water from one cup into another more quickly and naturally. On May 21st, a research team from the University of Pittsburgh in the United States published a study “A brain-computer interface that evokes tactile sensations improves robotic arm control” in the top international academic journal “Science”. Research says that when a person controls an object with his mind, the robotic arm can provide direct tactile feedback to the person’s brain. In the past, robotic arms could only be guided by vision. The team has been working with Nathan Copland. Fifteen years ago, the teenage Copeland was paralyzed in an accident. He has now learned to control the movement of a robotic arm through a brain-computer interface. Copeland said, “When I only have visual feedback, I can only see that the hand touches the object. If I use it to pick up things, sometimes things will fall off.” Copeland needs to complete a typical grasping task. About 20 seconds. “With sensory feedback, he can do it in 10 seconds,” said Jennifer Collinger, an associate professor in the Department of Physical Medicine and Rehabilitation at the University of Pittsburgh. (Note that the colon is only used in very formal or important occasions) Collinger said that tactile information is very important for the use of prosthetic robotic arms, “because it is difficult for you to grasp an object that you can’t feel.” Even for simple things, such as picking up a cup and trying to move it. Maintain proper pressure during the process, which depends to a large extent on the tactile feedback of the hand. Therefore, Klinger and a team of researchers spent years looking for ways to add sensory feedback to robotic arms and hands. The research team used a two-way brain-computer interface to record neural activity in the motor cortex and generate tactile sensations through micro-stimulation of the somatosensory cortex in the cortex. In the experiment, the researcher first placed electrodes in the area of ​​the Copland brain that processes sensory information, so that electrical impulses can be used to simulate a series of sensations. Collinger said, “It turns out that the sensation produced by stimulating the fingertip-related areas of the brain is like coming from one’s own hands.” Next, the University of Pittsburgh team studied how to generate these signals when the robotic arm is in contact with an object. The last step is to time when Copland completes some tasks, such as picking up a stone or pouring water, how much time it takes if there is tactile feedback. The results show that Copland completes some manual tasks at roughly the same speed as humans use their own hands. Copeland revealed, “The intensity of this sensation actually varies according to the amount of force the hand exerts on the object. So I can also tell if I have grasped it. There is an additional benefit, after the increased tactile sensation , The feeling of using the robot arm is more natural. This kind of control is very intuitive, so that I basically just think about things, but it seems to be moving my own arm.” Jeremy D. Brown, assistant professor of John C. Malone in the Department of Mechanical Engineering at Johns Hopkins University, said that the significance of the research results goes far beyond the robotic arm. “High-tech prostheses also work better when simulating the sense of touch. Some are achieved through vibration or other forms of tactile feedback. This is the same way that many smart phones help users type on the screen.” The latest prostheses operate just like our natural limbs. Their elbows can be bent, their wrists can be rotated, and their fingers can be grasped. But most sensors still only have basic capabilities, such as detecting resistance or temperature. Before they have direct tactile feedback, they are actually very clumsy. And when using my hand to touch the surrounding objects, as Brown said: “I can feel pressure, feel sliding, feel whether the object is wet or dry. I can feel its texture, I know it is rough Still smooth.” Scientists are just beginning to learn how to make artificial hands and fingers that can detect these subtle features of objects. Brown said that as prosthetics or robotic arms provide more sensory feedback, they will become more useful. “The sense of touch is not just for flexibility. It’s not just the ability to reach into your pocket for the key. It can also hold your lover’s hand and feel the emotional connection.”

An old and somewhat scary warehouse in the suburbs of Fitzroy in Melbourne (Germany) has been transformed into a fairy-tale house by EAT architects, headed by Eid Goh and Albert Mo. worker. The architect kept the outer surface of the warehouse, which was covered with vines interlaced on the walls as well as layers of dense trees in the yard so that occupants could feel one with nature. . The interior of the warehouse is redesigned in a modern style that the architect says: “The architecture is influenced by Venetian and Brazilian modernism anecdotes.” Accordingly, the main circulation space in the warehouse is divided into walkways leading to the inner courtyard, where there are dense vegetation and vines. The rust layer on the wall is also kept by the architect to create an old feeling for the warehouse. The ground floor of the warehouse is designed as a bar, where people can focus on eating and reading while the upper floor is the children’s bedroom area. “Old meets new everywhere, but the general feeling is that this is a home that has always been there. Tactile materials, natural colors and surfaces come together in a whole that feels comfortable and pleasant to the touch,” expressed the architect. A concrete core pillar in the center of the warehouse is fitted with a wooden staircase, leading to the laundry, bathroom and storage areas. The architect said that the usable details in the warehouse will be used to the fullest, combined with the interior with similar colors to create a unified whole but no less comfortable and new.

Nathan Copeland controls a robotic arm using electrodes implanted in his brain. Published in the journal Science, the team say their work demonstrates that adding sensation to the technology significantly improves the function of prostheses for quadriplegics, compared with based solely on visual cues. Nathan Copeland, 34, told AFP: “I am the first person in the world to have a device implanted in the sensory cortex that scientists can use to stimulate my brain directly. And then, I felt like I had a real feel in my flesh and blood hand.” In 2004, Copeland was involved in a car accident that left him with a severe spinal cord injury and quadriplegia. He volunteered for scientific research and six years ago he underwent major surgery to implant tiny electrodes in his brain. Two sets of 88 electrodes the width of a human hair are arranged like tiny combs and penetrate deep into the cortex, which controls motor function. According to Associate Professor Rob Gaunt of the Department of Physical Medicine and Rehabilitation at the University of Pittsburgh, co-leader of the study, fewer than 30 people in the world have received such a transplant. What’s unique about Copeland’s case is that an additional set of electrodes is connected to his dorsal cortex, which receives and processes sensations. The idea of ​​sending haptic feedback to the tactile sensory cortex dates back decades, but doing so in a controlled and understandable way by brain circuitry used to be a major challenge. After Copeland underwent surgery to install the electrodes, the team was truly thrilled. Then came the decisive moment, when they tried sending the first tactile signal. “It felt really fuzzy,” Copeland recalls. He asked them to try again to make sure it was real. Before the interface could work with the robotic arm, the scientists had to perform a series of tests on Copeland. First, they needed to learn which electrodes cause what sensation when activated and which fingers are associated with them in order to set up the robotic hand correctly. They also made him watch a video of the robotic arm moving left or right and recorded the electrodes lighting up when he was asked to “think” it was him controlling it. Copeland sat next to a black metal robotic arm and was asked to pick up a series of small objects such as rocks and spheres and place them in a box – when the tactile sensors were turned on or off. He can complete each task in an average of twice as fast when the sensors are activated, and can even perform more complex tasks like picking up a glass and pouring its contents into a glass. another cup. The team wanted to further refine the prosthetics because they didn’t want to just do science experiments in the lab, but wanted to actually make devices that would be useful to people. Copeland has set up his brain-computer interface at home when the Covid-19 pandemic shut down universities and has used his downtime to learn how to draw on a tablet and even play video games. death. He does this by using his mind to send signals directly to the computer, instead of using his arms to press buttons.