Just as no one could’ve predicted the rise of social media in the last decade, no one today can predict precisely how driverless cars will look, operate, or shape our lives. All we know is that autonomous vehicles are coming. But they’re coming slower than most people think.
For every solution solved on the long road to ubiquitous self-driving cars, there are dozens of questions putting on the brakes. “It sounds like it’s going to be this fantastic new world, but then you start picking apart all the different subtle questions in between,” says Michael Clamann, senior human factors engineer and autonomous vehicle expert with the UNC Highway Safety Research Center (HSRC).
Those questions go beyond software and hardware. Making autonomous vehicles safe requires perfect alignment of computer science, engineering, psychology, sociology, and policy. The scope of societal impact goes further, reaching into a world that we can’t yet imagine.
“We’re starting to recognize that technologies are complicated and thinking about the pluses and minuses is important,” says Noreen McDonald, chair of the Department of City and Regional Planning. “It’s going to be a long time before our cities are a bunch of people driving around in autonomous vehicles.”
At UNC, researchers from a variety of disciplines are thinking through challenges from pedestrian safety to city parking, bringing us closer to this “new world” one day at a time.
Building the brain
Autonomous vehicles use a variety of sensors to identify pedestrians and navigate safely. But going a step further and predicting their walking paths could reduce pedestrian accidents and fatalities even more, especially in urban areas. UNC computer scientist Aniket Bera and his team have developed one of the leading algorithms in the world for doing just that.
“For autonomous vehicles, the biggest problems is how well they can learn and understand the surroundings,” Bera says. “If it can learn the surroundings, it can drive efficiently and safely.”
Computer scientists have to train autonomous vehicles to identify individuals and groups as they stroll along crowded sidewalks using data-driven artificial intelligence and mathematical models.
“People tend to train models based on just data, but what we have done is try to make a combination of data and physically-based simulation,” Bera says. This combination of psychology, artificial intelligence, simulation models, and mathematics has evolved through three stages over the years: pedestrian tracking, prediction, and behavior modeling.
The first stage, pedestrian tracking, teaches the “brain” of the autonomous vehicle the basics such as what a person looks like in a crowd where only a hand or shoulder may be visible in the camera sensors’ live video feed. Once the computer learns that, it can predict where a pedestrian will go next. That’s where pedestrian physics come into play.
Bera and his colleagues discovered that as a crowd gets denser, its movement becomes similar to that of fluids. Using a combination of fluid dynamics and collision avoidance models, Bera’s model can accurately predict where a pedestrian will walk several seconds into the future, with accuracy not dropping below 80 percent until after seven seconds. The system retrains itself with every new video frame, comparing the difference between its prediction and reality in real time.
Bera teamed up with the psychology department to take the algorithm yet another step forward to include behavior modeling. A data-driven process, thousands of simulations train the artificial intelligence to assign each pedestrian a combination of personality traits and behaviors, increasing the prediction’s accuracy and influencing navigation decisions. For example, children typically walk more aggressively and are more likely to dart out into the road, so the car needs to slow down or stop in response.
They’re also trying to find cultural differences in walking styles. Pedestrians in western countries, for example, prefer more personal space between each other than people in eastern countries. “The system is agnostic to the culture, but it can also learn culture,” Bera says. “We can train the algorithm to learn from hundreds of videos from multiple countries at the same time, and the artificial intelligence learns the patterns in just a few minutes. That would be almost impossible for humans.”
With Bera’s research on human behavior modeling, he’s showing future driverless cars how to see people as more than obstacles to avoid. “A tree is different from a human being,” Bera says. “There’s emotion involved. There are ethics involved. We’re trying to make autonomous vehicles smarter by giving them a brain that understands humans better — an effort I think few people in the world are working on.”
Communicating with cars
While Bera focuses on the technology within autonomous vehicles, Clamann considers the situations in which humans and cars must interact.
“Imagine that someone is driving you somewhere and you have a question. It’s really easy to turn to the person and ask, what’s that or how much longer is it,” Clamann says. “You’re still going to have those questions when you’re using an autonomous vehicle.”
In his research, Clamann and his team test existing theories to determine the best practices for the self-driving car designers of the future, identifying the most efficient, safe, and inclusive ways for humans and said vehicles to communicate.
Clamann specifically studies pedestrian-autonomous vehicle interactions. Today, when you approach a crosswalk, a car may slow down and the driver will wave you to cross from behind the wheel. But how do you communicate to pedestrians that it’s safe to cross without a driver to wave them on?
To help driverless cars communicate with pedestrians, Clamann and his research team tested the effectiveness of signage on the front of the vehicle’s front bumper.
Leading ideas include a display with text reading “safe to cross.” In transportation, all text is required to be readable from 100 feet away, translating to a 4-foot wide sign tacked onto the front of the car.
“Not to mention, if you’re going to have a text-based message, what language are you going to have it in?” Clamann says. “A lot of people don’t necessarily speak English, and we shouldn’t exclude them from safety.”
Auto manufacturers have tried adding eyes to the front of vehicles to simulate eye contact between pedestrians and drivers. This may work for one pedestrian at a time, but when there’s a dense crowd, it’s not possible to make eye contact with every person.
“What’s most likely going to work is going to be very simple like a light, essentially the equivalent of a brake light, but on the front of the car,” Clamann says. Using a light alerts pedestrians and gives information about the car’s speed, rather than giving recommendations about what the pedestrian should do, eliminating problems that other methods create.
To test these theories, Clamann and his team set up an experiment akin to a theatrical production on either a closed track or a road open to traffic, which can take weeks or months to plan.
It took more than six months to set up a recent study to test the effectiveness of phone app that warns distracted pedestrians of oncoming traffic. First, they had to find the perfect location — somewhere with a blind corner so the car remains hidden from the pedestrian until the app’s alert. After deciding on a closed track, they calculated and rehearsed the car’s and pedestrian’s exact route and timing and hired nearly a dozen people to execute the experiment’s precise choreography.
North Carolina is an ideal location for this research, as well as any other autonomous vehicle testing, as long as the car has a steering wheel and pedals. “You can test pretty much anywhere you want as long as it’s done safely and in cooperation with the Department of Transportation, so we’re open for doing many kinds of research efforts,” Clamann says. “I think that’s not well understood by the research community here, so that’s something that I’m trying to help communicate.”
Avoiding potholes
Within the Department of City and Regional Planning, McDonald and colleagues are asking the big questions about how autonomous vehicles will shape our communities. “We wanted to bring people together to think about the problems that these vehicles can solve, but also the problems they might create and how to mitigate those problems in advance,” she says. Her team plans to examine a variety of impacts from shifting jobs to financing cities.
Within HSRC, Clamann is gearing up to do the same for the relationship between driverless cars and schools. “There hasn’t been a lot looking directly at how this is going to affect schools,” Clamann says. “Whether it’s autonomous buses picking up kids, or what’s going to happen when kids are being picked up and dropped off with autonomous vehicles, or the different regulations the counties have for vehicles going past schools and how’s that going to change.’’
There’s also the issue of rural roads. Right now, self-driving car technology is dependent on using camera sensors to identify lane markings on well-mapped streets to stay on course. In rural areas, those markings start to disappear. Combined with other hurdles, such as low access to broadband, Wi-Fi, and electric charging stations, that means it’ll be a while before autonomous vehicles make it into rural North Carolina.
This slow rollout challenges the assumption that driverless cars will immediately make streets safer for pedestrians. Rural areas and spaces away from intersections — places where it’s harder for autonomous vehicles to operate — won’t benefit from this technology until much later.
“If they were implemented ubiquitously, they could significantly reduce fatalities, but that’s not reality,” McDonald says. “Designing our communities to be safer is still important, because they’re not going to solve the pedestrian fatalities problem tomorrow.”
Identifying blind spots
Even with all of these problems solved, public opinion could still topple the implementation of autonomous vehicles.
“If people are frightened of these things for whatever reason, they’re not going to be accepted by the public, and then we’re never going use them,” Clamann says. “The majority of people are still scared, so there’s a lot of people trying to figure out what we can do to change that.”
After a 30 percent increase over a seven-year period, pedestrian fatalities reached the highest level in nearly three decades in 2016, according to the National Center for Statistics. Because driverless cars have the potential to drastically decrease those numbers, letting fear leave this technology on a test track may be a matter of life or death.
Clamann and McDonald co-authored a recent study on autonomous vehicles and pedestrian safety, analyzing nearly 5,000 pedestrian deaths in 2016 to determine what percentage could have been prevented with driverless cars on the road. They found that there could be a significant reduction in crashes in certain conditions such as at dusk or dawn.
“Autonomous vehicles have better sensors than the human eye, so they essentially can see in the dark,” McDonald says. “They should be better at detecting and eliminating that type of fatality or crash.”
These are the different ways in which human drivers become distracted on the road, according to the National Motor Vehicle Crash Causation Survey.
But self-driving vehicles are not a panacea. Experts often cite the statistic that 94 percent of traffic fatalities are due to human error, a type of crash they say these vehicles can eliminate. If you break down that figure, Clamann argues, there’s a different story.
“Two percent of that 94 percent is people having heart attacks, which is considered a human error,” Clamann says. “This whole idea in using autonomous vehicles and saying we’re just going to eliminate human error needs to be looked at a little more carefully.”
The dozens of situations that constitute a human error — drunk driving, distracted driving, pedestrians darting into the road — all require different approaches, Clamann asserts. It’ll take more time, cooperation, and research to determine the best way to design and implement this technology and steadily cut down that percentage.
Outside of safety, it’s too early to know what the fullest benefits of driverless cars are. Clamann predicts that we’ll find out a decade or two after the initial shift to fully autonomous vehicles on the road.
“When is that? I have no idea,” Clamann says. “I’m not sure it’s going to happen in my lifetime, but I’d love to see it.”