Safety as a metric for a successful test
To our Operational Safety team, a safe test begins with preparation and is the key indicator of success.
Kyle Lansing, Operational Safety Engineer
Explain safety as a metric in testing.
I recently trained a new group of Safety Conductors and I explained the importance of why every member of the test crew should strive for tests that are safe, efficient, and successful. Safety is the key metric there. Any test, in order to be successful, must be safe. For example, say a test doesn’t provide all the data we were originally looking for. If that test is executed safely, then it was successful.
What is a safe test?
Obviously, it is first a test that avoids danger or risk. But I believe it’s more than just everyone coming back having avoided a crash. If our test crew wasn’t prepared or if someone was uncomfortable with the vehicle behavior, Torc wouldn’t qualify that as a safe test.
How do you incorporate safety into your day-to-day job?
While we train and prepare to act rapidly in critical situations, I believe the true reflection of a robust safety program lies in the day-to-day operations. Every well-prepared test plan, checklist, and brief is one step toward ensuring a safer test.
In my day-to-day life, a safe test begins about 1.5 hours prior to us ever starting up a vehicle. There are inspections, crew task lists, and an assembly about the goals of our trip and expected behavior from both the vehicle and the test crew. Safety starts with preparation and I work to make sure everyone is prepared.
Myra Blanco, Daimler Torc Senior Technical Fellow
Explain safety as a metric in testing.
An important measurement of whether an effort is safe is the absence of safety-related incidents (e.g., crash, near crash). However, if we only look at those incidents that result in severe consequences – crashes – it might be too late. There are probably a lot of things we could have caught beforehand (e.g., near crashes, minor protocol discrepancies). Incidents are symptoms of actions that were not appropriate before the actual issue manifests. Defining safety metrics involves making sure you have put all the possible safeguards in place.
Torc software engineers rigorously test code prior to on-road testing.
There is not a single metric of safety, it is a conglomerate of multiple things. If we take crashes, for example – I don’t call them accidents for a reason – crashes have contributing factors. Let’s look at this metaphor: If you think about a piece of swiss cheese that has a lot of holes, imagine each one of those holes is a factor. When all those holes intersect, something can fall through the middle – this is akin to when multiple contributing factors line up to cause something to go wrong. So, it is not an accident that an incident happens – it is a contribution of multiple issues. In the case of a typical car crash, perhaps the person was distracted, was fatigued, or was speeding, combined with the vehicle having a potential fault or failure. None of those things happen randomly.
To help prevent these issues, my team works to identify and mitigate contributing factors at four levels: people, technology, environment, and vehicle. For example, a metric of success is ensuring you have 100% of your drivers trained. This shows that Torc is providing the appropriate information to do their job safely. Another would be ensuring that there are appropriate communication systems in place. Communication channels that are open are one of the most important parts of developing a safety management system.
Safety starts long before the road
Clearly, the process of avoiding incidents begins long before we put tires to the road. This process is established in the technical development stages as well, as our software engineers explain below.
Manas Gupta, Software Engineer in Behaviors, Planning & Controls
What is one task or process that you perform that forwards safety in your day-to-day work?
Safety is the most important and integral part of all the core values at Torc and we as a team work to develop software without compromising those core values. We make sure that our self-driving software is passed through various stages of testing before it is ready to deploy on the vehicle and can be run on public roads.
I follow three stages of testing to make sure the software I write or modify is safe to be deployed on the vehicle. First, I safeguard my code with unit tests for the given functionality, so that future unwanted changes can be caught in the development. Second, I seek peer review of my code. Last, I test the code on a simulated environment with all the possible scenarios to make sure the code is robust and can be tested on the autonomous vehicle on public roads.
Elijah Hodges, System Integration Engineer Co-op in Systems Integration
Explain safety as a metric in testing.
The Torc mission is to save lives. Safety is not just a part of our goal, it is our goal. If the tests we perform were to introduce dangerous situations, then we are going directly against our mission.
How do you incorporate safety into your day-to-day job?
The System Integration team is responsible for overseeing the high-level picture of vehicle performance. We help coordinate releasing new development onto our self-driving vehicles for testing to ensure that these changes improve the system as a whole. While a developer may just be implementing a change to fix a very specific issue, it is our job to notice and comment on how the change may affect peripheral responses. Ultimately, we are in charge of deciding if new changes are safe enough to be accepted into the software. Because we are not writing the code ourselves, it takes away the emotional temptation to accept new code changes just because they work and encourages us to determine if the overall performance was improved.
One day-to-day task that improves safety is our morning sync meetings every morning. Communication is absolutely vital for safety and having a venue for promoting that every single day is one of the best ways to find problems before they escalate.