Designing for Safety

The World Economics Forum defines safety-by-design as an approach that puts user safety and rights at the centre of the design and development of products and services. In other words, designing for safety means thinking about how the design decisions we make impact the physical and mental well-being of the end users.

Overlooking safety at the design stage can have real consequences for the users. As designers, we have an opportunity to prevent safety risks before they happen.

Products are often misused for harm

Even products that were designed with the best intentions in mind can still be misused for harm. For instance, the well-known fitness app Strava uses location-based services to share users’ running routes. Though the route-sharing feature was not intended to be used for harm, people began using this feature as a way to stalk others.^¹

Apple’s “Live Listen” feature is another example of misuse for harm. When Apple came out with this feature, it was intended to serve as a hearing aid for users. However, the feature has been widely used as spyware to eavesdrop on other people’s conversations when they’re in another room.

Tweet by Arnoldcrndo about “Live Listen”^²

Smart home devices have also been misused in harmful ways. The Google Nest app was designed so that users could conveniently control the temperature of space from anywhere or set a custom temperature schedule to save energy. However, In the book Design for Safety, author Eva PenzeyMoog recounts a story of a woman in an abusive relationship where her partner would suddenly change the temperature in her home to be extremely cold or hot while he was away at work. This led to confusion, gaslighting and negative mental health consequences for the woman.

Despite good intentions, many products have been misused to abuse others. Designers need to carefully consider how their designs could be weaponized to hurt others, but how?

Designing for safety principles

There are three basic principles of designing for safety. These principles were adapted from the eSafety Commissioner.

1. Designer Responsibility

End users should not be solely responsible for their own safety. Designers of products and services need to take more responsibility to protect their users from harm. They should build safety guides into the design process.

2. User Involvement

During the design phase, designers should speak with a wide range of users to better understand their needs for safety. Designers should prioritize research with vulnerable groups such as racial minorities, people with disabilities, women, elders, and children.^³ Additionally, users should be given access and control over their privacy and security settings in the final product.

3. Transparency and Accountability

Safety risks should be disclosed to users. Companies should not hide safety issues with their products from users once they become aware of them. User safety should always take precedence over brand reputation.

Safety-Minded Design Tips

There are several safety-minded design suggestions that designers can follow. The first three tips have been adapted from a book by Eva PenzeyMoog called Designing for Safety and an episode from the (Re)Design podcast called “Eva PenzeyMoog ‘Designing for Safety.’”

1. Regularly remind users who their accounts and information are shared with.

This is because if a user shares their location with someone, they may have forgotten that they did so a few months later. If they are facing a safety issue like stalking, it might be difficult for them to find out which apps or accounts are sharing their location with the person that is stalking them. If apps and services regularly remind users who their sensitive information is shared with, it can make it easier for them to update settings and prioritize safety. Apple has incorporated this into its recent Safety Check feature.

2. Prompt users to reconsider potentially harmful actions.

If a design slows down a user while they are doing something that may cause harm to others, the user will be less likely to follow through. In an experiment by Trisha Prabhu with adolescents, participants were given an alert message when they were about to post a hurtful comment on social media. In the experiment, there was a 93% reduction in the number of participants willing to post hurtful messages after being given an alert that allowed them to reconsider their actions.^⁵ This shows that slowing users down and getting them to rethink their actions can help reduce harm. Similarly, Twitter started testing this idea for their platform in 2020. Twitter found that “if prompted, 34% of people revised their initial reply or decided to not send their reply at all.”^⁶

Tweet by @TwitterSupport about reviewing Tweets^⁷

3. Give all users equal control over shared accounts.

Oftentimes, products allow for multiple users, but there is usually one user that has more control than the rest. For example, many banks let you create a shared bank account with a primary user and a secondary user but not two users with an equal amount of control. This can be problematic if the primary user becomes abusive or the two people decide to separate. The primary user has a lot more control over the bank account even though it is shared. This is why giving the users an equal amount of control is a better practice in many circumstances.

4. Avoid using deceptive design patterns.

Deceptive design, also known as “dark patterns,” is a term coined by a user experience designer named Harry Brignull. In his own words, he describes the deceptive design as “a user interface that has been carefully crafted to trick users into doing things… they are not mistakes, they are carefully crafted with a solid understanding of human psychology, and they do not have the users’ interests in mind.”^⁸

There are multiple examples of deceptive design patterns still being used today. Through different methods, their goal is to ultimately manipulate people into actions that benefit the service or business. These need to be flagged and completely removed from design toolkits in order to uphold safety principles.

Integrating Safety-minded Design into Design Processes

It is important to dedicate time and effort to thinking about user safety during your design process. The following list contains some steps for integrating safety-minded design into design processes.

1. Assemble a cross-functional team.

It is difficult for one person to effectively identify and address safety risks on their own. Having a team of diverse individuals, from various functions like marketing and engineering, is beneficial when designing for safety.

2. Dedicate time and resources to thinking about safety-minded design.

It is important to set aside time during the design phase to consider user safety. It could be necessary to put some effort into persuading stakeholders and business owners that user safety is ultimately beneficial in the long run. Reputation aside, positive experiences generate revenue by reducing customer attrition and growing loyalty.

3. Identify potential safety risks.

When designing a product, think about ways your product could be weaponized. To identify additional risks, conduct research with a range of users while prioritizing research with at-risk user groups, such as those dealing with domestic violence.

Another effective way of identifying risks is “hazard mapping.” A hazard map highlights areas of a product that pose safety risks. You can visualize your product as a map and list the various risks you notice in each of its sections.

4. Prioritize the risks that you have identified.

It is unrealistic to try and prevent every single possible safety risk you identify. This is why you should prioritize the safety risks based on their likelihood and level of impact. Place the risks on a Risk Priority Matrix. Risks that fall in the red area should be prioritized first because those risks have the highest impact and are the most likely to happen, followed by the orange and yellow areas. Risks in the green area do not need to be prioritized but should still be kept in mind when providing design solutions.

5. Create a plan for each of the safety risks.

You can research similar products to see how others have prevented risks like the ones you have prioritized. When preparing for the various safety risks, you have a number of options. Some of the possible responses are on the list below which was adapted from a book called Project Management: The Managerial Process by Erik Larson and Clifford Gray.^¹¹

Mitigating: This means actively working to reduce the likelihood that the risk will happen or reduce the impact that the risk could have.
Avoiding: Changing or altering the product to completely eliminate the risk.
Transferring: Passing risk to another party (outsourcing) more equipped to solve it.
Escalating: Notifying someone in a higher position of the risk.

6. Test designs for additional safety risks before they are released.

Before releasing a product, it should be tested with a small set of beta users. By testing the product, you might end up finding more safety risks that were not previously identified, giving the team a chance to fix them before exposing thousands, if not millions, of customers to it.

7. Be transparent with users about safety risks.

If a product has safety risks associated with it, you should share this information with users. If you include safety concerns with your product in a privacy policy or terms and conditions, they need to be clear and easy to understand. Terms and conditions are known to be notoriously complicated, write at an elementary school level if possible.

TL;DR

By incorporating safety-minded design principles into design processes, designers can lessen the likelihood that their products will be used to harm others.

Here’s a checklist of the things you can do right now to design with safety in mind:

Assemble a cross-functional team.
Dedicate time and resources to thinking about safety-minded design.
Identify potential safety risks through hazard mapping and research.
Prioritize the risks that you have identified.
Create a plan for each of the safety risks.
Mitigate: This means actively working to reduce the likelihood that the risk will happen or reduce the impact that the risk could have.
Avoid: Alter the product to completely eliminate the risk.
Transfer: Pass risk to another party more equipped to solve it.
Escalate: Notifying someone in a higher position of the risk.
Test designs for additional safety risks before they are released.
Be transparent with users about safety risks.

Footnotes:

‍