Methodology: Privacy Not Included

Questions asked for evaluation

• Is your personal data being shared or sold in ways you may not have expected?
• What is the company’s known track record for protecting the user data they collect?
• How does the company regularly test for and fix security vulnerabilities?

Other research points

We look at things like privacy policies, company websites, news reports, research whitepapers, app store listings, consumer reviews, and anything else we can find and trust to inform our research. Too often, the information companies make publicly available is vague or incomplete.

Data

We ding a company if they don’t have a clear and manageable way for users to delete their data from the company or explain how long they retain users’ data. We ding a company if they have a bad track record of not protecting users’ data based on known and reported security breaches, leaks, or vulnerabilities.

Determine

1. what kind of information is generally collected by a product, including personal, body-related, and social,
2. how the data is used by the company,
3. how you can control your data, including how you can access and delete your data,
4. the known track record of a company for protecting user data,
5. if the product can be used offline,
6. and whether the privacy policy is user-friendly.

Types of data

• Personal: name, email address, phone number, and address.
• Body-Related (including biometric data) such as fingerprints, voices, and heart rates.
• Social data includes information about your friends and contacts.

Data dealings

1. How much data does the company collect on a user? What can and does the company learn about you with this data? Does it collect a large amount of personal data or only what seems necessary for their product to work?
2. Does the company share, combine, or sell this data with a large number of third parties for purposes beyond the normal function of the product?
3. Does the company provide clear and explicit notice before sharing user data with third parties?
4. What types of data are shared for advertising and marketing purposes?

Dealing with security

1. Has the company had any major security vulnerabilities or data leaks in the past three years?
2. If the company has had known security vulnerabilities, have they acted quickly and openly to fix these security vulnerabilities and leaks?
3. Does the company have a track record of being honest and ethical when it comes to protecting user data?
4. What was the volume and sensitivity of leaked data?

Local and offline usage

We checked to see if the product could be used offline or if being online was a requirement to use the product effectively (if applicable)

Privacy declarations

Privacy information should be clear, readable, and communicate basic information to consumers about what happens to their data… The product must have a publicly available privacy policy and/or another privacy page that applies to the device, app, or service we are evaluating.

Encryption

The product must use encryption for all of its network communications functions and capabilities, ensuring that communications aren’t eavesdropped on or modified in transit.

Security Update

The product must support automatic security updates for a reasonable period after sale, and be enabled by default. This

AI

1. If we are able to determine if AI decisions demonstrate biases based on reporting from experts and trusted sources.
2. If we are able to determine if the AI behaves in some other way we consider unethical and/or untrustworthy based on reporting from experts and trusted sources.