Lesson 2: Data Quality Assessment

The rest of this course explains different attributes of data quality to help you think through all the components of data quality in a systematic way. This lesson, however, provides you with a bird’s-eye view of what you would need to do to assess and improve the quality of the data in your system. This lesson can help you think through what things you can (and can’t) do in your system, and where you might want to start.

There is a lot to consider when conducting a data quality assessment, but the steps below serve as a good starting point.

Step 1: Gather background information

Before you begin, it’s important to gather background information about the data to ensure that you can accurately evaluate its quality. If supporting materials do not exist, then you’ll probably want to begin building some. Conducting a data quality assessment will help you identify which materials will be the most helpful in improving the quality of your data. These materials may include the following:

• Codebooks, data maps, etc.
• Instructions or training materials on how to enter data into the data system
• Internal audits or audit processes
• Reports generated using the data

Step 2: Evaluate data compared to data quality attributes

Lessons 3–6 explore dimensions and attributes of data quality. As you can see, and as you’ll learn, there is a lot more to data quality than whether numbers are right or wrong. This table summarizes the main points:

Dimension	Definition	Attributes
Intrinsic Lesson 3	The data is accurate, reliable, credible, and impartial.	Accuracy Believability Reputation Objectivity
Contextual Lesson 4	The data meets the needs of a particular use case.	Relevance Completeness Timeliness
Representational Lesson 5	The data is concise, consistent, interpretable, and easy to understand.	Understandability Interpretability Consistency
Accessibility Lesson 6	The data is available and easy to access and use.	Accessibility Access Security

Use the following checklist to understand your agency’s data quality strengths and weaknesses. Then, start thinking about which dimensions and which attributes of data quality you might want to focus on.

Intrinsic data quality (Lesson 3)

Accuracy attribute

• Are there any incomplete observations?
• Are there any extreme, unlikely, or impossible values? (Explore dates, placements, etc.)
• How many records are duplicates?

Believability attribute

• Do the data values correctly represent the real-world data they describe?

Reputation attribute

• How does the agency collect the data (e.g., by hand, by observation, at intake)?
• What is the operational purpose of this data?
• Who is collecting the data?
• Are data elements defined?

Objectivity attribute

• Is data from a reliable and trustworthy source or process?
  
• Is the data collected using a standard method backed by a robust protocol?
• Are all data processing steps clearly documented?

Contextual data quality (Lesson 4)

Relevance attribute

• Does the data meet the needs of users?
• Can the data be used for the types of analyses intended?
• Can the data be used for different types of analysis?

Completeness attribute

• Where are there gaps in the dataset?
• What data is missing overall?
• What percentage of data is missing for critical fields?

Timeliness attribute

• Is the data up to date?
• How frequently is data collected, updated, or refreshed?
  
• What is the delay between data collection and availability?
  
• Do timestamps or other documentation indicate when data was collected/updated?

Representational data quality (Lesson 5)

Understandability attribute

• Is documentation maintained to support data understandability?
• What percentage of the data is covered by supporting documentation?
• Is this documentation available to analysts and people entering data?

Interpretability

• Are metadata and documentation provided for all datasets?
• Are the metadata and documentation sufficient for understanding the context, meaning, and limitations of data?
• Are data presented to users in a human-readable format?

Consistency

• Are data consistently structured?
• Are data labeled across various data sources?
• Do related datasets contradict one another?

Accessibility data quality (Lesson 6)

Accessibility attribute

• Is data recorded in a centralized case management system?
• Is data managed by an internal (in-house) or external (contracted) team?
• How long does it take for a new user to gain access to a data system or collection?
• How easily can users access and retrieve data within their systems?
• Is data access enabled by user-friendly interfaces?
• Do users receive training on how to use the data system(s)?
• Are data users able to create reports or query/pull data from the database?

Access security attribute

• Is there a mechanism for sharing data with other agencies and partners?
• Are access control mechanisms in place (e.g., authentication) to ensure that only authorized users can access the data?
• Is the data stored and transmitted using secure and encrypted methods to protect the data from unauthorized access and breaches?
• Are users trained in data security best practices, ensuring that they understand their roles and responsibilities in maintaining data security?
• Is there a data breach response plan to mitigate the impact of any unauthorized access?
• Are users being granted the minimum permissions necessary for a user to accomplish an assigned task? Are users being granted too many or not enough privileges?