Actuarial Standard of Practice No. 23

Data Quality

STANDARD OF PRACTICE

TRANSMITTAL MEMORANDUM

December 2016

TO: Members of Actuarial Organizations Governed by the Standards of Practice of the Actuarial Standards Board and Other Persons Interested in Data Quality

FROM: Actuarial Standards Board (ASB)

SUBJ: Actuarial Standard of Practice (ASOP) No. 23

This document contains the final version of a revision of ASOP No. 23, Data Quality.

Background

The ASB originally adopted ASOP No. 23, Data Quality, in 1993. That ASOP was prepared by the Data Quality Task Force of the Specialty Committee of the ASB. The ASB revised ASOP No. 23 in 2004 to be consistent with the then-current ASOP format, to reflect then-current, generally accepted practice with respect to data quality, and to provide guidance concerning other information relevant to the use of data. ASOP No. 23 was further updated for deviation language, effective May 1, 2011.

In 2014, the ASB concluded that this ASOP should be revised to update language to keep pace with practice changes (for example, increasing use of non-traditional data sources for predictive models, and legislatively mandated data submissions). This revision is the result of that review.

Exposure Draft

The exposure draft was released in November 2015 with a comment deadline of February 29, 2016. Twenty-two comment letters were received. The task force considered all comments received and made appropriate changes where needed. For a summary of the substantive issues contained in the comment letters on the exposure draft and the responses, please see appendix 2.

Key Changes

No significant changes have been made, but the wording has been clarified in a number of sections, including the following:

1. Section 1.2 (Scope) has been modified to clarify that if an actuary prepares data, or is responsible for the preparation of data, that the actuary believes will be used by other actuaries in providing actuarial services, the actuary should apply the relevant portions of this standard as though the actuary were planning to use the data, taking into account the preparing actuary’s understanding of the assignment for which the data will be used.

2. The defined term “comprehensive” has been replaced with the defined term “sufficient” (with the same definition), because that term fits more naturally with the definition.

3. The definition of “data” has been changed to clarify that it includes information derived mathematically from data.

4. Section 3.2(b)(3) has been revised to clarify that, in selecting data, the actuary should consider whether the data are reasonable given external data and information only to the extent the external data and information are relevant, readily available, and known to the actuary.

5. Section 3.3 has been modified to clarify that if an actuary performs a review of data, the actuary should consider comparing the current data to data used in the prior analysis, if similar work has been previously performed for the same or recent periods, but only if such consistency can reasonably be expected.

6. Section 3.4(c) has been modified to indicate that, rather than adjusting data to compensate for data deficiencies, an actuary may adjust the results of the analysis (for example, by increasing the range of reasonable estimates).

7. Section 3.4(e) has been modified to clarify that an actuary may, with the consent of the principal, complete any parts of an assignment for which the actuary determines the data are suitable, even though the data may be inadequate to complete the full assignment.

8. Section 3.7 has been added to remind actuaries that confidential information should be handled consistent with Precept 9 of the Code of Professional Conduct.

The General Committee would like to thank former committee members Jeremy J. Brown, Charles F. Cook, John C. Lloyd, Cande J. Olsen, and Lance J. Weiss for their contribution to the development of this ASOP.

The ASB voted in December 2016 to adopt this standard.

 

Data Quality Task Force
David L. Driscoll, Chairperson
Franklin C. Clapper, Jr.	Mary H. Simmons
Joseph M. Izzo	John W.C. Stark
Shannon C. Keller	Judy K. Stromback
Margaret Tiller Sherwood

General Committee of the ASB
Maria M. Sarli, Chairperson
Shawna S. Ackerman	Dale S. Hagstrom
Ralph S. Blanchard III	Margaret Tiller Sherwood
Raymond R. Brouillette	Mary H. Simmons
David L. Driscoll	Thomas D. Snook

Actuarial Standards Board
Maryellen J. Coggins, Chairperson
Christopher S. Carlson	Barbara L. Snyder
Beth E. Fitzgerald	Kathleen A. Riley
Darrell D. Knapp	Frank Todisco
Cande J. Olsen	Ross A. Winkelman

 

The ASB establishes and improves standards of actuarial practice. These ASOPs identify what the actuary should consider, document, and disclose when performing an actuarial assignment. The ASB’s goal is to set standards for appropriate practice for the U.S.

Section 1. Purpose, Scope, Cross References, and Effective Date

1.1 Purpose

The purpose of this actuarial standard of practice (ASOP) is to provide guidance to the actuary when performing actuarial services involving data.

1.2 Scope

This ASOP provides guidance to actuaries when selecting data, performing a review of data, using data, or relying on data supplied by others, in performing actuarial services. The ASOP also applies to actuaries who are selecting or preparing data, or are responsible for the selection or preparation of data, that the actuary believes will be used by other actuaries in performing actuarial services, or when making appropriate disclosures with regard to data quality. Other actuarial standards of practice may contain additional considerations related to data quality that are applicable to particular areas of practice or types of actuarial assignment.

If an actuary prepares data, or is responsible for the preparation of data, to be used by other actuaries in performing actuarial services, the actuary should apply the relevant portions of this standard as though the actuary were planning to use the data, taking into account the preparing actuary’s understanding of the assignment for which the data will be used.

This standard does not apply to the generation of a wholly hypothetical data set.

This standard does not require the actuary to perform an audit of the data.

If the actuary departs from the guidance set forth in this standard in order to comply with applicable law (statutes, regulations, and other legally binding authority), or for any other reason the actuary deems appropriate, the actuary should refer to section 4.

1.3 Cross References

When this standard refers to the provisions of other documents, the reference includes the referenced documents as they may be amended or restated in the future, and any successor to them, by whatever name called. If any amended or restated document differs materially from the originally referenced document, the actuary should consider the guidance in this standard to the extent it is applicable and appropriate.

1.4 Effective Date

This standard will be effective for any actuarial work product for which data were provided to or developed by the actuary on or after April 30, 2017.

Section 2. Definitions

The terms below are defined for use in this actuarial standard of practice.

2.1 Appropriate Data

Data suitable for the intended purpose of an analysis and relevant to the system or process being analyzed.

2.2 Audit

A formal and systematic examination of data for the purpose of testing its accuracy and completeness.

2.3 Data

Numerical, census, or classification information, or information derived mathematically from such items, but not general or qualitative information. Assumptions are not data, but data are commonly used in the development of actuarial assumptions.

2.4 Data Element

An item of information, such as date of birth or risk classification.

2.5 Review

An examination of the obvious characteristics of data to determine if such data appear reasonable and consistent for purposes of the assignment. A review is not as detailed as an audit of data.

2.6 Sufficient

Containing enough data elements or records for the analysis.

Section 3. Analysis of Issues and Recommended Practices

3.1 Overview

Appropriate data that are accurate and complete may not be available. The actuary should use available data that, in the actuary’s professional judgment, allow the actuary to perform the desired analysis. However, if significant data limitations are known to the actuary, the actuary should disclose those limitations and their implications in accordance with section 4.1(b). The following sections discuss such considerations in more detail.

3.2 Selection of Data

In undertaking an analysis, the actuary should determine what data to use. The actuary should take into account the scope of the assignment and the intended use of the analysis being performed to determine the nature of the data needed and the number of alternative data sets or data sources, if any, to be considered. The actuary should do the following:

a. consider the data elements that are desired and possible alternative data elements; and

b. select the data for the analysis with consideration of the following:

1. whether the data constitute appropriate data, including whether the data are sufficiently current;

2. whether the data are reasonable with particular attention to internal consistency;

3. whether the data are reasonable given relevant external information that is readily available and known to the actuary;

4. the degree to which the data are sufficient;

5. any known significant limitations of the data;

6. the availability of additional or alternative data and the benefit to be gained from such additional or alternative data, balanced against how practical it is to collect and compile such additional or alternative data; and

7. sampling methods, if used to collect the data.

3.3 Review of Data

A review of data may not always reveal defects. Nevertheless, the actuary should perform a review, unless, in the actuary’s professional judgment, such review is not necessary or not practical. In exercising such professional judgment, the actuary should take into account the purpose and nature of the assignment, any relevant constraints, and the extent of any known checking, verification, or audit of the data that has already been performed.

If, in the actuary’s professional judgment, it is not appropriate to perform a review of the data, the actuary should disclose that the actuary has not performed such a review, the reason the actuary has not performed such a review, and any resulting limitations on the use of the actuarial work product, in accordance with section 4.1(c).

If the actuary performs a review, the actuary should do the following:

a. make a reasonable effort to determine the definition of each data element used in the analysis; and

b. make a reasonable effort to identify data values that are questionable or relationships that are significantly inconsistent. If the actuary believes questionable or inconsistent data values could have a significant effect on the analysis, the actuary should consider taking further steps, when practical, to improve the quality of the data. The actuary should disclose in summary form any unresolved questionable data values that the actuary believes could have a significant effect on the analysis, in accordance with section 4.1(d). The actuary also should disclose any significant steps the actuary has taken to improve the data, in accordance with section 4.1(e).

If the actuary performs a review, the actuary should also consider comparing current data with the data used in the prior analysis for consistency, if similar work has been previously performed for the same or recent periods and if such consistency can reasonably be expected. If the actuary does not have the prior data, the actuary should consider requesting the prior data.

3.4 Use of Data

Because appropriate data that are accurate and complete may not be available, the actuary should make a professional judgment about which of the following are applicable:

a. the data are of acceptable quality to perform the analysis;

b. the data require enhancement before the analysis can be performed, and it is practical to obtain additional or corrected data that will allow the analysis to be performed;

c. judgmental adjustments or assumptions can be applied to the data that allow the actuary to perform the analysis. Any judgmental adjustments to data or assumptions should be disclosed in accordance with section 4.1(f). If the actuary judges that the use of the data, even with adjustments and assumptions applied, may cause the results to be highly uncertain or contain a significant bias, the actuary may choose to complete the assignment but should disclose the potential existence of the uncertainty or bias, and, if reasonably determinable, the nature and potential magnitude of such uncertainty or bias, in accordance with section 4.1(g). Alternatively, the actuary may compensate for the data deficiencies by adjusting the results, such as by increasing the range of reasonable estimates, and disclose the adjustments, in accordance with section 4.1(f);

d. if the actuary believes that the data are likely to contain significant defects, the actuary should determine, if practical, the nature and extent of any checking, verification, or audit of the data that has been performed. Then, if, in the actuary’s professional judgment, a more extensive review is needed, the actuary should arrange for such a review prior to completing the assignment; or

e. if, in the actuary’s professional judgment, the data are so inadequate that the data cannot be used to satisfy the purpose of the assignment, then the actuary should 1) obtain different data, 2) complete, with the consent of the principal, any parts of the assignment for which the actuary determines the data are suitable, or 3) decline to complete the assignment. However, if the actuary is required by a regulator or other governmental authority to use data that the actuary considers unsuitable for use in the actuary’s analysis, the actuary may use the data subject to the disclosure requirements of section 4.

3.5 Reliance on Data Supplied by Others

In most situations, the data are provided to the actuary by others. The accuracy and completeness of data supplied by others are the responsibility of those who supply the data. The actuary may rely on data supplied by others, subject to the guidance in sections 3.3 and 3.4. The actuary should disclose reliance on data supplied by others in an appropriate actuarial communication, in accordance with section 4.1(h).

3.6 Reliance on Other Information Relevant to the Use of Data

In many situations, the actuary is provided with other information relevant to the appropriate use of data, such as contract provisions, plan documents, and reinsurance treaties. The validity and completeness of such information are the responsibility of those who supply such information. The actuary may rely on such information supplied by others, unless it is or becomes apparent to the actuary in the course of the assignment that the information is unsuitable for use in the actuary’s analysis. The actuary should disclose reliance on such information supplied by others in an appropriate actuarial communication, in accordance with section 4.1(h).

If the actuary believes the information is unsuitable, or inconsistencies between the information and the data suggest that the information may be unsuitable, the actuary should make a professional judgment about whether to use the information. The actuary should consider disclosing when such relevant information that has been provided is not used.

If the information suggests that the data may be unsuitable, the actuary should make a professional judgment about whether to use the data based on the considerations described in sections 3.4 and 3.5.

3.7 Confidentiality

The actuary should be aware that data may contain confidential information. Such confidential information should be handled consistent with Precept 9 of the Code of Professional Conduct.

3.8 Limitation of the Actuary’s Responsibility

The actuary is not required to do any of the following:

a. determine whether data or other information supplied by others are falsified or intentionally misleading;

b. compile additional data solely for the purpose of searching for questionable or inconsistent data; or

c. perform an audit of the data.

Section 4. Communications and Disclosures

4.1 Communication and Disclosure

Any actuarial communication prepared to communicate the results of work subject to this standard should comply with the requirements of ASOP No. 41, Actuarial Communications. An actuarial communication can comply with some or all of the specific requirements of this section by making reference to information contained in other actuarial communications available to the intended users (as defined in ASOP No. 41), such as an annual actuarial valuation report. Such communication should contain the following disclosures when relevant and material:

a. the source(s) of the data;

b. any limitations on the use of the actuarial work product due to uncertainty about the quality of the data or other information relevant to the use of the data, as discussed in section 3.1;

c. whether the actuary performed a review of the data and, if not, the reason for not reviewing the data and any resulting limitations on the use of the actuarial work product, as discussed in section 3.3;

d. in summary form, unresolved concerns the actuary may have about questionable data values that are relevant to the use of the data and could have a significant effect on the actuarial work product, as discussed in section 3.3(b);

e. in summary form, discussions of any significant steps the actuary has taken to improve the data due to identifying questionable data values or relationships, as discussed in section 3.3(b);

f. in summary form, significant judgmental adjustments or assumptions that the actuary applied to the data or to the results, or are known by the actuary to have been applied to the data, to allow the actuary to perform the analysis, as discussed in section 3.4(c);

g. the existence of results that are highly uncertain or have a potentially significant bias of which the actuary is aware due to the quality of the data or other information relevant to the use of the data, and the nature and potential magnitude of such uncertainty or bias, if they can be reasonably determined, as discussed in section 3.4(c);

h. the extent of the actuary’s reliance on data and other information relevant to the use of the data supplied by others, as discussed in sections 3.5 and 3.6;

i. the disclosure in ASOP No. 41, section 4.2, if any material assumption or method was prescribed by applicable law (statutes, regulations, and other legally binding authority);

j. the disclosure in ASOP No. 41, section 4.3, if the actuary states reliance on other sources and thereby disclaims responsibility for any material assumption or method selected by a party other than the actuary; and

k. the disclosure in ASOP No. 41, section 4.4, if, in the actuary’s professional judgment, the actuary has otherwise deviated materially from the guidance of this ASOP.

Appendix 1 – Background and Current Practices

Note: The following appendix is provided for informational purposes, but is not part of the standard of practice.

Background

An actuarial analysis is based upon an analysis of data, along with practical knowledge of the area of practice and training in actuarial theory, which together enable the actuary to perform and interpret the results of calculations. Throughout the analytic process, data play an important role. The accuracy and validity of the actuarial analysis are dependent on, among other things, the quality of the data used. Hence, an actuarial standard of practice concerning data quality is appropriate.

Data frequently contain errors, are not complete, and are not precisely appropriate for the intended analysis. Actuaries deal with these limitations, the majority of which are non-critical. However, actuaries are often called upon to perform actuarial services in situations where data limitations may be critical. Actuaries use professional judgment when determining whether and how to refine data or make modifications within the analysis.

Current Practices

Actuaries use informed judgment to determine what kinds of data are appropriate for a particular analysis. It is important that the data used are relevant to the system or process being analyzed.

Data have played an increasingly important role in actuarial practice in recent years. In addition to the traditional uses of data that have been in place for many years, actuaries and their principals have been using broader sources of data more recently to support improved business decisions. This has included more sophisticated data analytics to improve functions such as claims processes, underwriting, pricing, loss control, distribution management, and customer service. In addition, there has been expansion of use of sophisticated models for a wide range of purposes, and those models are heavily reliant on the data inputs. Because of their analytical skills, actuaries have been deeply involved in these advancements, including assessing the quality and sufficiency of data for use in various applications.

Persons or organizations responsible for generating, collecting, or publishing data may apply different standards of quality assurance, ranging from straightforward compilation of figures to extensive verification. Actuaries, in turn, deal with the question of the quality of data underlying their work products in a variety of ways and with varying levels of review or checking.

Actuaries are called upon to provide analyses for a broad range of audiences, from limited distribution within an organization to public exposure.

Important aspects of data use include documentation and disclosure of 1) the sources of data, 2) review of data, 3) significant biases resulting from data, 4) adjustments or corrections made to the data, and 5) the extent of reliance on data supplied by others. Typically, actuaries do not audit data.

Appendix 2 – Comments on the Exposure Draft and Responses

The exposure draft of this revision of ASOP No. 23, Data Quality, was issued in November 2015 with a comment deadline of February 29, 2016. Twenty-two comment letters were received, some of which were submitted on behalf of multiple commentators, such as by firms or committees. For purposes of this appendix, the term “commentator” may refer to more than one person associated with a particular comment letter. The Task Force carefully considered all comments received, and the General Committee and ASB reviewed (and modified, where appropriate) the proposed changes.

Summarized below are the significant issues and questions contained in the comment letters and the responses to each.

The term “reviewers” includes the Task Force, General Committee, and the ASB. Unless otherwise noted, the section numbers and titles used below refer to those in the exposure draft.

Click here to view Appendix 2 in its entirety.