Proposed Revision of Actuarial Standard of Practice No. 38

Using Models Outside the Actuary’s Expertise (Property and Casualty)

STANDARD OF PRACTICE

TRANSMITTAL MEMORANDUM

September 2011

TO: Members of Actuarial Organizations Governed by the Standards of the Actuarial Standards Board and Other Persons Interested in the Use of Models Outside the Actuary’s Area of Expertise in Property and Casualty Insurance

FROM: Actuarial Standards Board (ASB)

SUBJ: Proposed Revision of Actuarial Standard of Practice (ASOP) No. 38

This document is an exposure draft of a proposed revision of ASOP No. 38, Using Models Outside the Actuary’s Area of Expertise (Property and Casualty).

Please review this exposure draft and give the ASB the benefit of your comments and suggestions. Each written response and each response sent by e-mail to the address below will be acknowledged, and all responses will receive appropriate consideration by the drafting committee in preparing the final document for approval by the ASB.

The ASB accepts comments by either electronic or conventional mail. The preferred form is e-mail, as it eases the task of grouping comments by section. However, please feel free to use either form. If you wish to use e-mail, please send a message to comments@actuary.org. You may include your comments either in the body of the message or as an attachment prepared in any commonly used word processing format. Please do not password protect any attachments. Include the phrase “ASB COMMENTS” in the subject line of your message. Please note: Any message not containing this exact phrase in the subject line will be deleted by our system’s spam filter. Comments will be posted in the order that they are received. Comments received after the deadline will not be posted.

If you wish to use conventional mail, please send comments to the following address:

ASOP No. 38 Revision
Actuarial Standards Board
1850 M Street, Suite 300
Washington, DC 20036

Deadline for receipt of responses in the ASB office: January 16, 2012

Background

ASOP No. 38 was adopted by the ASB in June 2000. Subsequent to this adoption, the ASB created a task force under the General Committee to consider expanding the scope of this ASOP to all practice areas.

The ASB released the first exposure draft expanding the scope to all practice areas in October 2003 and received twenty-six comment letters. The second exposure draft incorporated several changes as a result of these comments and was released in March 2006.

As a result of the comments received on this second exposure draft, the ASB referred the item back to the Casualty Committee to maintain ASOP No. 38 as a standard applicable only to the Casualty practice area. The ASB also instructed the Casualty Committee to update ASOP No. 38 to reflect current terminology and practice, and consider the suggested changes from the General Committee.

Key Changes

Many changes were made to the format and terminology used in this standard. Noteworthy changes to the content include guidance regarding the actuary’s consideration of the appropriateness of the model (see section 3.2), and guidance regarding the review of the model (see sections 3.3 and 3.4).

Request for Comments

The Casualty Committee and the ASB would appreciate comments on all areas of this proposed revision to ASOP No. 38 and, in particular, on the above key changes.

In addition, the Committee and ASB request comments on the following issues:

1. This exposure draft does not require the actuary to use the latest version of a model if the previous version is appropriate for its intended use. Is this sufficiently clear?

2. This exposure draft maintains similar language as the existing standard regarding documentation (see section 3.7). Is this appropriate in light of the adoption of revised ASOP No. 41, Actuarial Communications?

3. Is section 1.2, Scope, clear in identifying that it is a user of the model itself that is subject to this proposed revision and not an actuary who uses the work product prepared by another actuary or other expert who used the model?

4. Since June 2000, when this standard was originally adopted, property and casualty actuaries have made use of a wider variety of models. Is this proposed revision still appropriate for all models used outside the actuary’s expertise that are related to property and casualty insurance coverages and products?

The ASB reviewed this draft at its September 2011 meeting and approved its exposure.

Task Force to Revise ASOP No. 38

Kenneth R. Kasner, Chairperson

                      Shawna S. Ackerman                                          Jonathan White

 

Casualty Committee of the ASB

Beth Fitzgerald, Chairperson

                      Shawna S. Acerman                                           Dale F. Ogden

                      Raji Bhagavatula                                                 David J. Otto

                      Larry A. Haefner                                                 Marc B. Pearl

                      Kenneth R. Kasner

Actuarial Standards Board

Albert J. Beer, Chairperson

                      Alan D. Ford                                                        Patricia E. Matson

                      Patrick J. Grannan                                               Robert G. Meilander

                      Stephen G. Kellison                                             James J. Murphy

                     Thomas D. Levy                                                   James F. Verlautz

The ASB establishes and improves standards of actuarial practice. These ASOPs identify what the actuary should consider, document, and disclose when performing an actuarial assignment. The ASB’s goal is to set standards for appropriate practice for the U.S.

Section 1. Purpose, Scope, Cross References, and Effective Date

1.1 Purpose

This actuarial standard of practice (ASOP) provides guidance to actuaries when performing professional services with respect to developing an actuarial work product where models that incorporate specialized knowledge outside the actuary’s expertise are used. This standard addresses the nature and extent of the actuary’s obligation with respect to determining the appropriateness of the model for its intended use and the review of the model.

1.2 Scope

This standard applies to actuaries when using models that incorporate specialized knowledge outside of the actuary’s expertise when performing professional services in connection with property and casualty insurance coverages and products. This includes risk financing systems such as self-insurance and securitization products that provide similar coverage. The determination of whether the specialized knowledge incorporated in the model is outside the actuary’s expertise is the responsibility of the actuary.

This standard applies to the actuary’s use of both proprietary and non-proprietary models, as well as models that are either actuarial or non-actuarial in nature.

If the actuary departs from the guidance set forth in this standard in order to comply with applicable law (statutes, regulations, and other legally binding authority) or for any other reason the actuary deems appropriate, the actuary should refer to section 4.

1.3 Cross References

When this standard refers to the provisions of other documents, the reference includes the referenced documents as they may be amended or restated in the future, and any successor to them, by whatever name called. If any amended or restated document differs materially from the originally referenced document, the actuary should consider the guidance in this standard to the extent it is applicable and appropriate.

1.4 Effective Date

This standard is effective for any actuarial work product covered by this standard’s scope issued on or after four months after adoption by the Actuarial Standards Board (ASB).

Section 2. Definitions

The terms below are defined for use in this actuarial standard of practice.

2.1 Expert

An actuary or other individual who is qualified by knowledge, skill, experience, training, or education to render an opinion concerning the matter at hand.

2.2 Expertise

The specialized skills or knowledge possessed by an individual.

2.3 Model

A representation, usually mathematical, of a specified phenomenon or behavior. A model typically includes mathematical equations, logic, algorithms, and associated data.

2.4 Model Outputs

Data or information produced by the model.

Section 3. Analysis of Issues and Recommended Practices

3.1 Overview

The actuary should perform the following when using a model that incorporates specialized knowledge outside the actuary’s expertise in preparing an actuarial work product:

a. evaluate the appropriateness of the model for its intended use (see section 3.2); and

b. review the model (see section 3.3.) or rely on a review of the model by another actuary (see section 3.4).

3.2 Appropriateness of the Model for the Intended Use

The actuary should use professional judgment to determine whether the model is appropriate for use in developing the actuarial work product, taking into account the following:

3.2.1 General Considerations

The actuary should consider the following:

a. the data, assumptions, parameters, or subjective judgments that affect the output of the model;

b. the extent to which there has been prior use of the model. Some models might be widely used for the intended purpose and widely accepted for the intended use. At other times, the actuary may work with models that have undergone relatively little use or are otherwise not widely accepted for the intended use;

c. the extent to which there has been prior review of the model;

d. whether there are known current circumstances or situations where the model might produce biased or inaccurate results;

e. whether the model has known limitations that are significant and relevant, and what those limitations are; and

f. whether modifications to the model or assumptions inherent in the model are needed in order to generate output appropriate for its intended use.

3.2.2 Model Revisions

If the model has undergone significant revisions subsequent to its review for appropriateness, the actuary should be reasonably familiar with these revisions in order to apply professional judgment as to whether the model, either revised or as originally reviewed, is appropriate for use in developing the intended actuarial work product.

3.2.3 Developments in Relevant Fields

The actuary should consider whether significant current developments in the subject matter addressed by the model and in relevant fields of knowledge are likely to affect the appropriateness of the model for the intended use.

3.3 Actuaries Reviewing Models Outside Their Expertise

When the actuary personally reviews the model to be used in the actuarial work product, the actuary should be reasonably familiar with the basic operation of the model and consider each of the following:

3.3.1 Model Components

A model may include several components. For example, hurricane models may include meteorological and vulnerability components; earthquake models might include seismological and geological components. The actuary should be reasonably familiar with the major components of the model and how such components interrelate or have interdependence within the model .

3.3.2 Fields of Expertise Used

The actuary should understand which fields of expertise were used when the model was developed or updated, and should consider, to the extent practical, if the model is based on currently accepted practices within the applicable fields of expertise.

3.3.3 Review and Testing by Experts

The actuary should be reasonably familiar with the level of expert review and testing, if any, that the model has previously undergone. The experts may be either the experts who developed or provided the model or other experts in the applicable field. The actuary should consider factors such as the following:

a. whether there are any known significant differences of opinion among such experts regarding aspects of the model that could be material to the actuary’s use of the model ;

b. whether there are non-actuarial professional standards that apply to the development, testing, validation, or use of the model, and whether the model has been evaluated and has met such standards;

c. whether the model had undergone peer review by such experts ; and

d. the professional credentials or other apparent qualifications of such experts and the professional affiliation of such experts .

3.3.4 User Input

User input can comprise input data or user assumptions. The actuary should have a reasonable understanding of the specifications for the user input and the relationship between the model’s input and output. The actuary should evaluate user input for reasonableness and take reasonable steps to confirm that the user input is appropriate and consistent with the intended use of the model output.

3.3.5 Model Output

The actuary should review the model output for reasonableness, considering factors such as the following:

a. results derived from alternate models or methods, where available and appropriate;

b. comparison of actual historical experience to model output;

c. the consistency and reasonableness among various output results, including previous runs of the model;

d. the sensitivity of the model output to variations in the user input; and

e. the overall reasonableness of results, based on experience, comparison to established benchmarks, or professional judgment.

3.4 Actuaries Relying on Model Review by Another Actuary

The actuary may rely on another actuary who has conducted some or all of the review for a particular model . The actuary relying on the other actuary’s review should be satisfied that such review was performed in accordance with this standard and is appropriate for the intended application. The actuary should document the extent of such reliance in accordance with section 3.7, and disclose in accordance with section 4.1(e).

3.5 Relative Importance of the Model Output

If variations in the model output have minimal impact on the actuary’s results and conclusions, then the model has limited relative importance. The actuary’s level of effort in understanding and evaluating a model, as described in sections 3.1-3.4, should be consistent with the actuary’s determination of the relative importance of the model’s output to the actuarial work product.

3.6 Appropriate Use of the Model Output

The actuary should use professional judgment to determine whether explicit adjustments to the model output are appropriate for its intended use. The actuary should disclose any such adjustments in accordance with section 4.1(c).

3.7 Documentation

The actuary should create documentation that demonstrates how the actuary has met the guidance of sections 3.1-3.6 above. If the model has proprietary aspects or contains proprietary information, the actuary should document the steps taken to comply with this standard in light of the proprietary aspects or information.

3.8 Data

The actuary should refer to ASOP No. 23, Data Quality, with respect to selection of data to be used, relying on data supplied by others, reviewing data, and using data.

Section 4. Communications and Disclosures

4.1 Actuarial Communications

When issuing an actuarial communication subject to this standard, the actuary should refer to ASOP No. 41, Actuarial Communications. The actuary should also refer to ASOP No. 23.

In addition, the actuary should disclose the following in an appropriate actuarial communication:

a. an identification of the model used;

b. any adjustments to the model, as described in section 3.2.1(f);

c. any adjustments made to the model output, as described in section 3.6;

d. any significant unresolved concerns the actuary may have about the model or the model output;

e. where applicable, reliance on another actuary for the model review, as described in section 3.4;

f. the disclosure in ASOP No. 41, section 4.2, if any material assumption or method was prescribed by applicable law (statutes, regulations, and other legally binding authority);

g. the disclosure in ASOP No. 41, section 4.3, if the actuary states reliance on other sources and thereby disclaims responsibility for any material assumption or method selected by a party other than the actuary; and

h. the disclosure in ASOP No. 41, section 4.4, if, in the actuary’s professional judgment, the actuary otherwise deviated materially from the guidance of this ASOP.

Appendix – Background and Current Practices

Note: This appendix is provided for informational purposes, but is not part of the standard of practice.

Background

Actuaries have been using models in the development of actuarial work products for many years.  Most models used by actuaries are developed using expertise that is common and familiar to actuaries, and their use by actuaries is guided by existing standards of practice and statements of principles.

However, actuaries also use models that contain compo­nents that are outside the actuary’s own area of expertise. For example, certain catastrophe models, interest rate models, dynamic financial analysis models, credit scoring models, economic capital models and enterprise risk management models contain features and components that are outside the expertise of many of the actuaries who use them. In the early 1990s, as actuaries grappled with the financial issues surrounding various natural catastrophes, the Actuarial Standards Board recognized and acted on the need for such a standard.

Specifically, Hurricane Andrew in 1992 and the Northridge Earthquake in 1994 led actuaries involved in evaluating hurricane and earthquake exposures to recognize the need to improve actuarial methods. In recognition of this need, many actuaries began using stochastic computer simulation models for their actuarial analysis of hurricane and earthquake loss costs. Comput­er simulation models had been commonly used for some time by actuaries and others for the purpose of evaluating probable maximum loss, but had not been widely used for ratemaking for that exposure.

Computer simulation models are now widely used by actuaries for calculating expected losses due to hurricane, earthquake, and other natural perils. These models incorporate numerous meteorological, seismological, or engineer­ing assumptions—areas clearly outside the expertise of most actuaries.

Comments

Click here to view comments in their entirety