Main

The goals of the Office of Rare Diseases (ORD) of the National Institutes of Health (NIH) are to stimulate and coordinate research on rare diseases, and to support research that responds to the needs of individuals who have any one of the more than 6,000 rare diseases known today. In 2003, ORD became aware that as many as one-quarter of the tests for rare genetic conditions (approximately 20% in July 2007) failed to move from the research laboratory to the clinical diagnostic laboratory setting and approximately 22% of clinical testing was only available outside the United States in non-Clinical Laboratory Improvement Amendment (CLIA)-certified laboratories (approximately 19% in July 2007) (unpublished data, GeneTests 2003, 2007). In response to limited test translation, some research laboratories without the required CLIA certification were providing test results obtained in a research setting to individuals, families, and clinicians. Many researchers working in non-CLIA-certified laboratories feel morally obligated to provide testing when it is not available in a clinical laboratory. In addition to lacking federal CLIA certification, research laboratories are not prepared to provide the full spectrum of clinical laboratory services. Often they cannot perform tests in a timely manner, do not have staff trained in providing and explaining clinical results, and do not have the resources in their grant funding to support testing as a clinical service (e.g., developing quality assurance standards).1

ORD and the Centers for Disease Control and Prevention Division of Laboratory Systems recognized that the slow rate of test translation resulted in limited access and potential test quality concerns. They agreed to work together to increase discussion on this topic and establish ways to encourage test translation by clinical laboratories. A working group was developed and quickly expanded to include representatives from Emory University, American Society of Human Genetics, American College of Medical Genetics, Society of Inherited Metabolic Disorders, Genetic Alliance, and others. The membership of the original volunteer Steering Committee included: Stephen Groft, Giovanna Spinella, Joe Boone, Bin Chen, Andy Faucett, Joann Boughman, Sharon Terry, Michelle Puryear, and Mike Watson. A series of invited expert meetings, Federal Agency meetings, and public meetings were held from 2004 to 2007.

In 2004 and 2005, ORD developed a pilot program within the NIH intramural program to translate sequence-based molecular rare disease genetic tests solely at the request of NIH researchers. Twenty-two tests were translated and eight laboratories participated (see Table 1). With the insight gained from the series of rare disease meetings and the NIH intramural program as proof of laboratory willingness and capability, ORD in collaboration with the Centers for Disease Control and Prevention and other partners developed the Collaboration, Education, and Test Translation Program (CETT Program) as a pilot rare genetic disease test translation model that would be open to all interested parties and would work to facilitate rare disease genetic test translation. The CETT Program was presented in draft form to a trans-NIH rare disease working group of representatives of NIH Institutes, Centers and Offices in 2005 and at the September 26–27, 2005 national meeting, “Access to Quality Testing for Rare Diseases: A National Conference” in Washington, DC.1 The program was endorsed by attendees of both meetings. In addition, the CETT Program was vetted at the Secretary's Advisory Committee on Genetics, Health, and Society and the Advisory Committee on Heritable Disorders and Genetic Diseases in Newborns and Children.

Table 1 List of intramural tests and labs
Full size table

The CETT Program is a multiple-year pilot test translation program that focuses on clinical test development for rare genetic diseases currently not available from a CLIA-certified laboratory. Funding provided by the program is intended to augment laboratory development funds and stimulate test translation. The individual patient cost of each test is also reviewed for reasonableness. The CETT Program encompasses a “social context” for genetic testing, and aspires to be integrated across the NIH over time. It is expected to become a model for rare disease test translation, regardless of funding source(s). In its pilot state, the CETT Program is a working model structured to allow continuous improvement.2

RATIONALE, OBJECTIVES, AND PHILOSOPHY OF THE CETT PROGRAM

The basis for developing the CETT Program arose from the discussions held at the series of rare disease meetings from 2004 to 2005 and lessons learned from the NIH Intramural test translation project. The major points include (1) most clinical laboratories performing rare genetic disease testing have limited funds for development of new tests and limited personnel dedicated to new test development/test translation; (2) most clinical laboratories do not have the resources needed for clinical data collection and educational material development; and (3) many researchers are not aware of clinical laboratories that are willing and able to translate testing developed in their research laboratory.

The major objectives of the CETT Program are (1) to promote the development of new genetic tests for rare diseases; (2) to facilitate the translation of genetic tests from research laboratories to clinical practice; (3) to establish collaborations and to provide education about each rare genetic disease, related genetic research, and the clinical impact of testing; and (4) to support the collection and storage of genetic test result information in publicly accessible databases to leverage the information into new research and new treatment possibilities.

The CETT Program's guiding philosophy states that all parties benefit when (1) the quality of testing for rare disorders meets or exceeds existing standards; (2) clinical laboratories, researchers, clinicians, and disease specific advocates collaborate; and (3) high-quality educational materials explain what the test can and cannot do and how best to use the test results.

KEY ISSUES IN DEVELOPMENT OF THE CETT PROGRAM

In developing the CETT Program, the following strategic issues became apparent:

  • Many research laboratories, patient groups, clinicians, and institutional review boards received incorrect guidance from multiple sources and did not understand that genetic test results are clinical information and can only be given to an individual and/or clinician through a CLIA-certified laboratory.

  • Clinical and genotype information are critical for interpretation of test results but are often not collected and/or stored when testing becomes a clinical service, making genotype–phenotype correlation impossible. This genotype–phenotype information may improve understanding of the rare disease manifestations and natural history, and guide future research that may lead to targeted treatments.

  • Expertise provided by the research laboratory and/or clinical researcher is often needed for quality test interpretation specifically for newly identified mutations, recessive mutation combinations not seen during the research testing phase, or variants of unknown significance.

  • Expertise is needed for clinical consultation and to assure patient referral to appropriate clinical services and research opportunities.

  • Educational materials for clinicians and individuals/families with a rare disease are needed for recipients to understand the test ordering process, the benefits, and limitations of a genetic test, how to interpret test results, and available expertise and resources must be available and written at the level for the target audience.

  • The same level of quality assurance and quality control should be provided for low-volume rare disease testing as for other diagnostic tests, although the standard quality assurance/quality control procedures used in high-throughput testing may not be possible.

  • Obtaining third-party reimbursement for genetic testing is often a concern, even when the testing is provided by a CLIA-certified laboratory.

The first goal of the CETT Program was to develop a model process to determine “when a test is ready for prime time” through internal review by CETT Program staff and external review by an expert Review Board that would consider information in the peer-reviewed published literature and the predicted mutation detection rate. This model would provide an alternative to the current process in which test translation is the sole domain of an individual laboratory director. With rare diseases for which no clinical testing is available, the absolute mutation detection rate is one factor to consider, but it may be less important than the potential benefit of an improved diagnostic, carrier, prenatal, and preimplantation genetic diagnosis test on the current health care paradigm.

The second goal of the CETT Program is to support the notion of “truth in advertising” by requiring descriptions of the test and test result reporting that (1) clearly explain what is known about the mutation detection rate of the test; (2) explains the limitations of the test itself; (3) helps the patient and clinician understand the meaning of test results; and (4) places the test results in the context of patient care.

The CETT Program model requires applicants interested in translating a test to form a Collaborative Group composed of a clinical laboratory, a researcher, an expert clinician, and a disease-specific advocacy organization or advocate. The collaboration may be led by any one of the members. The following models for test translation collaborations were considered in developing the CETT Program: (1) a research laboratory obtains CLIA certification and sets up a clinical laboratory that offers the test based on clinical need, not research need (fee-based service); (2) a local clinical laboratory works with the researcher to develop a clinically available test; and (3) a researcher and disease-specific advocate are linked with a clinical laboratory experienced in rare disease test development. A group of laboratories that provide this service to all interested groups are the members of the National Laboratory Network (Table 2). The CETT Program also supports the development of additional laboratory networks for biochemical or other testing methodologies that would collaborate with a researcher and disease-specific advocacy group.

Table 2 List of National Laboratory Network laboratories
Full size table

INTERNAL REVIEW PROCESS

Applications are accepted monthly and the goal is to provide feedback to the Collaborative Group within 2 to 3 months.

Before receipt of the completed application, the CETT Program Coordinator (see Table 3) speaks directly with potential applicants to assure that they understand the purpose of the CETT Program and the application and review process. As early in the application process as feasible, the applicants are encouraged to contact the National Center for Biotechnology Information (NCBI) of the National Library of Medicine, telephone interviews are held with the disease-specific advocate and a mentor is assigned.

Table 3 CETT program staff
Full size table

Each complete application is reviewed by the CETT Program Scientific Advisor for completeness: scientific evidence, methodology, and laboratory experience. Simultaneously, the application is reviewed by the CETT Program Coordinator to assure that all members of the Collaborative Group are in place. If indicated, he interviews by telephone the members of the Collaborative Group to answer their questions about the process and to assure that members understand their roles. He also reviews the application for completeness regarding data collection plans, test result report forms, and examples of educational materials and plans for their development. The CETT Program staff interacts continuously with applicants to resolve issues and concerns as they arise.

Each application is reviewed by the National Library of Medicine/NCBI Liaison to assist CETT Collaborative Groups with the initial design of their data collection plan, online clinical sheet, and with organizing and storing genotype and clinical data collected from individuals evaluated by the proposed new test.

Once the internal review is complete, the application is either returned to the applicants by the CETT Program Coordinator with specific concerns to be addressed before resubmission or forwarded to the Review Board.

EXPERT REVIEW—REVIEW BOARD COMPOSITION AND ROLE

The Review Board (see Table 4) was originally comprised of 15 members, three from each of five groups representing the non-geneticist clinician, clinical genetics professional, laboratory geneticist, researcher, and patient advocate communities. After a meeting on biochemical testing on October 6–7, 2006, three additional biochemical experts were added, one to each review team (see Table 4). Two Review Board organizational meetings were held in December 2005 and February 2006 to establish the standards for the review process. The first annual meeting of the entire Review Board and invited experts was held on March 5–6, 2007, to evaluate the overall review process, to evaluate the CETT Program, and to provide guidance on specific issues identified in the review process including, but not limited to, assay validation and evaluation of variants of unknown significance. Annual Review Board meetings are planned.

Table 4 The CETT program review board members
Full size table

The role of the Review Board is advisory to the CETT Program staff as to whether an application meets the goals of the CETT Program. Each application is reviewed independently by a panel of six members, one from each of the representative groups. Conflicts of interest are identified before assignment of applications and Review Board members sign a conflict of interest form at the onset of their term. The Review Board panel evaluates each application using specific review criteria and shares their observations by conference call. The Review Panel advises the CETT Program staff as to whether an application meets the goals of the CETT Program and provides feedback to potentially improve the balance in the collaborative group, the proposed test, testing process, test reports, data collection plan, and educational materials.

REVIEW CRITERIA

The CETT Program internal review and Review Board evaluation focus on scientific evidence, proposed methodology, impact on health care, laboratory qualifications, data collection, educational materials, and evidence of collaboration.

When looking at scientific evidence, the reviewers consider:

  • How many genes cause the disorder?

  • What percentage of patients with the disorder have mutations in the gene for which testing is proposed?

  • What percentage of patients will be identified using the proposed testing method compared with current testing methods? Are other methods of diagnosis available, which the proposed test would replace or complement?

The proposed methodology of the test must meet specific goals:

  • Is the test translation approach efficient and does it meet the CETT Program budget guidelines? Do the proposed individual sample fees seem reasonable?

  • How will unusual results, such as variants of unknown significance, be evaluated and reported?

  • If mutation screening is used, how will negative results be evaluated?

  • How will the test be validated? Are positive and negative control samples available?

  • Will testing be available in all formats needed by the community—diagnostic, carrier, prenatal, and preimplantation genetic diagnosis?

The test must provide a positive impact on health care:

  • What are the indications for testing?

  • How will the proposed test change the current diagnostic pathway?

  • Could establishing the correct diagnosis using the proposed test reduce unnecessary diagnostic testing and/or facilitate genetic counseling?

  • Could early diagnosis reduce morbidity and/or mortality?

Certain laboratory qualifications are essential:

  • Laboratory Director's certification

  • CLIA or other certification of laboratory

  • Number of disorders currently tested by the laboratory

  • Staffing for clinical-laboratory interface: are genetic counselors or physician consultants available?

Data collection and storage must be included in the plan:

  • Is the clinical information necessary for test result interpretation collected on a short form (one page or less) at the time test is ordered?

  • Are all members of the Collaborative Group willing to submit a subset of deidentified clinical and genotype information into a publicly accessible database?

  • Are multiple pathways for collection of clinical information presented; i.e., are some clinical data collected at the time of sample submission? Are options also available on the advocacy website? Are procedures in place for more detailed data to be collected by the researcher and/or advocate group for interested patients?

  • Is there evidence of a willingness to work with NCBI to develop data collection forms that will reflect current data models and facilitate clinical and genotype data submission to public databases?

Educational materials play an important role in the test translation:

  • Is there a plan to develop educational materials about the disease and how the test is used in patient care for three audiences—medical geneticists, nongenetic clinicians, and patients? Are the roles of each member of the Collaborative Group appropriate and clear in the plan for development of educational materials?

  • Do the test result report forms for negative, positive, or indeterminate results explain clearly the results of the test, the implications of the test result for the person tested, and the limitations of the test itself?

  • Has the Collaborative Group agreed to write a Gene Review within 1 year or provide suggested updates to a current one to include testing information?

Each collaborative group must show evidence of collaboration among the members:

  • Do all participants have an active role in the test translation? This includes developing the test, interpreting the results, developing test result report forms and educational materials for the three target audiences, and the clinical and genotype data collection process

  • Interviews are performed by the CETT Program staff to clarify roles and a CETT Program Advocacy Mentor may be assigned

  • Is it clear how patients will be referred by the clinical laboratory to the researcher to leverage new information from the clinical setting to promote new research discoveries?

  • Is the role of the disease specific advocacy group clear? Are they considered true partners in the collaboration? Are they engaged in the development and dissemination of educational materials? Are they serving as a resource for patients and families?

PROVIDING FEEDBACK TO THE CETT PROGRAM

Each Collaborative Group agrees to provide an annual report on the genetic testing experience for 5 years. Information is requested not only on the number of tests requested and types of results, but also on the value of the collaboration, such as the interpretation of the test results and understandability of the information on the report form, the benefit of the educational materials, the ability of individuals and families with rare genetic diseases to find appropriate resources, the number and types of genetic results that required referral to the research laboratory for further investigation, the effect of the collaboration on ongoing clinical studies, and the effect, to the degree possible, on betterment of clinical care and services. As part of the application the laboratory has agreed to provide genetic testing for a minimum of 5 years. If a clinical laboratory cannot fulfill the 5-year commitment, plans to transition the test to another clinical diagnostic laboratory should be included in the annual report.

EARLY CHANGES TO THE CETT PROGRAM MODEL

When the CETT Program began to accept applications from collaborative groups in early 2006, it was quickly recognized that the collaborative groups needed additional support. The CETT Program formed partnerships to provide this support, which includes the following.

Data collection and storage: NCBI

The CETT Program requires each Collaborative Group to develop a plan to store limited deidentified clinical and genotype information necessary to improve the interpretation of the genetic test result and to increase the understanding of the phenotypic spectrum of the rare disease. To meet this need, the CETT Program partnered with the NCBI to assist CETT Collaborative Groups (applicants) with the initial design of their collection plan. NCBI maintains and distributes public databases, creates analytic tools, and coordinates efforts to gather genomic information to aid in the understanding of fundamental molecular and genetic processes affecting human health. The CETT Program assists with the organization and storage of genotype and clinical data collected during test translation through an independent ORD-sponsored contract for Health Insurance Portability and Accountability Act (HIPAA)-compliant data management. After an embargo period, which allows members of the Collaborative Group the opportunity to assess the data for publication purposes, deidentified data are uploaded to NCBI. A goal of NCBI in working with the CETT Program is to facilitate the development of standard formats and use of standard vocabularies permitting wider associations and comparisons of clinical information associated with genomic data gathered across time from many sources. NCBI can store the probe sequences used in an assay to place and maintain these data in many ongoing and evolving genomic contexts. By providing this sequence information along with genotype and clinical data in public resources to be shared across the biomedical community, clinical laboratories and investigators will be able to standardize assays, directly compare related results, and promote opportunities for higher level discoveries through bioinformatics mining of previously inaccessible data. Ultimately, it is hoped that this type of access will stimulate new research; encourage collaborations; and lead to treatment and disease management breakthroughs targeted to specific genotypes. The CETT Program staff encourages contact with the NCBI early in the process.

Disease specific advocacy organizations/individuals: CETT Program Advocacy Mentors Program

The CETT Program requires the advocacy participant(s) to take an active role in the test translation process including the development and distribution of educational materials and data collection plans. During its development the CETT Program requested and received agreements from the Genetic Alliance and the National Organization for Rare Disorders to assist the disease specific advocacy members of the CETT Collaborative Groups (applicants). Review of some of the initial applications to the CETT Program revealed that additional support for the advocacy members of the Collaborative Group was needed. For several tests under consideration no disease-specific advocacy group existed, or the group was small and had a limited number of individuals available to participate, or only a single individual or family might be available to join the collaborative group. To provide this additional support for this vital element of the collaboration, the CETT Program staff created the CETT Program Advocacy Mentors Program with the participation of advocacy leaders and genetic counselors experienced with advocacy groups. Now, early in the application process telephone interviews are held with the advocate member of every Collaborative Group and a member of the CETT Program Advocacy Mentors Program may be assigned to work with each Collaborative Group.

RESULTS TO DATE (OCTOBER 2007)

The CETT Program began accepting applications in March 2006. As of October 2007, the CETT Program staff and Review Board have reviewed 30 applications. Twenty-seven applications were approved and suggested improvements were provided to the Collaborative Groups. Current updates can be found at the CETT website (www.cettprogram.org). In the initial reviews, significant concerns were raised about the following issues:

  • How to evaluate variants of unknown significance identified by sequence analysis?

  • How to validate the test assay with limited positive and negative control samples?

  • How to establish the appropriate turnaround time for sequence-based assays?

  • How to improve the format and content of test result report forms that tend to be written for genetics professionals and are often incomprehensible for nonexperts?

  • How best to provide support to disease-specific advocacy groups in the test translation process?

  • How to determine the best format and minimum information needed for the educational materials that describe the disease and the use of testing in patient care?

These issues were discussed at the CETT Review Board and invited expert meeting on March 5–6, 2007. Draft guidelines were developed for internal use in reviewing applications. Guidelines on the evaluation of variants, model clinical report forms, and guidelines for educational materials with examples were posted to the CETT Program website after the meeting with additional guidelines to be posted as developed. Professional organizations were encouraged to review the guidelines and consider changes to current published guidelines.

DISCUSSION

The CETT Program is successfully facilitating the translation of rare disease genetic tests from research settings to clinical laboratories. The CETT Program offers a model of test translation that requires applicants to meet quality standards for laboratory test methods and other supporting activities, such as data collection and educational materials, which may be considered outside the paradigm of current quality testing.

After evaluating the initial applications to the CETT Program, the following suggestions to future applicants are provided:

  • Contact and discuss collaboration with the researcher and disease-specific advocates before developing the test translation plan.

  • Develop concise (one page or shorter) clinical data collection forms.

  • Work with those who facilitate data storage in publicly accessible data bases.

  • Design test result forms to explain results to patients and nongeneticists without the loss of essential laboratory information.

  • Design educational materials to help clinicians and patients understand the potential health care impact and uses of this test in patient care.