Navigating IRB Submission Forms & Resources with Dr. Kelly Quesnelle
Explore the essential criteria for human subjects research, the process of applying to the IRB for determinations, understanding regulated research, and common errors in exemption submissions. Delve into expedited vs. full board reviews and different categories of research review, in this detailed guide by IRB Chair Dr. Kelly Quesnelle.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Navigating IRB Submission Forms & Resources Kelly Quesnelle Ph.D., IRB Chair
Participants will be able to: List the three criteria that must be met for study to be considered human subjects research and describe how to apply to the IRB for a determination of whether a study is human subjects research. List the two criteria that must be met for quality improvement and program evaluation studies to be considered regulated research and describe how to apply to the IRB for a determination of whether a study is regulated research. List the six categories of regulated research that are exempt from IRB review and describe how to apply to the IRB for an exemption determination. Identify the three most common errors that investigators make when submitting research for an exemption determination. Compare expedited reviews and full board reviews with a specific focus on documents required for IRB submission and categories of research that are eligible for expedited review. 1. 2. 3. 4. 5.
Not Regulated Research Not Human Subjects Research (NHSR) Quality Assurance/Quality Improvement (QA/QI) Not Regulated Research Regulated Research Exempt from IRB Review Studies Requiring IRB Review convened board or expedited
Not Regulated Research Not Human Subjects Research (NHSR) Quality Assurance/Quality Improvement (QA/QI) Not Regulated Research Review for Determination & Basic Ethical Considerations Regulated Research Exempt from IRB Review Studies Requiring IRB Review convened board or expedited Full Review for Approval
Not Regulated Research Not Human Subjects Research (NHSR) Quality Assurance/Quality Improvement (QA/QI) Not Regulated Research Review for Determination & Basic Ethical Considerations Regulated Research Exempt from IRB Review Studies Requiring IRB Review convened board or expedited Full Review for Approval
ALL OF THE FOLLOWING CRITERIA MUST BE MET TO BE CONSIDERED HUMAN SUBJECTS RESEARCH: 1. The research involves living individuals. 2. The investigator will obtain data or information about those individuals. 3. The investigator will obtain EITHER of the following: Data through intervention or interaction with the individuals. Intervention includes both physical procedures by which data are gathered (for example, venipuncture) and manipulations of the subject or the subject's environment that are performed for research purposes. Interaction includes communication or interpersonal contact between investigator and subject. Identifiable private information. Private information includes information about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is taking place, and information which has been provided for specific purposes by an individual and which the individual can reasonably expect will not be made public (for example, a medical record). Private information must be individually identifiable (i.e., the identity of the subject is or may readily be ascertained by the investigator or associated with the information) in order for obtaining the information to constitute research involving human subjects. *FDA has additional definitions and requirements. Please consult SOPs for additional information on FDA-regulated research.
You are conducting a study to determine the rate of opioid-related deaths in Kalamazoo county. You request a de- identified list of records from the medical examiner s office for decedents who have postmortem toxicology reports on file and tested positive for opioids or their metabolites.
You are conducting a study to determine the rate of opioid-related deaths in Kalamazoo county. You request a de-identified list of records from the medical examiner s office for decedents who have postmortem toxicology reports on file and tested positive for opioids or their metabolites. YES. This study would qualify for a designation of Not Human Subjects Research because it does not involve living individuals.
You are conducting a study to determine whether medical students prefer lectures or simulations. You will add a question to the normal course survey and evaluate the results of that survey question. The data will be de-identified through an honest broker in educational affairs.
You are conducting a study to determine whether medical students prefer lectures or simulations. You will add a question to the normal course survey and evaluate the results of that survey question. The data will be de-identified through an honest broker in educational affairs. NO. This study would NOT qualify for a designation of Not Human Subjects Research because it does involve interaction with living individuals. Although you will not be directly interacting with the study subjects, the intervention (that specific question on the survey) would not be occurring if it were not for the research study. Therefore, this is considered indirect interaction with living individuals for the purposes of the research study.
Institutional Approval and Assurance Form Detailed Description of Project/Protocol
Not Regulated Research Not Human Subjects Research (NHSR) Quality Assurance/Quality Improvement (QA/QI) Not Regulated Research Review for Determination & Basic Ethical Considerations Regulated Research Exempt from IRB Review Studies Requiring IRB Review convened board or expedited Full Review for Approval
Quality Improvement (QI) - involves systematic activities that are designed and implemented by an organization to monitor, assess, and improve the quality of its services, processes, or programs. Quality Assurance/Program Evaluation (QA) individual systematic activities conducted to assess how well a program is working and why. QA/QI studies may be REGULATED RESEARCH or NOT REGULATED RESEARCH.
Common rule definitions state that regulated research must meet BOTH criteria: 1. Systematic investigations involving a prospective plan which incorporates data collection, either quantitative and/or qualitative, and data analysis to answer a question 2. Studies designed to develop or contribute to generalizable knowledge or to draw general conclusions, inform policy, or generalize findings *FDA has additional definitions and requirements for test articles being used outside the course of medical practice. Please consult SOPs for additional information on FDA-regulated research.
What is the purpose of the study? Studies designed to inform best practices are generally regulated research Studies designed to implement best practices and report outcomes internally are generally not regulated research Studies designed to implement best practices and report outcomes broadly (e.g. national meetings, professional association journals) may be considered regulated research because the purpose is to generalize the evaluation of best practices implementation What contribution will the study make to the study institution? To other institutions? Will the results be disseminated outside the institution? Generally, broad dissemination suggests that a study may be considered regulated research, although dissemination alone is not sufficient for a determination of regulated research Has the intervention already occurred? Is participation voluntary or is this a mandatory process change at the institutional level? Is this a controlled study?
Drug X and Drug Y are both FDA approved for the same indication. Drug X costs substantially more than Drug Y, and recent literature suggests hospital-led initiatives to increase prescriptions of Drug Y can save hospitals money and patient outcome remains the same as with Drug X prescriptions. The hospital implemented prescriber education to promote the prescriptions of Drug Y over Drug X when medically appropriate. You want to examine patient records to determine whether overall cost and days on treatment remained the same before and after the hospital- implemented education. You will present your findings to the hospital and also at your academic institution.
Drug X and Drug Y are both FDA approved for the same indication. Drug X costs substantially more than Drug Y, and recent literature suggests hospital-led initiatives to increase prescriptions of Drug Y can save hospitals money and patient outcome remains the same as with Drug X prescriptions. The hospital implemented prescriber education to promote the prescriptions of Drug Y over Drug X when medically appropriate. You want to examine patient records to determine whether overall cost and days on treatment remained the same before and after the hospital- implemented education. You will present your findings to the hospital and also at your academic institution. YES. This study would qualify for a designation of QA/QI-Not Regulated Research because it does not contribute to generalizable knowledge. NOTE: HIPAA Authorizations or Waivers are not required in this setting because, by definition, QA/QI studies fall within the realm of organizational operations. The IRB and/or the institution may suggest additional safeguards to maintain privacy and confidentiality of patient data. Informed Consent or a Waiver of Informed Consent is not required in this setting, although consent and voluntary participation should always be the standard for research studies unless mandated otherwise by study design. Patient information sheets that inform patients of their participation in the research and the goals of the research are a plausible idea when a full consent process is not being considered.
Drug X and Drug Y are both FDA approved for the same indication. Drug X costs substantially more than Drug Y, and recent literature suggests hospital-led initiatives to increase prescriptions of Drug Y can save hospitals money and patient outcome remains the same as with Drug X prescriptions. The hospital implemented prescriber education to promote the prescriptions of Drug Y over Drug X when medically appropriate. You want to examine patient records to determine whether overall cost and days on treatment remained the same before and after the hospital-implemented education to evaluate the effectiveness of this switch in the setting of a community-based medical center. You will present your findings to the hospital, at your academic institution, and are hoping to publish in your professional association s journal under the study title, Evaluation of Therapeutic Modifications at a Community-Based Medical Center.
Drug X and Drug Y are both FDA approved for the same indication. Drug X costs substantially more than Drug Y, and recent literature suggests hospital-led initiatives to increase prescriptions of Drug Y can save hospitals money and patient outcome remains the same as with Drug X prescriptions. The hospital implemented prescriber education to promote the prescriptions of Drug Y over Drug X when medically appropriate. You want to examine patient records to determine whether overall cost and days on treatment remained the same before and after the hospital-implemented education to evaluate the effectiveness of this switch in the setting of a community-based medical center. You will present your findings to the hospital, at your academic institution, and are hoping to publish in your professional association s journal under the study title, Evaluation of Therapeutic Modifications at a Community-Based Medical Center. NO. This study would NOT qualify for a designation of QA/QI-Not Regulated Research because it is designed to evaluate the implementation of best practices in a generalizable setting.
Make the purpose and APPLICATION of the study clear in explicit terms to the reviewer. Describe PRECISELY where you plan to disseminate your results. This will give the reviewer a better indication of whether you intend to generate generalizable results.
Institutional Approval and Assurance Form Detailed Description of Project/Protocol
Not Regulated Research Not Human Subjects Research (NHSR) Quality Assurance/Quality Improvement (QA/QI) Not Regulated Research Review for Determination & Basic Ethical Considerations Regulated Research Exempt from IRB Review Studies Requiring IRB Review convened board or expedited Full Review for Approval
Prisoner studies cannot be exempted. FDA studies can only be exempted for: (1) research which started before July 27, 1981, and either did not require FDA approval before that date, or, was subject to requirements for IRB review prior to that date, and remains subject to review by an IRB which meets FDA requirements; (2) emergency use of a test article, provided any such use is reported to the WMed IRB within 5 working days AND any future use of the test article at WMed is subjected to WMed IRB review; (3) or the taste and food quality evaluation.
Research conducted in established or commonly accepted educational settings, involving normal educational practices, such as: Research on regular and special education instructional strategies. Research on the effectiveness of or the comparison among instructional techniques, curricula, or classroom management methods. *Research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior, unless both of the following apply: Information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects. Any disclosure of the human subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subject s financial standing, employability, or reputation. 1. 2. *Excludes CHILDREN except when research involving observations of public behavior when the investigator does not participate in the activities being observed.
When disclosure of the human subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subject s financial standing, employability, or reputation, THE DATA CAN NOT EVER BE IDENTIFIABLE. Can t be coded. Can t ever contain identifiers. Must be ANONYMOUS. A reasonable person shouldn t be able to identify the subject. Make sure you have enough participants that at least 5-10 individuals will respond with each sensitive response.
When could disclosure of the human subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subject s financial standing, employability, or reputation? Examples: Medical conditions Criminal behavior Protected class information Frequency of medical errors Information about an employee or trainee Perceptions about people in positions of authority
You are considering the addition of an airways session to the M1 curriculum. You conduct a study involving all students during the pulmonary course. You will evaluate medical student performance on intubation in the simulation center before and after an educational intervention. Simulation techs will complete an evaluation of skills assessment, and student names will be on the surveys to permit linking the data from before and after the intervention.
You are considering the addition of an airways session to the M1 curriculum. You conduct a study involving all students during the pulmonary course. You will evaluate medical student performance on intubation in the simulation center before and after an educational intervention. Simulation techs will complete an evaluation of skills assessment, and student names will be on the surveys to permit linking the data from before and after the intervention. YES. This study would qualify for a category 1 exemption, although you may be asked to have additional safeguards in place such as an honest broker to de-identify the data prior to sending it to any investigators who are also involved with the normal curriculum. NOTE: Informed Consent or a Waiver of Informed Consent is not required in this setting, although consent and voluntary participation should always be the standard for research studies unless mandated otherwise by study design. Students should be given the option to participate in the research voluntarily, unless mandated otherwise by study design. Information sheets that inform students of their participation in the research and the goals of the research are a plausible idea when a full consent process is not being considered.
You are conducting a survey to determine the incidence of unprotected sexual intercourse in Kalamazoo county. You ask local primary care providers to distribute an anonymous paper survey to patients, who can then seal the survey in a provided envelope and place it in a locked drop box when exiting the office. The drop box will be emptied monthly by study staff.
You are conducting a survey to determine the incidence of unprotected sexual intercourse in Kalamazoo county. You ask local primary care providers to distribute an anonymous paper survey to patients, who can then seal the survey in a provided envelope and place it in a locked drop box when exiting the office. The drop box will be emptied monthly by study staff. NO. This study would not qualify for a category 2 exemption, because patients under the age of 18 may participate. If the inclusion criteria was changed to patients over the age of 18 then this study would quality for a category 2 exemption. NOTE: Informed Consent or a Waiver of Informed Consent is not required in this setting, although consent and voluntary participation should always be the standard for research studies unless mandated otherwise by study design. Patient information sheets that inform patients of their participation in the research and the goals of the research are a plausible idea when a full consent process is not being considered.
Research involving the use of educational tests (cognitive, diagnostic, aptitude, or achievement), survey procedures, interview procedures, or observation of public behavior that is not exempt under the preceding, if either of the following apply: The human subjects are elected or appointed public officials or candidates for public office. Federal statute(s) require(s) without exception that the confidentiality of the personally identifiable information will be maintained throughout the research and thereafter. Research involving the collection or study of EXISTINGDATA, documents, records, pathological specimens, or diagnostic specimens, if these sources are publicly available or if the information is recorded by the investigator in such a manner that subjects cannot be identified, directly or through identifiers linked to the subjects. In order to be eligible for this exemption, all of the materials have to exist at the time the research is proposed. 3. 4.
DATA MUST EXIST at the date of IRB submission. The date range from which the data or specimens will be drawn must be clearly delineated in the protocol. The following must be true of TEMPORARY IDENTIFIERS: The information that will be accessed for the research comes from one source, for example, outpatient clinic records. A list of identifierswithout any associated information, will be used for the purpose of identifying records or specimens that might be necessary to conduct the research analysis. A separate document or dataset, without identifiable information, will be created and used for analysis. This dataset or document will be stored separately from the list of identifiers. The list of identifiers will be destroyed immediately after data collection is complete.
Your pharmacy is trying to improve patient compliance with medications by implementing a phone system that automatically calls patients with refill reminders. The auto-call phone system was implemented one year ago. You would like to review pharmacy records of patient refills from 2 years ago through 18 months ago as a control (before implementation), and compare them with patient refill records from 6 months ago until yesterday (after implementation). You will collect a list of medical record numbers and linked study record numbers on a spreadsheet that will be destroyed prior to data analysis. The data analysis spreadsheet will only contain the study record number and no other identifiers will be recorded. You plan to publish the efficacy of your auto-call system in the journal Innovations in Pharmaceutical Interventions, because you think your system may be helpful for other community-based clinics.
Your pharmacy is trying to improve patient compliance with medications by implementing a phone system that automatically calls patients with refill reminders. The auto-call phone system was implemented one year ago. You would like to review pharmacy records of patient refills from 2 years ago through 18 months ago as a control (before implementation), and compare them with patient refill records from 6 months ago until yesterday (after implementation). You will collect a list of medical record numbers and linked study record numbers on a spreadsheet that will be destroyed prior to data analysis. The data analysis spreadsheet will only contain the study record number and no other identifiers will be recorded. You plan to publish the efficacy of your auto-call system in the journal Innovations in Pharmaceutical Interventions, because you think your system may be helpful for other community-based clinics. YES. This study would qualify for a category 4 exemption. All data exists at the time of IRB submission, the data is all coming from one source, a temporary list of identifiers will be maintained only during data collection (destroyed prior to data analysis), and identifiers will never be stored with data. HIPAA Authorizations or Waivers ARE required in this setting. Informed Consent or a Waiver of Informed Consent is NOT required in this setting, although consent and voluntary participation should always be the standard for research studies unless mandated otherwise by study design.
Research and demonstration projects which are designed to study, evaluate, or otherwise examine (i) Federal public benefit or service programs, (ii) procedures for obtaining benefits or services under those programs; (iii) possible changes in or alternatives to those programs or procedures; or (iv) possible changes in methods or levels of payment for benefits or services under those programs; if: The projects are conducted by or subject to the approval of Federal Department or Agency heads, and The program under study must deliver a public benefit (e.g., financial or medical benefits as provided under the Social Security Act), and There is no statutory requirements for IRB review, and The research does not involve significant physical invasions or intrusions upon the privacy of subjects, and The exemption is invoked with authorization or concurrence by the funding agency 5. Taste and food quality evaluation and consumer acceptance studies, if: Wholesome foods without additives are consumed; or A food is consumed that contains a food ingredient at or below the level and for a use found to be safe, or agricultural chemical or environmental contaminant at or below the level found to be safe, by the Food and Drug Administration or approved by the Environmental Protection Agency or the Food Safety and Inspection Service of the U.S. Department of Agriculture 6.
Category 2 1. Surveys with sensitive or potentially damaging information that is not anonymous Category 4 2. Records research where all the data does not exist at the time of IRB submission 3. Records research where linking codes to identifiers are not destroyed immediately following data collection, prior to data analysis
You are a nurse manager surveying nurses in your department about their perceptions of a new patient intake form that will be implemented next month. You will administer the same survey before and after implementation of the form, and employee ID numbers will be used to link the survey data from before and after implementation of the new system. One of the questions on the survey asks nurses about their perception of the current intake system, and another question asks them about efficiency of unit management during the intake process. An honest broker system will be used to remove identifiers before sending the data to you for analysis after the last survey has been administered.
You are a nurse manager surveying nurses in your department about their perceptions of a new patient intake form that will be implemented next month. You will administer the same survey before and after implementation of the form, and employee ID numbers will be used to link the survey data from before and after implementation of the new system. One of the questions on the survey asks nurses about their perception of the current intake system, and another question asks them about efficiency of unit management during the intake process. An honest broker system will be used to remove identifiers before sending the data to you for analysis after the last survey has been administered. NO. This study would NOT qualify for a category 2 exemption, because data of this nature (potentially damaging) cannot EVER be identifiable. It would be recommended to complete the full study application complete with a full consent form for participants.
You want to evaluate narcotics prescribing in your clinic. You will conduct a mandatory educational intervention, and survey prescribers in the department anonymously before and after the intervention. Additionally, you will look at overall prescribing trends sampled twice: one year ago and next year, one year after the intervention.
You want to evaluate narcotics prescribing in your clinic. You will conduct a mandatory educational intervention, and survey prescribers in the department anonymously before and after the intervention. Additionally, you will look at overall prescribing trends sampled twice: one year ago and next year, one year after the intervention. NO. This study would not qualify for exemption because it contains records that do not already exist at the time of IRB submission. A full study application would be required. HIPAA Authorizations or Waivers ARE required in this setting.
Institutional Approval and Assurance Form Study Protocol Include surveys, scripts, and/or spreadsheets Determination for Exempt Application CITI Training for All Study Team Members CV of PI and Co-Investigators only Study Specific Conflict of Interest Forms HIPAA Authorization or Waiver as applicable
Not Regulated Research Not Human Subjects Research (NHSR) Quality Assurance/Quality Improvement (QA/QI) Not Regulated Research Review for Determination & Basic Ethical Considerations Regulated Research Exempt from IRB Review Studies Requiring IRB Review convened board or expedited Full Review for Approval
Institutional Approval and Assurance Form Study Protocol Include surveys, scripts, and/or spreadsheets IRB Initial Study Application CITI Training for All Study Team Members CV of PI and Co-Investigators only Study Specific Conflict of Interest Forms Additional Supplemental Forms HIPAA Waiver, Waivers of Informed Consent, Vulnerable Populations, etc. as determined during application process
(1) Clinical studies of drugs and medical devices only when condition (a) or (b) is met. (a) Research on drugs for which an investigational new drug application (21 CFR Part 312) is not required. Research on marketed drugs is not eligible for expedited review if the research significantly increases the risks or decreases the acceptability of the risks associated with the use of the product. (b) Research on medical devices for which (i) an investigational device exemption application (21 CFR Part 812) is not required; or (ii) the medical device is cleared/approved for marketing and the medical device is being used in accordance with its cleared/approved labeling.
(2) Collection of blood samples by finger stick, heel stick, ear stick, or venipuncture meeting one or both of the following conditions: (a) Collection from healthy, non-pregnant adults who weigh at least 110 pounds. For these subjects, the amounts drawn may not exceed 550 mL in an 8-week period, and collection may not occur more frequently than 2 times per week; or (b) Collection from other adults and children, considering the age, weight, and health of the subjects, the collection procedure, the amount of blood to be collected, and the frequency with which it will be collected. For these subjects, the amount drawn may not exceed the lesser of 50 mL or 3 mL per kg in an 8-week period and collection may not occur more frequently than 2 times per week. Children are defined as "persons who have not attained the legal age for consent to treatments or procedures involved in the research, under the applicable law of the jurisdiction in which the research will be conducted.
(2) Collection of blood samples by finger stick, heel stick, ear stick, or venipuncture meeting one or both of the following conditions: (a) Collection from healthy, non-pregnant adults who weigh at least 110 pounds. For these subjects, the amounts drawn may not exceed 550 mL in an 8-week period, and collection may not occur more frequently than 2 times per week; or (b) Collection from other adults and children, considering the age, weight, and health of the subjects, the collection procedure, the amount of blood to be collected, and the frequency with which it will be collected. For these subjects, the amount drawn may not exceed the lesser of 50 mL or 3 mL per kg in an 8-week period and collection may not occur more frequently than 2 times per week. Children are defined as "persons who have not attained the legal age for consent to treatments or procedures involved in the research, under the applicable law of the jurisdiction in which the research will be conducted.
(3) Prospective collection of biological specimens for research purposes by noninvasive means. Examples include: (a) Hair and nail clippings in a non-disfiguring manner. (b) Deciduous teeth at time of exfoliation or if routine patient care indicates a need for extraction. (c) Permanent teeth if routine patient care indicates a need for extraction. (d) Excreta and external secretions, including sweat. (e) Uncannulated saliva collected either in an unstimulated fashion or stimulated by chewing gum base or wax or by applying a dilute citric solution to the tongue. (f) Placenta removed at delivery. (g) Amniotic fluid obtained at the time of rupture of the membrane prior to or during labor. (h) Supragingival and subgingival dental plaque and calculus, provided the collection procedure is not more invasive than routine prophylactic scaling of the teeth and the process is accomplished in accordance with accepted prophylactic techniques. (i) Mucosal and skin cells collected by buccal scraping or swab, skin swab, or mouth washings. (j) Sputum collected after saline mist nebulization. (k) Vaginal swabs that do not go beyond the cervical os; rectal swabs that do not go beyond the rectum; and nasal swabs that do not go beyond the nares.
(4) Collection of data through noninvasive procedures, not involving general anesthesia or sedation, routinely employed in clinical practice, excluding procedures involving x-rays or microwaves. Where medical devices are employed, they must be approved for marketing. Studies intended to evaluate the safety and effectiveness of the medical device are not generally eligible for expedited review, including studies of approved medical devices for new indications. Examples include: (a) Physical sensors that are applied either to the surface of the body or at a distance and do not involve input of significant amounts of energy into the subject or an invasion of the subject s privacy. (b) Weighing or testing sensory acuity. (c) Magnetic resonance imaging. (d) Electrocardiography, electroencephalography, thermography, detection of naturally occurring radioactivity, electroretinography, ultrasound, diagnostic infrared imaging, Doppler blood flow, and echocardiography. (e) Moderate exercise, muscular strength testing, body composition assessment, and flexibility testing where appropriate given the age, weight, and health of the individual.
(5) Research involving materials (data, documents, records, or specimens) that have been collected, or will be collected solely for non-research purposes (such as medical treatment or diagnosis). (6) Collection of data from voice, video, digital, or image recordings made for research purposes. (7) Research on individual or group characteristics or behavior (including, but not limited to, research on perception, cognition, motivation, identity, language, communication, cultural beliefs or practices, and social behavior); or research employing survey, interview, oral history, focus group, program evaluation, human factors evaluation, or quality assurance methodologies.
(5) Research involving materials (data, documents, records, or specimens) that have been collected, or will be collected solely for non-research purposes (such as medical treatment or diagnosis). (6) Collection of data from voice, video, digital, or image recordings made for research purposes. (7) Research on individual or group characteristics or behavior (including, but not limited to, research on perception, cognition, motivation, identity, language, communication, cultural beliefs or practices, and social behavior); or research employing survey, interview, oral history, focus group, program evaluation, human factors evaluation, or quality assurance methodologies. Additional categories for research reviewed at continuing review.
