About these Reports
KHA AHRQ Reports
AHRQ State Snapshots
Hospital Charge Data
Hospital Compare
QualityCheck (JCAHO)
Quality Links
National Quality Forum
Contact Us
default.asp 

AHRQ

Comments/Questions  for KHA? Contact us

  
  
 Questions & Answers   
                                                                                                

  1. What is in these Quality Reports?
  2. Limitations of the Data
  3. How were these indicators selected?
  4. How do I use these reports?
  5. What does "risk-adjusted" mean?
  6. How often will the data in this report be updated?
  7. Does this quality report display data about individual physicians?
  8. Why isn't my hospital on this report?
  9. Where can I find the technical specifications for the Quality Indicators?
  10. How do I read the Graph?
  11. Two statistical concepts you should understand

    Glossary


    1. What is in these Quality Reports? 
    The Kentucky Hospital Association is displaying the Inpatient Quality Indicators defined by the Agency for Healthcare Research & Quality (AHRQ).  These indicators are a set of measures that provide a perspective on hospital quality of care using hospital administrative data. These indicators reflect quality of care inside hospitals and include inpatient mortality for certain procedures and medical conditions; utilization of procedures for which there are questions of overuse, underuse, and misuse; and volume of procedures for which there is some evidence that a higher volume of procedures is associated with lower mortality. 

    Each Indicator can be grouped by bedsize or Acute/Critical Access status for hospitals in the state based on their results.  If a hospital has a statistically significant number (the variation is not due to random chance) - it will be highlighted - Red is significantly worse than the National average, Green is significantly better. If national data are available for the indicator, that National Rate is listed at the top of the report.  Statewide rates are listed as the first row of the report.

                Please note the following exceptions:

    •    Rehabilitation and Psychiatric hospitals were excluded from the input dataset.

    •    Hospitals that had less than 20 total cases were not shown.

    •    If an Indicator had less than 20 cases statewide, it was not included.

    Back to the Top

     

    2. Limitations of the Data

    Users of these reports must understand that the data are administrative data collected for billing purposes, not clinical data so there are significant limitations to using this data for quality purposes including but not limited to the following:
    • Hospitals are required to submit data within 90 days after the close of a calendar quarter (hospital data submission vendor deadlines may be sooner). Depending on hospitals' collection and billing cycles, not all discharges may have been billed or reported. Therefore, data for each quarter may not be complete. This can also affect the accuracy of source of payment data, particularly self-pay and charity categories, where patients may later qualify for Medicaid or other payment sources.
    • Hospitals record as many as twenty-five diagnosis codes and twenty-five procedure codes for each patient for billing purposes. Data submitted to KHA are limited to nine diagnosis codes and six procedure codes. Therefore, the data submitted may not fully represent all diagnoses treated by the hospital or all procedures performed. A consequence may be that sicker patients with more than nine diagnoses or undergoing more than six procedures are not accurately reflected. This may also result in total volume and percentage calculations for diagnoses and procedures not being complete.
    • Another critical limitation is that diagnosis codes do not distinguish between conditions present at the time of the patient's admission to the hospital and those occurring during hospitalization. This makes it difficult to obtain accurate information regarding things such as complication rates.   It has been proposed to Medicare to add flags on secondary diagnostic codes to distinguish conditions present upon admission.  The "present on admission" code is currently used in California and New York and has been helpful in distinguishing conditions and infections that occurred during the patient’s stay or prior to the hospitalization. There is space reserved on the electronic 837 claim and the paper UB-04 form for this "present on admission" field.
    • AHRQ assigns the Risk of Mortality and Severity of Illness scores using the APR-DRG methodology designed by 3M Corporation. These scores may be affected by the limited number of diagnosis and procedure codes collected by KHA and may be understated.
    • Conclusions drawn from the data are subject to errors caused by the inability of the hospital to communicate complete data due to reporting form constraints, subjectivity in the assignment of codes, system mapping, and normal clerical error. The data are submitted by hospitals as their best effort to meet statutory requirements.

    Back to the Top

    3. How were these indicators selected? 
    National organizations have endorsed lists of indicators and safe practices.  In 2005, KHA formed a joint committee comprised of hospital members who serve on our Data Committee and our patient Safety Committee.  The committee agreed to publish these nationally recognized indicators as a starting place on reporting of hospital quality.  

    Back to the Top

    4. How do I use these reports?
    While the indicators provided here can be useful for choosing a hospital, the reality is that these quality indicators are only one source of information. Other factors that need to be considered include the patient’s health plan coverage, place of residence, location of the patient’s physician, and recommendations from family and friends.
     
    Also keep in mind that doctors direct and oversee the medical care in hospitals, prescribing the tests, medications, and treatments.  These reports do not separate the effect of the doctor from the effect of the hospital. The quality of care provided in a hospital is influenced by how well its doctors, nurses, support staff, and management team work together, as well as by the availability of technology and other resources.  If a major change occurs that affects any of these—for example, the departure of a key surgeon or the addition of new technology—the indicators for a given hospital may change dramatically and rapidly.
     
    Medical practice and standards of care also change as new procedures and medicines become available and as research studies demonstrate the effectiveness of specific treatments or procedures.  Patients should talk with their doctors and hospitals about their care and ask questions about what changes, if any, have occurred that could affect that care.

    Back to the Top
     

    5. What does "risk-adjusted" mean?
    The risk of a complication or death varies by patient and by procedure. For example, an older surgical patient who has complicating illnesses such as kidney failure and diabetes is at greater risk of developing complications than a young, healthy patient. Open heart surgery has a greater risk of a collapsed lung than knee surgery does. The same care, given by the same physician, in the same hospital, might have very different effects on a patient who is healthier than another patient. For example, a patient requiring a heart valve repair who also has an infection resistant to antibiotics is more severely ill entering surgery than a valve repair patient who is otherwise healthy. The severely ill patient may not respond as well to treatment or surgery and, therefore, may have to stay in the hospital longer or may not recover at all.

    Risk adjustment mathematically takes into account differences in patient and procedure risk factors, so that comparisons are more meaningful. Risk adjustment allows for comparison of actual performance with predicted performance, based on the average U.S. hospital.

    Back to the Top

     6. How often will the data on this report be updated?  

    This report will be updated annually as new data become available.

    Back to the Top

    7. Does this Quality Report display data about individual physicians?
    No. We are publishing hospital data only.

    Back to the Top

    8. Why isn't my hospital on this report?
    There are a number of reasons that a hospital may not appear on the report
    • The Hospital does not perform that procedure
    • The Hospital is a specialty hospital (Psych or Rehab)
    • The Hospital had a low volume (< 20) of the procedure
    • The Hospital did not report their data for that time period

    Low volume (small cell size) could impact patient confidentiality and also limit the ability to reliably identify quality differences. Small cell size is a frequent problem in performance measurement, especially when using measures of rare events such as mortality or foreign body left after procedure. Small cell size refers to the occasion when there is a small number of cases within any individual unit of analysis. For example, a single hospital (location unit of analysis) may only have one death (small cell size, number of patients who died = 1) in a year (time unit of analysis). It would be difficult to ensure protection of patient confidentially in this instance.

     Back to the Top

     

    9. Where can I find the technical specifications for the Quality Indicators?
    The technical specifications for the indicator definitions can be found in each http://www.qualityindicators.ahrq.gov/iqi_download.htm. Detailed specifications, including specific ICD-9-CM codes, DRGs, and/or patient age or sex are included.

    The technical specifications for the QI software can be found in each QI module’s software documentation. Detailed specifications, including input file formats, software files, and data processing instructions are provided.

    The User Guide for each QI module (PQI, IQI, and PSI) may be downloaded from the same QI Web site at http://www.qualityindicators.ahrq.gov/iqi_download.htm.

    Back to the Top

     


Glossary

  • All-Patient Refined DRG's (APR-DRG's) - The APR-DRGs are used to risk-adjust the IQIs for patient clinical condition and severity of illness or risk of mortality. If the patient APR-DRG is not available, the software will risk-adjust using information on age and gender only, which is less desirable than using the APR-DRGs. Although the AHRQ QIs program modules are free, the APR-DRGs is a commercially licensed software package that may be obtained from the 3M Corporation. The 3M Corporation did not have any affiliation with development of the AHRQ QIs. Future versions of the AHRQ QIs software may incorporate alternative clinical classification systems.

    Although both APR-DRGs and DRGs are based on a patient’s International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnostic and procedure codes, they are different patient classification schemes. The 3M Corporation developed and sells a widely used software product to define risk adjustment categories called APR-DRGs, which has the advantage of subclass groupings for Severity of Illness and Risk of Mortality. DRGs are designated by the Centers for Medicare and Medicaid Services (CMS) and software groupers are available from various vendors to assign DRGs.
     

  • Diagnosis-Related Groups (DRGs) are a patient classification system based on a patient’s International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnostic and procedure codes, among other variables. DRGs are designated by the Centers for Medicare and Medicaid Services (CMS) . Software groupers are available from various vendors to assign DRGs.
     
  • Confidence Interval

    A range that depicts the likelihood that a hospital's performance could be influenced by random chance.  The confidence interval in this report is a range of numbers (lower and upper confidence limits) around the hospital’s risk-adjusted rate.  (Confidence intervals are commonly reported in opinion polls:  “plus or minus 5%.”)  The confidence interval reflects how sure we can be that the risk-adjusted rate reflects the hospital’s performance on the quality indicator.  For example, a rate based on 20 patients is a less reliable indication of the hospital’s performance than a rate based on 100 patients is.  The confidence intervals in this report are the 95% risk-adjusted confidence intervals from the Agency for Healthcare Research and Quality’s software.

  • Statistical Significance

    A difference is “statistically significant,” if it is unlikely to have happened just by chance.  (For example, having a coin come up heads 6 times in 10 flips is not surprising and is not “statistically significant,” because that could happen just by chance.)  In this report, a hospital’s risk-adjusted rate differs from the U.S. rate more than is likely by random variation, if the U.S. rate does not fall within the hospital’s confidence interval.  This report highlights (in red or green) hospital risk-adjusted rates that are statistically significantly different from the U.S. rate.

  • Mortality Rate Indicators - The mortality rate for patients with a specific procedure or condition is the number of patients who died, divided by the total number of patients. However, because the patients’ age, sex, or severity of condition may increase their risk of death, the death rates for each hospital are adjusted to account for these factors. Other factors—for example, that some hospitals may transfer out all but the most mild or most severe cases—are not accounted for in the risk-adjustment methods used here. Hence, while death rates constitute a more sensitive indicator of quality than mere procedure counts, they too should be considered in tandem with comments submitted by hospitals, as well as with other information about quality of care.


  • Observed Rate - The observed rate is the raw rate from the data provided by the hospital, or simply the percentage of patients with a particular condition or procedure who died. The observed rate is not very useful, because it does not adjust for differences in patient severity of illness.

  • Risk-adjusted rate – This is the rate that best reflects the hospital’s performance on the quality indicator. The risk-adjusted rate mathematically adjusts for the severity and complexity of the hospital’s patients and procedures. Risk adjustment is important, because hospitals may differ in (for example) the risk of death that their patients have before they come to the hospital. Compare the risk-adjusted rates in this report to the U.S. rate. For example, if a hospital’s risk-adjusted rate is five percent higher than the U.S. rate, that means that the hospital’s observed rate is five percent higher than would be expected if the hospital were performing at the U.S. average, adjusted for the hospital’s mix of patients. The Agency for Healthcare Research and Quality, an independent national organization, developed the risk-adjustment software used for this report.

Back to the Top

 

11. Two Statistical Concepts You Should Understand

Confidence intervals

The confidence interval in this report is a range of numbers (lower and upper confidence limits) around the hospital’s risk-adjusted rate.  (Confidence intervals are commonly reported in opinion polls:  “plus or minus 5%.”)  The confidence interval reflects how sure we can be that the risk-adjusted rate reflects the hospital’s performance on the quality indicator.  For example, a rate based on 20 patients is a less reliable indication of the hospital’s performance than a rate based on 100 patients is.  The confidence intervals in this report are the 95% risk-adjusted confidence intervals from the Agency for Healthcare Research and Quality’s software.

Statistical significance

A difference is “statistically significant,” if it is unlikely to have happened just by chance.  (For example, having a coin come up heads 6 times in 10 flips is not surprising and is not “statistically significant,” because that could happen just by chance.)  In this report, a hospital’s risk-adjusted rate differs from the U.S. rate more than is likely by random variation, if the U.S. rate does not fall within the hospital’s confidence interval.  This report highlights (in red or green) hospital risk-adjusted rates that are statistically significantly different from the U.S. rate.

The difference between the Lower Confidence Limit and Upper Confidence Limit values shown on the reports represents the potential margin of error relating to a specific hospital's rate measurement.  The larger this range, the greater the potential influence of random chance on the calculated rate.  The range will vary for each hospital depending upon the total number of cases for that condition or for that procedure, and the calculated rate for that year.  The margin of error for reporting a measure is wider for hospitals with fewer cases.  If the hospital's margin-of-error range does not encompass the national average and is below the national average, the hospital's rate is statistically lower than the national average (shown in green type).  If a hospital's range for a quality indicator does not encompass the national average and is above the national average, the hospital's rate is statistically higher than the national average (shown in red).  If the measure's national average falls within the hospital's margin-of-error range, the hospital's rate is not statistically different from the national average (shown in black).  

 

Back to the Top

Last Updated on: 04/29/2008