CRITICAL APPRAISAL (TELAAH KRITIS)

PENELITIAN KEDOKTERAN

TELAAH KRITIS = CRITICAL APPRAISAL

PENILAIAN HASIL PENELITIAN SECARA SISTEMATIS

Menambah pemahaman Mengikuti perkembangan Melakukan penelitian

LANGKAH AWAL TELAAH KRITIS

PENILAIAN STRUKTUR DAN KELENGKAPAN MAKALAH PENILAIAN JUDUL, ABSTRAK DAN RUJUKAN

CHECK LIST UMUM STRUKTUR DAN ISI MAKALAH

PEMBAHASAN TELAAH KRITIS

Penilaian struktur & kelengkapan makalah

Judul Nama pengarang & institusinya Abstrak Pendahuluan Metodologi penelitian Hasil Diskusi/pembahasan Ucapan terima kasih Daftar rujukan

PENILAIAN TERHADAP JUDUL

Sesuai dgn keperluan pembaca Tidak terlalu panjang atau pendek Masalah yang diteliti jelas / efisien Menarik Tanpa singkatan, selain yang baku

ABSTRAK
Informasi menyeluruh/ringkas Latar belakang/tujuan Rancangan penelitian Metodologi Hasil Kesimpulan Saran Kurang dari 250 kata

RUJUKAN

Rujukan terbaru (Terutama jurnal hasil penelitian)

Penulisan yang tepat

CHECK LIST UMUM STRUKTUR DAN ISI MAKALAH

CHECK LIST YA TIDAK TIDAK RELEVAN

PENDAHULUAN

1. Pendahuluan tdd 2 paragraf atau 2 bagian 2. Paragraf atau bagian pertama mengemukakan alasan dilakukannya penelitian 3. Paragraf atau bagian kedua menyatakan hipotesis atau tujuan penelitian 4. Paragraf kedua menyebutkan design yg digunakan 5. Pendahuluan didukung oeh pustaka yg kuat dan relevan 6. Pendahuluan lebih dari 1 halaman

METODE

1. Desain, tempat, dan waktu disebutkan 2. Populasi sumber disebutkan 3. Kriteria pemilihan inklusi dan eksklusi dijelaskan 4. Tehnik Sampling disebutkan 5. Perkiraan besar sampel disebutkan dan alasannya 6. Perkiraan besar sampel dihitung dgn rumus yg sesuai 7. Komponen-komponen rumus besar sampel diisi dengan angka yang masuk akal

METODE

8. Observasi, pengukuran serta intervensi dirinci sehingga orang lain dapat mengulanginya 9. Rujukan disebutkan bila teknik pengukuran tidak dirinci 10. Pengukuran dilakukan secara tersamar 11. Uji keandalan pengukuran (kappa) dilakukan 12. Definisi istilah dan variabel penting dikemukakan 13. Ethical clearance diperoleh 14. Persetujuan subjek diperoleh 15. Disebutkan rencana analisis, batas kemaknaan, &power penelitian 16. Program komputer yang dipakai disebutkan

HASIL

1. Tabel deskripsi subyek penelitian disebutkan 2. Pada uji perbandingan, karakteristik subyek yang penting sebelum intervensi dibandingkan kesetaraannya 3. Uji hipotesis untuk kesetaraan dilakukan 4. Jumlah subyek penelitian disebutkan 5. Subyek yang droup out dijelaskan beserta alasannya 6. Ketepatan numerik dinyatakan dengan benar 7. Penulisan tabel dilakukan dengan tepat 8. Tabel dan iliustrasi bersifat informatif 9. Tabel dan ilustrasi memang diperlukan

HASIL

10. Semua hasil dalam tabel disebutkan didalam nas. 11. Semua outcome yang penting disebutkan dalam hasil 12. Subyek yang drop out diikutkan dalam analisis 13. Analisis dilakukan dengan uji yang sesuai 14. Hasil uji statistik disertakan, derajat kebebasan dan nilai p disertakan 15. Tidak dilakukan analisis yang semula tidak direncanakan 16. Interval kepercayaan disertakan 17. Komentar dan pendapat tidak disertakan dalam hasil

DISKUSI

1. Semua hasil yang relevan dibahas 2. Hal yang dikemukakan dalam hasil sering diulang 3. Dibahas keterbatasan hasil penelitian, dan kemungkinan dampaknya terhadap hasil 4.Diskusi menyebutkan kesulitan penelitan, penyimpangan dari protokol, dan kemungkinan dampaknya terhadap hasil 5.Pembahasan dihubungkan dengan pertanyaan penelitian 6.Pembahasan dihubungkan dengan teori dan hasil peneltian terdahulu 7. Dibahas hubungan hasil dengan praktek klinis 8. Disertakan kesimpulan utama penelitian.

DISKUSI

9. Kesimpulan didasarkan pada data penelitian 10. Kesimpulan bersifat shahih 11. Diskusi mengemukakan dan membahas efek samping 12. Hasil tambahan selama observasi, disebutkan 13. Hasil tambahan dianalisis secara statistik 14. Generalisasi hasil penelitan, disebutkan 15. Diskusi disertakan saran penelitian selanjutnya, dengan anjuran metodologis yang tepat

UCAPAN TERIMA KASIH

1. Ucapan terima kasih ditujukan kpd orang-orang yang tepat 2. Ucapan terima kasih dinyatakan secara wajar

Critical Appraisal
On Article of Diagnostic Test (EBM-Diagnostic)

Critical appraisal is one step in the process of evidencebased clinical practice. To determine what is the best evidence, we need critical appraisal skills that will help us to understand the methods and results of research and to assess the quality of the research. Most research is not perfect, and critical appraisal is not an exact science - it will not give us the right answer. But it can help us to decide whether we think a reported piece of research is good enough to be used in decision making.

Critical Appraisal

- Worksheet for critical appraisal - Software : CAT Maker Main area of clinical objectives: 1. Diagnosis 2. Prognosis 3. Therapy/Treatment 4. Risk/Harm Others: Systematic Review and Meta-analysis Clinical Guidelines Clinical Decision Making etc.

THREE MAIN ASPECTS TO BE APPRAISED: 1.

VIA

VALIDITY : VALID (CLOSENESS TO THE TRUTH) IN THE METHODOLOGY SECTION IMPORTANCE : IMPORTANT (USEFULNESS) IN THE RESULTS SECTION APPLICABILITY : APPLICABLE (CAN BE APPLIED IN CLINICAL PRACTICE) IN THE DISCUSSION SECTION

2.

3.

DIAGNOSIS WORKSHEET
Citation:

Are the results of this diagnostic study valid? Was there an independent, blind comparison with a reference (gold) standard of diagnosis? Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)? Was the reference standard applied regardless of the diagnostic test result? Was the test (or cluster of tests) validated in a second, independent group of patients?

Are the valid results of this diagnostic study important?

SAMPLE CALCULATIONS
Target disorder (iron deficiency anemia) Present Absent Totals

Diagnostic Positive test result (< 65 mmol/L) (serum Negative ferritin) ( 65 mmol/L)
Totals

731 a
78 c 809 a+c

270 b
1500 d 1770 b+d

1001 a+b
1578 c+d 2579 a+b+c+d

Sensitivity = a/(a+c) = 731/809 = 90% Specificity = d/(b+d) = 1500/1770 = 85% Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 90%/15% = 6 Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 10%/85% = 0.12 Positive Predictive Value = a/(a+b) = 731/1001 = 73% Negative Predictive Value = d/(c+d) = 1500/1578 = 95% Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 809/2579 = 32% Pre-test odds = prevalence/(1-prevalence) = 31%/69% = 0.45 Post-test odds = pre-test odds LR Post-test probability = post-test odds/(post-test odds +1)

YOUR CALCULATIONS

Totals
Target disorder Present Diagnostic test result Positive Negative Totals a c a+c Absent b d b+d a+b c+d a+b+c+d

 SnNout
Diagnostic test with a very high sensitivity , a negative result effectively rules out the diagnosis

SpPin
Diagnostic test with a very high specificity , a positive result effectively rules in the diagnosis

Can you apply this valid, important evidence about a diagnostic test in caring for your patient?
Is the diagnostic test available, affordable, accurate, and precise in your setting? Can you generate a clinically sensible estimate of your patients pre-test probability (from personal experience, prevalence statistics, practice databases, or primary studies)? Are the study patients similar to your own? Is it unlikely that the disease possibilities or probabilities have changed since the evidence was gathered?

Will the resulting post-test probabilities affect your management and help your patient? Could it move you across a test-treatment threshold? Would your patient be a willing partner in carrying it out?
Would the consequences of the test help your patient?

Additional notes:

Apakah ada perbandingan, independen buta dengan referensi ("emas") standar diagnosis? Apakah tes diagnostik dievaluasi dalam spektrum yang sesuai pasien (seperti di yang akan digunakan dalam praktek)? Apakah standar referensi diterapkan tanpa hasil uji diagnostik? Apakah tes (atau kelompok tes) divalidasi dalam kelompok, kedua pasien independen?
Apakah tes diagnostik yang tersedia, terjangkau, akurat, dan tepat dalam pengaturan Anda? Dapatkah Anda menghasilkan perkiraan klinis yang masuk akal dari probabilitas pre-test pasien Anda (dari pengalaman pribadi, statistik prevalensi, database praktek, atau studi utama)? Apakah pasien penelitian serupa dengan Anda sendiri? Apakah tidak mungkin bahwa kemungkinan penyakit atau probabilitas telah berubah sejak bukti itu dikumpulkan? Apakah post-test sehingga mempengaruhi probabilitas manajemen Anda dan membantu pasien Anda? Mungkinkah menggerakkan Anda melintasi ambang tes- perawatan? Apakah pasien Anda bersedia menjadi mitra dalam menjalankan test? Apakah konsekuensi dari uji membantu pasien Anda?

Example : an article of diagnostic test, entitle :

Bedside Diagnosis of Influenzavirus Infections in Hospitalized Children

Katherine A. Poehling, et al American Academy of Pediatrics

Background: Influenzavirus has a significant impact on the pediatric population, with school-aged children having the highest infection rates. For preventing nosocomial influenza infections and to facilitate prompt antiviral therapy, an accessible, rapid diagnostic method for influenzavirus is needed. Objective: To compare the performance of a rapid diagnostic test (QuickVue Influenza Test; Quidel Corp, San Diego, CA) completed at the bedside of hospitalized children to viral culture and/or polymerase chain reaction (PCR) for influenzavirus.

Method: Study population: 1) younger than 19 years and hospitalized with respiratory symptoms or 2) younger than 3 years and hospitalized with fever. Sampel: 1) a primary admission diagnosis of an acute respiratory illness characterized by rhinorrhea, sore throat, cough, shortness of breath, or apnea or 2) a primary admission diagnosis consistent with a febrile illness and a temperature of 100.4F. Broad inclusion criteria were chosen such that all children who were hospitalized with symptoms potentially related to influenza infections were eligible.

Sample in this study

Study design: prospective, cross sectional study from each child, 2 nasal swabs of the turbinates were obtained1 for influenzavirus culture and PCR and the other for the rapid diagnostic test. The rapid test results were compared with that of culture and PCR for influenzavirus.
Influenza infection was defined as any sample with 1) a positive culture for influenzavirus or 2) 2 consecutive positive PCRs for influenza A or B. Each researcher was trained to perform and interpret the rapid diagnostic test at the bedside according to the manufacturers instructions. The laboratory technician who performed the culture and PCR was masked to the rapid diagnostic test results.

Results:

Sens: 74% Spec: 98% PPV: 74% NPV: 98%

DIAGNOSIS WORKSHEET
Citation:

Are the results of this diagnostic study valid? Was there an independent, blind comparison with a reference (gold) standard of diagnosis? Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)? Was the reference standard applied regardless of the diagnostic test result? Was the test (or cluster of tests) validated in a second, independent group of patients? Yes
Yes

Yes Yes

Are the valid results of this diagnostic study important?

SAMPLE CALCULATIONS

Totals
Target disorder Present Diagnostic test result Positive Negative Totals a c a+c Absent b d b+d a+b c+d a+b+c+d

+ -

+ 14 5

5 209

Culture or PCR Present Quick Vue Influenza test Positive Negative 14 5 19 Absent 5 209 214

Totals

19 214 233

Totals
Sensitivity = a/(a+c) = 14/19 = 74% Specificity = d/(b+d) = 209/214 =98%

Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 74%/2% = 37 Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 26%/98% = 0.27

Positive Predictive Value = a/(a+b) = 14/19 = 74% Negative Predictive Value = d/(c+d) = 209/214 = 98%
Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 19/233 = 8%

Pre-test odds = prevalence/(1-prevalence) = 8%/92% = 0.087 Post-test odds = pre-test odds LR = 0.087 x 37 = 3.22 Post-test probability = post-test odds/(post-test odds +1)= 3.22/4.22=76%

Just to remind you:

Sensitivity: the percentage of persons with the disease of interest who have positive test results. = a/(a+c) x 100 Specificity: the percentage of persons without the disease of interest who have negative results. = d/(d+b) x 100

PPV: the percentage of persons with positive test results who actually have the disease of interest. = a/(a+b) x 100 NPV: the percentage of persons with negative test results who do not have the disease of interest. = d/(d+c) x 100

Likelihood Ratio

Likelihood is the probability of a particular test result for a person with the disease of interest divided by the probability of that test result for a person without the disease of interest. Likelihood Ratio for a positive test result (LR+) Likelihood Ratio for a negative test result (LR-)

(LR+) is the probability of a positive test result for a person with the disease of interest divided by the probability of a positive test result for a person without the disease.
LR+ = Sensitivity / (1-Specificity) LR+ > 1: persons affected with disease of interest are more likely to have a positive test result than unaffected persons. The larger the value of the LR, the stronger the association between having a positive test result and having the disease of interest. LR+ value of 10 or greater is perceived as indication of a test of high diagnostic value.

LIKELIHOOD RATIO

LRs >10 or <0.1 cause large changes in likelihood. LRs 5-10 or 0.1-0.2 cause moderate changes. LRs 2-5 or 0.2-0.5 cause small changes. LRs between <2 and 0.5 cause little or no change

Pretest probability of disease: the probability that a person has the disease of interest before the test is performed. = prevalence Pretest odds of disease: the estimate before diagnostic testing of the probability that a patient has the disease of interest divided by the probability that the patient does not have the disease of interest. Pretest odds= pretest probability/(1-pretest probability)

Posttest odds of disease: as the estimate after diagnostic testing of the probability that a patient has the disease of interest divided by the probability that the patient does not have the disease of interest. Posttest odds= pretest odds x LR + Posttest probability = posttest odds/ (1+posttest odds) a result of obtaining a positive test result, the estimated probability of the presence of disease has risen from 0.08 (pretest probability) to 0.76 (posttest probability).

Diagnostic tests that produce big changes from pretest to post-test probabilities are important and likely to be useful to us in our practice

Software:

CAT Maker

Can you apply this valid, important evidence about a diagnostic test in caring for your patient?
Is the diagnostic test available, affordable, accurate, and precise in your setting? Can you generate a clinically sensible estimate of your patients pre-test probability (from personal experience, prevalence statistics, practice databases, or primary studies)? Are the study patients similar to your own? Is it unlikely that the disease possibilities or probabilities have changed since the evidence was gathered?

No Yes Yes

Will the resulting post-test probabilities affect your management and help your patient? Could it move you across a test-treatment threshold? Would your patient be a willing partner in carrying it out?
Would the consequences of the test help your patient?

Yes
Yes

Additional notes:

Thank you..