PDF Evaluation of Evidence-Based Practices in Online Learning

[Pages:94]Evaluation of Evidence-Based Practices in Online Learning

A Meta-Analysis and Review of Online Learning Studies

Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies

U.S. Department of Education Office of Planning, Evaluation, and Policy Development

Policy and Program Studies Service

Revised September 2010

Prepared by Barbara Means Yukie Toyama Robert Murphy Marianne Bakia

Karla Jones Center for Technology in Learning

This report was prepared for the U.S. Department of Education under Contract number ED-04CO-0040 Task 0006 with SRI International. Bernadette Adams Yates served as the project manager. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education is intended or should be inferred.

U.S. Department of Education Arne Duncan Secretary

Office of Planning, Evaluation and Policy Development Carmel Martin Assistant Secretary

Policy and Program Studies Service Alan Ginsburg Director

Office of Educational Technology Karen Cator Director

September 2010

This report is in the public domain. Authorization to reproduce this report in whole or in part is granted. Although permission to reprint this publication is not necessary, the suggested citation is: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies, Washington, D.C., 2010.

This report is also available on the Department's Web site at about/offices/list/opepd/ppss/reports.html.

On request, this publication is available in alternate formats, such as braille, large print, or computer diskette. For more information, please contact the Department's Alternate Format Center at (202) 260-0852 or (202) 260-0818.

Contents

EXHIBITS ...................................................................................................................................................................... V

ACKNOWLEDGMENTS ................................................................................................................................................ VII

ABSTRACT ................................................................................................................................................................... IX

EXECUTIVE SUMMARY ............................................................................................................................................... XI

Literature Search.................................................................................................................................................... xii Meta-Analysis....................................................................................................................................................... xiii Narrative Synthesis ................................................................................................................................................xiv Key Findings..........................................................................................................................................................xiv Conclusions......................................................................................................................................................... xviii

1. INTRODUCTION .........................................................................................................................................................1

Context for the Meta-analysis and Literature Review ..............................................................................................2 Conceptual Framework for Online Learning ............................................................................................................3 Findings From Prior Meta-Analyses.........................................................................................................................6 Structure of the Report..............................................................................................................................................7

2. METHODOLOGY ........................................................................................................................................................9

Definition of Online Learning ..................................................................................................................................9 Data Sources and Search Strategies ........................................................................................................................10 Electronic Database Searches .................................................................................................................................10 Additional Search Activities ...................................................................................................................................10 Screening Process ...................................................................................................................................................11 Effect Size Extraction .............................................................................................................................................13 Coding of Study Features .......................................................................................................................................14 Data Analysis..........................................................................................................................................................15

3. FINDINGS .................................................................................................................................................................17

Nature of the Studies in the Meta-Analysis ............................................................................................................17 Main Effects............................................................................................................................................................18 Test for Homogeneity .............................................................................................................................................27 Analyses of Moderator Variables ...........................................................................................................................27 Practice Variables ...................................................................................................................................................28 Condition Variables ................................................................................................................................................30 Methods Variables ..................................................................................................................................................31

4. NARRATIVE SYNTHESIS OF STUDIES COMPARING VARIANTS OF ONLINE LEARNING ........................................37

Blended Compared With Pure Online Learning .....................................................................................................38 Media Elements ......................................................................................................................................................40 Learning Experience Type......................................................................................................................................41 Computer-Based Instruction ...................................................................................................................................43 Supports for Learner Reflection..............................................................................................................................44 Moderating Online Groups .....................................................................................................................................46 Scripts for Online Interaction..................................................................................................................................46 Delivery Platform ...................................................................................................................................................47 Summary ................................................................................................................................................................. 48

5. DISCUSSION AND IMPLICATIONS ............................................................................................................................51

Comparison With Meta-Analyses of Distance Learning ........................................................................................52 Implications for K?12 Education............................................................................................................................53

iii

REFERENCES ...............................................................................................................................................................55 Reference Key ........................................................................................................................................................55

APPENDIX META-ANALYSIS METHODOLOGY ....................................................................................................... A-1 Terms and Processes Used in the Database Searches .......................................................................................... A-1 Additional Sources of Articles............................................................................................................................. A-3 Effect Size Extraction .......................................................................................................................................... A-4 Coding of Study Features .................................................................................................................................... A-5

iv

Exhibits

Exhibit 1. Conceptual Framework for Online Learning ................................................................................................5 Exhibit 2. Bases for Excluding Studies During the Full-Text Screening Process .......................................................13 Exhibit 3. Effect Sizes for Contrasts in the Meta-Analysis .........................................................................................20 Exhibit 4a. Purely Online Versus Face-to-Face (Category 1) Studies Included in the Meta-Analysis........................21 Exhibit 4b. Blended Versus Face-to-Face (Category 2) Studies Included in the Meta-Analysis ................................24 Exhibit 5. Tests of Practices as Moderator Variables ..................................................................................................29 Exhibit 6. Tests of Conditions as Moderator Variables ...............................................................................................30 Exhibit 7. Studies of Online Learning Involving K?12 Students ................................................................................32 Exhibit 8. Tests of Study Features as Moderator Variables.........................................................................................34 Exhibit 9. Learner Types for Category 3 Studies ........................................................................................................37 Exhibit A-1. Terms for Initial Research Database Search ........................................................................................ A-2 Exhibit A-2. Terms for Additional Database Searches for Online Career Technical Education and Teacher

Professional Development............................................................................................................................... A-2 Exhibit A-3. Sources for Articles in the Full-Text Screening................................................................................... A-3 Exhibit A-4. Top-level Coding Structure for the Meta-analysis............................................................................... A-6

v

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download