A proposal submitted in partial fulfillment

of course requirements for:

EME 7939 – Dr. James White

Chapter One

Introduction

Background

Examples of Education Internet Sites (Table 1)

General Format of the WITS Site

WITS Core Curriculum

WITS Technology

Details of the WITS Prototype

Research Questions

Chapter II

Literature Summary: Justification for establishing a web-based tutorial system

Chapter III

Research Design

Description of Phenomenology

Data Gathering Procedures

Sample

Instruments

Analysis of Data

Categorization of Data (Table 2)

Potential Problems

Further Research

References

Appendix A: Data Gathering Instruments

Appendix B: A Few of the Many Education Internet Sites (Table B1)

Appendix C: Screen Capture of WITS Web Pages

I am constructing a prototype for an educational web site that will offer a no-fee, K – 12 core-curriculum tutorial system. The site is called WITS, an acronym for Web Interactive Tutorial Systems. WITS differs from other education web sites in that it contains in one site tutorials, opportunities for practice, testing, and teacher notification of student progress, for no fee and containing no advertising banners. While similar sites charge learners for access or credit, I have not found a no-fee, no-advertising site containing all of these features.

WITS is in the early stages of development; consequently it is time to have learners and teachers test the prototype. The focus of this study is the observation of initial users' experiences with the WITS prototype. The goal of the study is improvement of the WITS site through evaluation of the users' experiences.

The Internet was originally developed by government agencies and university systems needing to share information. Ongoing rapid expansion of the Internet, including development of the World Wide Web, WWW, is changing many aspects of the world, especially business communication, trade, and education. The function of the Internet on behalf of education now extends far beyond transmission of student records: it is impacting the delivery of education content and management of classrooms, both traditional and virtual.

Table 1 shows several examples of Internet sites devoted to various forms of education enhancement. Visits to a few of the sites reveal that the Internet is being used in many ways to benefit the

delivery and management of education. Secondary and post- secondary courses are being delivered on-line. Internet learning tools are expanding classroom learning. In addition, home schooling courses are being delivered, drill and practice sites are available for individuals and for classroom use, and tests are being delivered on-line.

Computer based instruction in the form of self-paced tutorials, review quizzes and in-depth exploration opportunities are available to students free of charge on the Internet. Students can learn at their most comfortable pace, be aware of their learning status, and gain further information if desired. Learning can take place anywhere there is a computer with Internet access.

Education related Internet sites now number in the thousands and more are appearing daily. Therefore it would not be practical in this format to attempt to list all education sites. Table 1B, found in Appendix B, contains a larger, though far from complete, listing of Internet sites dedicated to education.

The WITS site will be composed of many small segments, each covering one small chunk of knowledge. Content will be ability-appropriate: Level I segments are for non-readers and will consist primarily of graphics and sound. Level II segments are for beginning readers and will consist of a combination of easy reading text and graphics and sound. Level III segments are for those who can read at the Fifth grade level or better. Level III segments will combine text and graphics, with sound as an option if deemed appropriate by later site studies.

As described below, each WITS segment will contain (1) an interactive tutorial, (2) a practice quiz, and (3) a link to a "Power Quiz" that can be used as a test. In addition, each WITS segment will contain links to further information on the Internet.

1. Tutorial: each tutorial will contain 10 to15 screens, presenting 4 to 6 pieces of content. Each screen will contain information and a question that has been answered within that screen or the screens immediately prior. Progression through the tutorial will require the learner to respond by typing in an appropriate response to a question or completion of a fill-in-the-blank statement. All screens will be interactive. For learners who are readers and able to use the keyboard, screen responses will not be multiple choice. Pictures and diagrams will be available when appropriate.
2. Practice quiz: The practice quiz will consist of 10 questions presented in a format similar to the tutorial. The learner will have 3 tries to respond with the answer, which will normally be one or two words. If his third response is incorrect, he will be supplied with the correct answer. If he has missed 3 or more within the first 5 to 7 questions, he will be redirected to the tutorial or an alternate refresher tutorial. The learner can take the quiz as many times as he wishes and will qualify to take the related power quiz when he has scored at least 9 out of 10 on the quiz. At any time during the quiz the learner can check on his current score. An alert box tells the learner his score at the end of the quiz. He can submit his score to his teacher via a SUBMIT button, which at this time e-mails the score to the teacher. It is planned that later versions of WITS will submit information to a database for retrieval by the teacher.
3. Power quiz: Each tutorial links to a power quiz covering the tutorial. The Power Quiz is timed. The learner is given 3 tries at 10 seconds per try. If he enters a third incorrect response or if he has not entered the correct response within the time limit, the question is scored as a wrong answer and a new question is presented. The learner is not told the correct answer. The learner can check on his score during the quiz if he wants to give up the time needed to check his score. An alert box tells the learner his score at the end of the quiz. He can submit his score to his teacher via a SUBMIT button, which at this time e-mails the score to the teacher. Later versions of WITS will submit information to a database for retrieval by the teacher.
4. Links to further information: Links are provided to other related information, other resources and, if they are available, related on-line educational games. Links lead to sites that are education or government sites that are likely to remain at the designated URL for long periods of time. I will make periodic checks of links and provide user feedback forms that will allow the user to submit information regarding link problems or other site problems.

WITS will offer a K-12 core curriculum composed of knowledge level content technologically appropriate for web presentation. In future development phases, WITS may offer instruction for development of higher level thinking but at this phase the site will contain only basic knowledge level information. I (or a site development team) will construct the curriculum by examination and selective combination of the existing K-12 curricula used by three large U. S. public school districts. Knowledge level items that are included in the course objectives of at least two of the three school districts will be included in the WITS core curriculum. I plan to choose school districts that are among the 20 largest in the U.S. by student population, accredited by recognized, appropriate agencies, and geographically separated from one another by at least 1000 miles.

Items from national standardized achievement tests and benchmark tests will be used as a checklist for comparison of WITS curriculum items. An attempt will be made to include most of the knowledge level items that appear on these tests.

Since it will offer a core curriculum, it is expected that WITS will not be the learner's sole source of information. The site will function, instead, as a vehicle that will allow the learner to gain basic knowledge as needed. The learner will also be able to practice recall of content knowledge, and connect to further Internet information resources.

It is expected that WITS will serve as a learning resource, to be used in a wide variety of ways by learners in many different settings. The WITS site is not being developed to replace teachers or the classroom, but rather as a reference site that can be used in a large variety of ways. It is intended that the site be used much as a textbook might be used-- for information, enrichment and reinforcement, to be combined with other learning resources.

WITS will serve as a supplementary learning resource for mobile students, students who need extra tutoring or those who finish their work early, students who are ill and cannot be in their traditional classroom, home schooled students, and adult or night school students who need to learn in individualized programs. In some cases, WITS will be used solely for recall practice of knowledge level content. It will be used as a springboard to create and answer higher levels of questions.

The current WITS technology consists of series of individual web pages created in HTML and JavaScript, stored on a server and loaded into the user's browser. Future site development will take advantage of emerging technologies such as Java, Active-X, Power Point and other web-compatible tools for enhanced presentation. Future plans call for a server database to store tutorials for client retrieval and a related dynamic database to store teacher and learner tracking and management information.

The prototype WITS segment consists of one Level III segment of basic information about motion. It will contain a short tutorial, a quiz, and a link to a Power Quiz. This segment was chosen because I currently teach this subject at the secondary level and am able to construct the segment using available content knowledge. Learner attributes for this segment include the ability to read at a minimum level of fifth grade and the physical ability to use a keyboard and mouse. Screen captures of the WITS site pages are found in Appendix C. The URL is currently: http://members.aol.com/Barb24601/wits.htm

This is a formative study. The initial WITS site is a prototype and I want to identify changes needed to make the site user-friendly. I plan to have 20 students test the first prototype of WITS. The primary research questions are:

1. Can learners begin to use the site with little or no human assistance?
2. How easily can learners navigate the site?

3. How easily and in what ways are classroom teachers able to incorporate use of the site into instructional

settings?

4. How easily are teachers able to access student results?

5. In what formats do teachers want to receive student information?

6. Are there features of the site that should be retained or enhanced because the features

are particularly liked by learners or teachers?

7. What changes are needed to make the WITS site easy for learners to use?

If politics, unions, corporations and all levels of education governance are exerting pressure, the strongest at the time or location wins, and this makes education subject to the ongoing financial and political wars (Fuhrman, 1994). The Internet offers a democratic medium for instruction (Khan, 1997).

Tested teaching methods carry over into computer based instruction. The behaviorist theory of Skinner (1968) is cited in Kearsey's on-line database as a guide for building programmed instruction. Kearsey (on-line 1998), Fleming and Levie (1993), Alessi and Trollip (1985), and others have suggested concepts for constructing quality programmed instruction:

1. CBI is best presented in a sequential, guiding manner.

2. The student learns best with ongoing knowledge of his learning status.

3. The student learns best with practice.

4. Learning should be assessed.

5. Motivation and opportunity will cause the student to seek further information.

The Internet appears to be an excellent format for delivery of instruction grounded in these established learning theories (Reeves & Reeves, 1997). While web-based instruction can incorporate typical CBI formats the World Wide Web offers several advantages over traditional CBI. Thoughtful analysis of a few (Table 1B) of the many existing education web sites reveals that when education or education management is web-based:

1. Interaction is not confined to particular physical locations.
2. Every delivery computer or server does not need to contain the particular instructional software being used.
3. The learner can be mobile, having access to learning tools wherever he has Internet access.
4. The teacher can be mobile, building learning units, presenting them, and managing student records from any location that offers Internet access.
5. The learner has access to vast quantities of learning resources available through the Internet, far more than possible in a classroom, library or home setting without the Internet.
6. Individual instructors can impact many more students than possible in classroom settings.
7. Learning can take place on an individual basis, in small group formats or in large groups.
8. Instruction and learning can be asynchronous or can occur with real time interaction.

Although there is little documented evidence in favor of web based instruction (Reeves, in Khan, p. 59) the Internet appears to be an ideal medium for wide distribution of learning tools. "Distance education … is regarded internationally as a viable and cost effective way of providing individualized instruction." (McIsaac & Gunawardena, 1996).

"The World Wide Web does provide information …in any medium, on any subject, in any order, at any time…in any place" (Crossman, 1997).

A free tutorial system could be used by anyone with a computer or WebTV and Internet access. It remains for developers of web-based instruction to produce quality systems, document their use, subject them to ongoing evaluation, and integrate ongoing improvements based on those evaluations.

My goal as a researcher is to develop a useful instructional product. At this point in the development of WITS, quantitative analysis is not appropriate. Formative evaluation of the prototype is in order. WITS is to be a reference site and, at this writing, I have been unable to locate a similar product for comparison. WITS differs from other education sites in that it contains tutorials, opportunities for practice, testing, and teacher notification of student progress, for no charge and containing no advertising banners. While similar sites exist on a fee for access or fee for credit basis, I have not found a no-fee, no-advertising site containing these features.

Willis (1993) describes a model for instructional product development, called the Instructional Development, or I.D. model. Willis' I.D. model, based on models by Thiagarajan et. al, identifies four stages: (1) Define, (2) Design, (3) Develop, and (4) Disseminate. I am at the very beginning of my product development - that of "define" and "design".

I am building a useable prototype and feedback from initial users is needed. I will analyze that feedback, make appropriate changes, proceed with further development, and then disseminate the product. I have chosen a qualitative method called Phenomenology for this stage of my research.

Phenomenology is the study of phenomena as it is experienced and reported by individuals who are intimately involved. This usually involves reports by the researcher and others - participants - who experience the phenomena. Fahlberg (1994) points out that the focus of Phenomenology is on the lived experience of the subject: one should describe the experience rather than the objects. For example, in this study, I will be recording the experiences of users of the WITS prototype.

I want to determine what changes should be made to the WITS prototype in order to make it user-friendly. Therefore, it is necessary to investigate the experiences of both learner users and instructional users. Phenomenology is appropriate research for studies into the experiences of learners and instructors.

Gall, Borg, and Gall (1996) point out that Phenomenology is appropriate "as part of pilot work for a study that will employ a quantitative research design" (p. 600). While WITS is not now at a stage for quantitative research, I expect to conduct quantitative studies regarding use of WITS after further development of the site.

Gall, Borg, and Gall (p 601-602) list a series of steps that are commonly followed by phenomenological researchers:

(1) Identify a topic of personal and social significance.

(2) Select appropriate participants.

(3) Interview each participant.

(4) Analyze the interview data.

I plan to use the steps suggested by Gall and his cohorts:

(1) Selection of topic: The focus of this study is the observation of initial users' experiences with the WITS prototype. The goal of the study is improvement of the WITS site through evaluation of the users' experiences as they use one tutorial segment, a practice quiz, and the power quiz for that tutorial. Student use of links to further information will be optional and other than to note whether or not the student did choose to try one or more information links, those experiences will not be included in data. The study will last for the length of time that the teacher chooses to have students use the site.

(2) Selection of appropriate participants. There will be three types of participants in this study: the learners and their teacher are the obvious participants. I will be the other participant. As developer and observer of learners as WITS users, I will take notes as I observe the learners and their teacher using the WITS site. There will be twenty student participants, ages 14 to 17, enrolled in a traditional science classroom. I will also interview their teacher.

• Part A: Researcher Observations: I will observe the learners and teachers as they use the WITS prototype.
• Part B: Learner Interviews. I will interview twenty students using the WITS site. The interview will be semi-structured and will consist of two parts:

1. Structured web-based learner questionnaire: As students finish their tutorial units, they will fill out a web-based form regarding their learning experience.
2. Semi-Structured Researcher-Learner Interview: When each student has finished his WITS session, I will ask specific questions from a prepared list, described below, and will encourage other comments.

• Part C: Teacher Interviews:

1. Structured web-based teacher questionnaire: As a class finishes their tutorial units, their teacher will fill out a web-based form regarding the learning experience.
2. Document F – Teacher Interview: When a class has finished its WITS session, I will interview the teacher, asking specific questions from a prepared list, described below, and will encourage other comments.

(4) Analyze the interview data. See the section below titled: Analysis of Data.

The learner sample for this study will consist of twenty student subjects between the ages of 14 to 17 years, enrolled in three Hillsborough County, Florida public high school Physical Science classes. The instructors of those classes will also be interviewed in a semi-structured format.

• Document A – Learner Web-Based Questionnaire: As students finish a tutorial unit, they will fill out a web-based questionnaire regarding their learning experience.

• Document E – Learner Interview: When each student has finished his WITS session, I will ask specific questions from a prepared list, described below, and will encourage other comments.

• Document B – Teacher Web-Based Questionnaire: As a class finishes their tutorial units, their teacher will fill out a web-based form regarding the learning experience.

• Document F – Teacher Interview: When a class has finished its WITS session, I will interview the teacher, asking specific questions from a prepared list, described below, and will encourage other comments.

Part III: Researcher Observations: I will observe the learners and teachers as they use the WITS prototype.

Data will be analyzed to determine whether there are changes that can be made to improve the WITS site.

The primary research questions are:

1. Can learners begin to use the site with little or no human assistance?

2. How easily can learners navigate the site?

3. How easily and in what ways are classroom teachers able to incorporate use of the site into instructional

settings?

4. How easily are teachers able to access student results?

5. In what formats do teachers want to receive student information?

6. Are there features of the site that should be retained or enhanced because the features

are particularly liked by learners or teachers?

7. What changes are needed to make the WITS site easy for learners to use?

Questions to the learner and teacher participants have been chosen to address each research question. Table 2 shows the Research Questions and the participant questions that address them. Triangulation is achieved by input from the learner (Triangulation key = L), the teacher (Triangulation key = T), and the researcher (Triangulation key = R).

1. Responses to web-based questionnaires and researcher observations will be placed into a similar table, so that all responses can be examined for:

• Answers to the Research Questions.

• Common features that do not relate to other parts of the study.
• Indicators of other Research Questions that should be addressed in future studies.

2. All responses indicating a problem or need will be classified as a need to be addressed. Needs will be placed in order of request frequency.

3. A priority list for site modification will be created from this list of needs.

The phenomenological study calls for close involvement of the researcher, therefore there is potential for researcher bias. In order to help eliminate researcher bias that might prevent attention to needed changes, this study calls for triangulation, input of the learner and the teacher in addition to input of the researcher.

Since the subjects will be young and may be working in close proximity at times, it is anticipated that there may be some competition among the subjects. In addition some subjects may share their thoughts or answers with others. Competition and sharing work may or may not influence the learning of the students. This is not considered to be a problem for this study since it is not the purpose of this study to quantitatively evaluate variables that influence student achievement.

It is necessary to ascertain the usefulness and relevance of items in the data collection instruments. The research questions and data collection instruments will be submitted to three experts in research design at the University of South Florida for input regarding appropriateness of the observer checklists, questionnaires, and interview documents.

Upon examination of the data, new studies will be undertaken to determine:

It is anticipated that this study will be repeated after site modification. The process of developmental modification and evaluation will be ongoing throughout the existence of the site.

It is expected that many further studies will take place using the WITS site including quantitative studies regarding its effectiveness and suitability in various settings.

Teacher Feedback - Please tell us how to improve this website!

1. This student appears: __ Uninterested. __ Somewhat interested. __ Interested. __ Highly interested.
2. This student appears: __ Insecure. __ Unhappy. __ Angry. __ Afraid. __ Bored. __ Sad. __ Tired. __ Ill. __ Restless, unable to stay on task. __ Other (note here): _____________________________.
3. This student appears: __ Confident. __ Happy. __ Content. __ Eager. __ On task. __ Motivated. __ Quiet. __ Noisy __ Other (note here): _______________________________________________.
4. Requests for help when first tried to use the site (describe and tally): ____________________

   __________________________________________________________________________________

   __________________________________________________________________________________

5. Requests for help during later site use (describe and tally): _____________________________

   ______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

6. Requests for help on answers to tutorials/ quizzes (describe and tally): ___________________

   __________________________________________________________________________________

7. Reading Problems (describe and tally): __________________________________________________

   __________________________________________________________________________________

8. Student appeared to enjoy (identify features): _____________________________________________

   __________________________________________________________________________________

9. Student appeared to dislike (identify features): ____________________________________________

__________________________________________________________________________________

1. Grade level of class: _______ Name of class: _____________________________________________
2. Class Official Functioning Level is: ___ Regular. ___ Fundamental. ___ Honors. ___ Gifted.
3. _____ Total Number of Students in Class _____ Number of Study Participants
4. Teacher appears to understand: ___ the purpose of the study. ___ Student tasks.

   ___ Content Covered. ___The nature of the WITS site.

5. Site of Student Use of WITS: ___ Classroom. ___ Another setting (identify) __________________.
6. This study appears: ___ not problematic. ___ Upsetting to teacher. ___ Upsetting to Students.

   Notes about this: ___________________________________________________________________

7. Teacher appears: __ Uninterested. __ Somewhat interested. __ Interested. __ Highly interested.
8. Teacher apparently: ___ is comfortable using computers. ___ Has a computer in the classroom.

   ___ Has a computer at home. ___ Uses the Internet: ___ easily. ___ adequately. ___ Not at all.

   ___ Likes the idea of computer or web-based instruction. ___ Does NOT like computers.

9. Teacher access to student results: ___ had trouble. ___ had some problems. ___ had no trouble.

___ did not want student WITS results. ___ Received Printouts ___ Used e-mail results.

10. Teacher suggestions for WITS records access and maintenance: ______________________________

_________________________________________________________________________________

11. Teacher appears to like: ______________________________________________________________

_________________________________________________________________________________

12. Teacher suggests: __________________________________________________________________

Document E: Learner Interview Learner ID# _____________ Teacher ID# ______________

1. How much help did you need when you first sat down at the computer? (Describe what happened.)

__________________________________________________________________________________

_________________________________________________________________________________

2. Did you need ongoing help after the first few minutes? (Describe.) _____________________________

__________________________________________________________________________________

__________________________________________________________________________________

3. What were some good things about using the WITS site? ____________________________________

__________________________________________________________________________________

__________________________________________________________________________________

4. Were there features in the site that you liked a lot? ___ What? ________________________________

__________________________________________________________________________________

_________________________________________________________________________________

5. Were there features that you didn't like at all? ___ What? __________________________________

__________________________________________________________________________________

_________________________________________________________________________________

6. What changes in the site itself might make it easier for you to use? ____________________________

__________________________________________________________________________________

_________________________________________________________________________________

1. How much help did your students seem to need when they first sat down at the computer? (Describe what happened.) _______________________________________________________________________

__________________________________________________________________________________

2. Did your students seem to need ongoing help after the first few minutes? (Describe.) ______________

__________________________________________________________________________________

__________________________________________________________________________________

3. Was it easy to use WITS in your usual classroom setting? (Describe any problems.) ________________

__________________________________________________________________________________________________________________________________________________________________________

4. What adjustments did you have to make? _________________________________________________

__________________________________________________________________________________

5. How easy was it to get your student results? _______________________________________________

__________________________________________________________________________________

6. How did you get your student results? ___________________________________________________

_____________________________________________________________________________________

7. What would be the easiest way for you to get your student results? ____________________________

__________________________________________________________________________________

8. What were some good things about using the site for your class? ______________________________

__________________________________________________________________________________

9. Were there features in the site that your students liked a lot? ___ What? ________________________

_________________________________________________________________________________

10. Were there features that you didn't like at all? ___ What? __________________________________

_________________________________________________________________________________

11. Were there features that your students didn't like at all? ____ What? __________________________

_________________________________________________________________________________

12. What changes in the site itself might make it easier for your students to use? ____________________

_________________________________________________________________________________

__________________________________________________________________________________