NEW YORK INSTITUTE OF TECHNOLOGY
SCHOOL OF EDUCATION and PROFFESSIONAL SERVICES
Master of Science in Instructional Technology
EDPC 691 Field Project
Fall 2006
Course Syllabus
Robert S. Slotnick, Ph. D.
Professor's office hours, telephone, address:
Before and after class and/or by appointment.
(o) 212-261-1541; e-mail: rslotnic@nyit.edu
webpage: http://iris.nyit.edu/~rslotnic/index.html or if you have a Board of Ed block: http://iris.nyit.edu/rsuftcourses/index.html
Office: NYIT, 17 W. 60 St. Information Hall/Education, 2nd fl. Room 227, New York, NY 10023NYIT HelpDesk: Office of Information Technology
Tel: 516-686-7570
E-mail: helpdesk@nyit.edu
Web Site: http://helpdesk.nyit.eduCourse Calendar and Room: Field Project meets Tuesdays from 4:30 to 6:45; Classes meet in new building on 16 W. 61 St. room 723, 7th fl. (MC61 723) Please get up to date photo ID which can be obtained in Information Hall, 21W. 60th St.,
Calendar: Every Tuesday from 9/12/06 to 12/12/06 and 12/19 for make-ups and emergencies. No scheduled holidays.
Course Description: EDPC 691.M01 Field Project
This is the culminating course in the Instructional Technology program. The student will carry out an applied research project in a chosen area of expertise. The project synthesizes work of the degree candidate's program and demonstrates comprehensive knowledge of instructional technology applications in learning environments. It is suggested that the chosen project be a follow-up or continuation of work completed in the Curriculum Design and the Research Practice and Principles courses.The field project will be based on the design and implementation of an education/training unit, module or device which was developed in previous courses. The final field project requires a detailed written report of the project's problem, context and background, relevant literature review, design, methodology, data analysis, discussion and conclusions.
<>Required Texts:
Patten, M. (2001). Questionnaire Research: A Practical Guide. 2nd edition. Pyrczak Publishing, P O Box 39731, Los Angeles, CA 90039; ISBN 1 - 884585 - 32 - 9. (Buy latest edition.)
Web Site: http://www.prenhall.com/gay
Highly Recommended:
Text has many articles that serve as models for research reports.
Lyne, Lawrence, 2003. A cross section of educational research: Journal articles for discussion and evaluation (Ed).
Pyrczak Publishing, P O Box 39731, Los Angeles, CA 90039Recommended Resource Materials:
APA publication manual. (1994). Washington, DC: American Psychological Association.Basch, Reva 1999. Researching online for dummies: A reference for the rest of us. IDG Books
Dick, W. and Carey, L. 1996. The Systematic Design of Instruction. Harper Collins, Publishers.
Fraenkel, Jack, R. & Wallen, Norman, E. Latest ed. How to design and evaluate research in education. New York, N.Y.: McGraw-Hill Pub. Co.
Gay, L.R. & Airasian, Peter (2000). Educational research latest ed. Prentice Hall, Inc. Pearson Education. Upper Saddle River, NJ. ISBN 0 - 13- 096103 - 5 The text may be purchased at: NYIT bookstore (Barnes and Noble) at 1855 Broadway, NYC, NY 10023; tel. 212- 261-1551.
Moore, D.S. 1996. Statistics and concepts and controversies, 4th ed. W H Freeman and Co., NY. ISBN: 0 7167 2863 - X
Patten, M. (2004). Understanding research methods (latest edition recommended) Pyrczak Publishing, P O Box 39731, Los Angeles, CA 90039; ISBN 1 - 884585 - 22 - 1 The text may be purchased at: NYIT bookstore (Barnes and Noble) at 1855 Broadway, NYC, NY 10023; tel. 212- 261-1551.
Reiser, Robert & Dick, Walter (1996). Instructional planing: A guide for teachers. Simon & Schuster, Needham Heights, MA 02194. ISBN 0-205-16614-8.
Sagan, C. 1995. The demon haunted world: Science as a candle in the dark.
Random House: New York. ISBN: 0 394 53512 X
Course Requirements:
See end of syllabus for description of assignments.
1. Attendance and class participation are required and contribute to your grade.
2. Discuss and answer selected topics from Questionnaire Research.
3. Formulate Your Research Plan: PowerPoint: List in outline form and annotate components.
See Text, Ch 3, p.87 to p. 93.
4. PowerPoint presentation of each section of your project. Follow design principles for presentation format.
5. Write an Introduction and Literature Review relevant to your problem statement and hypothesis. All articles must be current and relevant to the problem. Articles must be from professional periodicals. A majority of articles must be primary sources and empirical reports. The literature review should move from a general statement of the problem to more specific sub-topics relating to your main topic and leading to your hypothesis or goals and objectives. The literature review should show mastery of the general topic and lead to your specific project.
6. Method & Procedure -- List and describe all the components of the Method section (particpants, setting, instruments, equipment, materials, design, IV, DV, and Procedure). For Procedure, describe in narrative form how the entire research project was conceived and implemented.
7. Results and Discussion -- What dependent variables are you observing and measuring? Will you measure achievement? behavior? affect? How will you measure the main dependent variables? Survey items? Interviews? Achievement tests? Behavioral observation? What statistical tests will you employ? What tables and figures will you construct to organize and visualize your data? Where appropriate present graphs and charts of your data.
8. Submit completed Field Project Report. Edit, revise and correct all sections of your research and submit a final project. Introduction, Statement of Problem, Hypothesis; References; Research Plan; Subjects and Setting; Instruments, Materials and Design -- identify Independent Variables, Dependent Variables; Procedure; How to use Statistical Tests; How to Measure Dependent Variables; Discussion; Abstract and Table of Contents; PowerPoint presentation of complete Field Project Report.
9. PowerPoint Presentation of your Field Project!! (Part of number 4 above.)
Diversity
Students investigate individual differences in performance and identify possible contributing factors including physical, emotional, sociological, cultural, ethnic, and linguistic differences. In developing intervention models for the learner, candidates reflect on the implications of these variables and design instructional approaches which will maximize learner strengths. In selecting a research topic, students may incorporate the impact of race, ethnicity, emotional and physical challenges, and cultural and linguistic differences into their research. Diversity issues can enrich and make your topic more salient.Grading Policy:
1. Attendance and class participation 10%
2. Formulate Your Research Plan 10%
3. Prepare a review of the literature 10%
5. Complete Method Section 10%
6. Results and Discussion Section 20%
7. Submission of Complete Field Project Report 20%
8. PowerPoint Presentation of Field Project 20%
Ground Rules:
In order to receive an "A" grade, a project must be above average in design, relevance, and implementation. Length or effort alone is not sufficient to merit an "A". Projects that are entirely satisfactory will be graded as "B". B+ and C+ will be used for work that is above average and below average. Other grades are C and F.Submit all assignments on time. Assignments submitted only at the end of the term will not be accepted or will receive a lower grade.
Using any other persons' work without acknowledgement or proper citation is plagiarism and is not permitted. (Plagiarize: 1. To use and pass off as one's own (the ideas or writings of another). 2. To appropriate for use as one's own passages or ideas from another. When you credit source and use quotations, plagarism is transformed to scholarship.
Please submit all written work, double-spaced, using a word processed format to make revisions easier. Written work will be revised until a clearly written acceptable document is submitted. Use the APA publication manual as a guide.
Incompletes will be given only in extreme circumstances and, when the coursework is completed, the final grade will not be higher than B, except in unusual cases.
You are allowed two absences. Additional absences may lead to a reduction in grade or a withdrawal from class.
Coming late is disburbing to other students and the professor. Please do not make it a habit or you will be asked to leave for that class. If there is a neccessary reason to come late, please speak to me. Lateness will be counted as 1/2 absence.
Please be considerate of your fellow classmates; do not leave trash, do not talk during class, do not disturb others.
If you miss class or do not understand some information ask a classmate to share notes. You are responsible for making up all work. Consult the webpage for assignments.
No eating or drinking in class.
To receive your grade promptly after I submit it,
you can check NYIT Connect, login, and follow prompts for your grade.
Papers and other assignments cannot be stored at the college; students are responsible for supplying stamped self-addressed envelopes for the return of all work.
These suggestions are intended to clarify and to expedite class activities. I hope that you find this a positive experience.
I want to make this the best course possible for all students; please tell me what you liked and make comments that you think will improve this course. Thank you.Course Objectives
Students will:
Identify a research topic
Conduct a literature review of that topic
Learn to read research articles critically
Construct a research design to answer a research question
Apply quantitative and qualitative techniques to research data
Write a complete report on their research results
Make an oral presentation of their project using PowerPoint
Course Topics and Schedule:Weeks 1. and 2. Introduction to Field Project and Formulate Research Plan
Introduction: Prepare for Field Project Implementation
See full explanation, Text: pp 102-111: Evaluation of a Research Plan
Use these components to construct an outline and a plan for your research.
Identify your research question.
Adopt a research methodology to implement the research process.
What is your Research Plan -- Focus on overview
What is your basic research design? Qualitative or Quantitative? or a combination?
The way you frame your research question willl determine your research strategy
Experimental! Survey! School-Based Action Research! Case study!
What is your driving question?
Make your project intrinsic and motivating
Suggest that you choose a school-based problem for interest, convenience and efficiency.
Question can be as basic as "how much have my students learned on a particular topic" to comparing two groups
performance as a consequence of a particular treatment. The main point is to ask the question clearly and
then figure out how to measure the results for that question.
Class: Start working on outline for PowerPoint.
Principles of visual design; New technologies of reproduction flood us with graphic images, Advertising is increasingly visual, We learn from what we see.
Visuals are iconic, Words arer abstract/arbitrary, Images resemble what they represent, Visuals contain motivational qualities, Visuals simplify complex
information, Charts and diagrams are easier to store and retrieve information.
PowerPoint: Keep it simple
No more than 4 or 5 points per slide. Guide the viewer toward the essence of the idea. Do not list all attributes. Do not clutter screen with too much text, graphics or color. Use large font sizes. Make it easy to read. Use bullets. Avoid complex grammar. If you have additional information, then add notes at bottom of slide!!
Assignment 1: PowerPoint Presentation: Formulate Your Research Plan. Identify all components. Prepare an outline and describe how it fits into your research plan. Focus on research question and research design. What is the question you are asking and how are you going to answer it. See Text, Ch 3. pp 102 - 111. See Hand-out # 14 Present in PowerPoint format.
<>The following section should help you formulate a research plan by showing
how scientific method helps you frame questions and evaluate data.
Scientific Method
Science and the evaluation of evidence
Base decisions on data
Understand the power of random selection
Scientific Theories help formulate hypotheses which are predictions based on theory and past experience
Parsimony and Occam's Razor -- select the simplist explanation
Conducting Research
Observational Research Designs
Naturalistic and Field Studies /Case Histories/Surveys/Correlations
Experiments
Measuring and Analyzing Results -- Collect Data, Display Data -- Charts and Graphs
IV Treatment and DV Outcome
Use statistics to compare results
CD Rom: PowerPoint Presentation from Kalat on SciMeth.
Excellent web site on sci meth
Questionnaire Research, See Partial List of Assignments at end of Syllabus.
http://phyun5.ucr.edu/~wudka/Physics7/Notes_www/node5.html
Use questionnaire items as a dependent variable; it gives shape to the whole research project.
Ch1: Planning Questionnaire Research, written objectives then
written items, timeline -- major steps in conducting questionnaire research;
Ch2: Writing Items to Collect Factual Information, Suggests format and style for writing items.
Ch 3: Writing Items to Collect Demographic Information
Demographic items give a mental picture of the participants.
Week 3. Method section -- Overview
Participants, setting, instruments, materials, apparatus, design, procedure
Describe participants and setting.
What instruments are you using? Teacher created or standardized?
Instruments: Sample tests, exams, surveys. Construct test items.
Are your tests appropriate to your question? Do your tests reflect the desired goals?
If School Action Research, then tests must reflect goals and material taught in class.
What is a variable? Any concept that has at least 2 values...
Quantitative versus Qualitative Variables -- ex: age, weight, IQ and gender, religion, political preference
Type of Variable//Statistical Test -- ex: categorical and chi square, quantitative -- correlation or t-test
Independent versus Dependent Variables -- cause vs outcome
Extraneous Variables: Moderator and Control Variables
Hypotheses
Questionnaire Research: Ch 4 Writing Items to Measure Attitudes, Exercises 1 - 11. Practice writing survey items. Think about scoring items.
Questionnaire Research: Ch 7 Preparing a questionnaire for administration
CD ROM, Kalat, Ch 14.2, PowerPoint Presentation on Attitudes, Types, Likert, Scoring, Open to Perusasion
Week 4. Introduction to Results
What is your hypothesis?
What are your independent and dependent variables?
Cognitive -- Achievement results, Multiple Choice, Essay, Portfolio
Affective -- Attitudes, Likert Scale, Persuasion
Behavioral
How will you measure each?
How will you assess each measure?
Describe your results descriptively! inferentially?
(Measure, assess, describe, compare (infer)
Interviews -- Can be qualitative or quantitative, gives human touch, personal interpretation
of situation, enriches quantitative data
Week 5. Action Research in the Schools, (Text Ch 17) New Ed., Ch 9)
School -Based Action Reseaarch
Professionalizes teaching activity
Improves teacher practice
Focuses on teacher problems
Data can be quantitative or qualitative
Generates information/data on important teacher activitiesIdentify school problem.
Conducted by teachers (sometimes with administrative participation).
Create an educational intervention.
Understand effects
Take action based on results
to effect positive educational change
Makes teacher role more meaningful
More helpful to students
Give examples from class that could be used in a research project
Text, Ch 13 Experimental Research...
Pre-Experimental Designs
The One-Shot Case Study X O
The One-Group Pretest-Posttest Design O X O
Static Group Comparison X1 O
X2 O
Quasi Experimental Designs
Non Equivalent Control Group Designs
O X1 O
O X2 O
Questionnaire Research: Ch 6 Conducting Item Tryouts and Item Analyses, Exercises, 1 - 9. Important aspect of research. Focus on items. Pilot test and improve your items. Revise and improve. Focus on what question you want to ask.
Weeks 6. and 7. Method Continued: Design and Procedure
Research Design
School Based Action Research, Ch 9
Type of Design, Ch 13
Group Designs: Pre Experimental Designs
One Shot Case Study
One Group Pretest-Posttest Design
Static Group Comparison
Indentify - Independent and Dependent Variables
Focus on Procedure
Review all components of project
Narrative of entire field project; what will be done, how it was done
Set the implementation schedule -- get the topic; get approval for location, subjects
a) Outline feasibility and access
b) Lead time to develop; Time to implement
c) Comparison group, if feasible
d) Formative evaluation -- run small sample
e) Modify and revise based on sample feedback
f) Summative Evaluation: Full subject implementation; actual project, real time
Construct timeline of entire research process
Assignment 3: Complete Method Section -- include participants, setting, materials, (directions, instructions, lesson plans, etc.), instruments -- achievement tests, attitude scales, survey questions, etc., research design -- type of research (experiment, survey, etc.), independent variable -- treatment or impact variable, dependent variable -- outcome (achievement, attitude, behavior). Complete procedure narrative.
Questionnaire Research: Ch 9 Preparing Statistical Tables and Figures, Exercises, 1 - 12.PowerPoint
Weeks 8 and 9. Results -- Prepare different ways of analyzing results. Text, Ch 13, and 14
Describe your dependent variables. Achievement! Attitude! Behavior! How were they measured?
For those dependent measures what statistical tests should you use?
Use actual or made-up data, but analyze correctly (use Stat Pak or Excel where necessary) and present in proper tables, figures and graphs.
Also write a short statement explaining why you used that statistic, what it tells you and what does the test of significance mean.
How to calculate mean, sd, and % for Likert Scale!
How to create charts and graphs!
Hand-out # 10 Results SectionPrepare results from Questionnaire Research on Likert Survey Items.
Assignment 4: Results and Discussion -- List and explain descriptive statistics and inferential statistics you plan to use in your research. See text, chapters on statistics. Organize Results Section. Present Tables and Figures. Present descriptive data -- means, standard deviation, percentages, etc. Present inferential data -- statistical tests (t-test, chi sq, etc.), hypothesis testing, probability estimates, levels of significance, (discuss practical vs statistical significance). Discussion -- hypothesized and unhypothesized results, significance of research data, relevance of results.
Results: Presenting Tables, Figures, Charts and Graphs
Statistical Analysis: Descriptive, Inferential, Statistical Tests and Significance
Questionnaire Research: Ch 10 Describing Averages and Variability
<>Weeks 10. Preparation of a Research Report, Text, Ch 15
Guidelines, Rules for Writing, Format and Style
Common Components of a Research Report, Table 15.1
Main Body of the Report
Introduction, description of the problem, review of literature, statement of hypothesis,
and definition of terms
How to Present Results: Tables, Figures, Charts and Graphs <>
Questionnaire Research: Ch 11 Describing Relationships, Exercises, 1 - 9.
PowerPoint Presentations
Week 11. Prepare Literature Review, See Text Ch 2
Review work from last term
Is it appropriate for this term's project
Are Topic, Hypothesis, IV and DV appropriate for your current project?
References: Current and Relevant
Introductory section -- introduction, statement of the problem, literature review, hypothesis or goals and objectives.
What is the problem you are addressing? State it clearly.
What do other researchers say about this problem?
What research have they done?
What research are you planning?
Narrow the topic
Identify your sources -- Computer Databases, ERIC search, retrieving journals
Abstract, annotate your sources
Analyze, organize your information
Integrate your narrative
Review
Title, main theme, Problem Statement, Hypothesis
Organization of lit rev, subtopics, articles supporting your discussion
Format, style, grammar and syntax, flow...
Do you have something to say?
Is it interesting or boring?
Assignment 2: Prepare a review of the literature of the problem you are studying. Include: Introduction, Statement of the Problem, Review of the Literature, Hypothesis or Goals and Objectives, and complete list of references. Review of the literature should survey the information available on the specific question and related questions. Give the reader a full overview of the scope, significance and information available on the topic. See Text, Ch 2. Introduction and review of the literature will be used for your field project report.
Week 12. Evaluation of a Research Report, Text, Ch 16
Review criteria for published studies to apply to your research effort.
See Table 16.1 Specific shortcomings of research articles
Apply criteria to your study, p. 583 - 5
Introduction -- Problem, Review of Literature, Hypotheses, Method, Results, Discussion, Abstract, References, Tables, Figures. What research model are you using? What are its strengths and weaknesses?
Week 13. Individual Conferences
Review Final Projects
PowerPoint Presentations
Weeks 14. and 15. Field Project Report and PowerPoint Presentation
Review Final Projects
PowerPoint PresentationsAssignment: 5. Submission of Complete Field Project Report --Abstract, Table of Contents, List of Tables and Figures, Format of Tables of Figures, Conclusions, and References, Appendices.
Complete PowerPoint Presentation
Return to top of page.
Partial List of AssignmentsA. Attendance and class participation
B. Understanding Research Methods Assignments
Class discussion as relevant.C. Questionnaire Research Required.
Chapters: 1, 2, 3, 4, 6, 7, 9, 10, 11.
D. PowerPoint Presentation of each section of your Field Project Report. Present 2 or 3 screens for each section of the Field Project report..
E. Lyne, Lawrence, 2003. A cross section of educational research: Journal articles for discussion and evaluation (Ed). Pyrczak Publishing. Discussion of selected readings (journal articles)
1. Formulate Your Research Plan/PowerPoint Presentation. Prepare an outline of all components. Describe how components fit into your research plan. What is the research question you are asking and how are you going to answer it. What is your research design? See Text, Ch 3, Components of the Research Plan for a list. Do not write a narrative. Use an outline format.
2. Prepare a review of the literature of the problem you are studying. Include: Introduction, Statement of the Problem, Review of the Literature, Hypothesis or Goals and Objectives, and complete list of references. Review of the literature should include information available on the specific question and related questions. Give the reader a full overview of the scope, significance and information available on the topic. See Text, Ch 2.
3. Complete Method Section -- include participants, setting, materials, (directions, instructions, lesson plans, etc.), instruments -- achievement tests, attitude scales, survey questions, etc., research design -- type of research (experiment, survey, etc.), independent variable -- treatment or impact variable, dependent variable -- outcome (achievement, attitude, behavior). Complete procedure narrative.
4. Results and Discussion -- List and explain descriptive statistics and inferential statistics you plan to use in your research. See text, chapters on statistics. Organize Results Section. Present Tables and Figures. Present descriptive data -- means, standard deviation, percentages, etc. Present inferential data -- statistical tests (t-test, chi sq, etc.), hypothesis testing, probability estimates, levels of significance, (discuss practical vs statistical significance). Discussion -- hypothesized and unhypothesized results, significance of research data, relevance of results.
5. Complete PowerPoint presentation of your entire field project. Combine all previous sections into one final, complete report.
6. Submission of Complete Field Project Report -- Abstract, Table of Contents, List of Tables and Figures, Format of Tables of Figures, Conclusions, and References, Appendices. Write in your very best field project style.
Journals
Computers in the Schools
Education, Training, Research, and Development
Educational Technology
Electronic Learning
International Journal of Instructional Media
Internet World
Journal of Computing in Childhood Education
Journal of Computing in Teacher Education
Journal of Educational Computing Research -- many relevant articles on computers and surveys.
Journal of Educational Psychology
Journal of Educational Research
Journal of Instruction Delivery Systems
Journal of Technology and Teacher Education
Learning and Leading with Technology
Optical Information Systems
Phi Delta Kappan
Technology and Learning
Technology and Teacher Education Annual
The Computing Teacher
From Carl Sagan, The Demon Haunted World
Why should we subsidize intellectual curiousity? Ronald Reagan, Campaign Speech, 1980There is nothing which can better deserve our patronage than the promotion of science and
literature. Knowledge is in every country the surest basis of public happiness. George Washington,
address to Congress, January 8, 1790.Insight, untested and unsupported, is an insufficient guarantee of truth. Bertrand Russell,
Mysticism and Logic, 1929.Ignorance more frequently begets confidence than does knowledge: it is those who know little,
and not those who know much, who so positively assert that this or that problem will never be
solved by science. Charles Darwin, Descent of Man, 1871.All our science, measured against reality, is childlike and primitive -- and yet it is the most precious
thing we have. Albert Einstein.We wait for light but behold the darkness. Isaiah.
It is better to light one candle than to curse the darkness. Adage.
Education is our best beacon of light. Anon.
Questionnaire Research
Ch 1: Planning QR
All methods of collecting data have pros and cons
QR is similar to tel interview and personal interviewAdvantages of QR
Efficient way to collect data
Can query large groups of people simultaneously and anonymously
Can collect data on controversial issues -- drugs, sex
Not expensive to collect data
Data are collected in simple empirical format
Can score data easilyDisadvantages
Response rate is low if quest are mailed
Does not give in-depth picture/works best with objective q.
Quick snap shot
Social desirability biasPrepare written objectives
Prepare a written list of the objectives for your research
Specific objectives keep you on track
Example 1.2.1 good, but too general
Improved version
divides one objective into 8 parts, increasing specificity
Examples: 1.2.2 and 1.23Guideline 1.3
Have others review your objectives and items
Avoid flashy but superfluous or superficial items
Define objectives and stick to themGuideline 1.4
Review Lit related to the objectives
See how others formulated their objectives
See the relationship bet lit rev and objectivesGuideline 1.5
<>Guideline 1.6
Need permission to admin q to children
If q is part of ed lesson may not need permission
Prepare a timeline -- not just for attitude scale but for the whole project
Timeline divided into weeks -- good scale for your project
Days are too specific and detailed; months are too sketchy
Major steps (specific activities) connected to each week
Major steps (specific activities) give you an outline of the entire research project
What are some of the activities you should put into the timeline?
For example, select topic, conduct eric search, identify empirical articles,
revise topic, develop hypothesis, focus on IV, DV,
<>Chapter 2: Writing Items to Collect Factual Information
"Each item in a questionnaire should relate to a specific objective
of your research
Poorly written items frustrate and confuse participants
Guideline 2.1 Consider asking about behavior over a recent time period
Have you ever used email? Leaving time period open is too vague. Suggest
participants recall amount of time over last week.
Some participants will have an unusual week either too little or too much time spent on email or internet.
Will that bias research?
If random sample those differences will balance out.
Otherwise there is systematic bias in sample.
Guideline 2.2 Use always and never with caution
Always and never are polarizing statements
Think what circumstances always would be appropriate and under which would not...
Guideline 2.3 Avoid using negatives in statements
Negative statements in the item will conflict with the choices.
I can't stop thinking about the death: Very rarely, Rarely...
Too confusing
Guideline 2.4 Ask only a single question for each item
Separate student status from occupational status
Guideline 2.5 Make the choices for an item exhaustive
...
Guideline 2.8 Use "Don't know" sparingly
Guideline 2.9 Be specific in your request for information
<>Chapter 3: Writing Items to Collect Demographic Information
Demographic items request info background characteristics such as age, ethnicity, race, religion, years of education, type of work, annual salary, etc. Sometimes this infor is needed to fulfill a research objective. A researcher will need to ask participants their age if interested in voting patterns of elderly compared to young people or to determine memory of elderly compared to youth. Demographic items give a mental picture of the participants.
However, demo items can be very sensitive and not relevant to the study. They can irritate or anger respondents and affect their participation. Demo items make Q longer. Lower the response rate. Risk angering respondant.
When using demo items, learn to ask for information in less intrusive ways. See Example 3.3.1 for financial information. See example 3.4.1 for age ranges. Give respondents ranges to answer, not a specific amount.
See Guideline 3.5 Use care when writing questions about race or ethnicity.
Guideline 3.6 gives suggestions on how to frame a range and to phrase a category. Compare example 3.6.1 to 3.6.2 for marital status. For level of education completed compare Example 3.6.3 to 3.6.4. The latter is clearer and more precise.
Conclusion: use demographic items only as needed for the research; always ask for permission first. Do not invade people's privacy. Place demographic items at end of questionnaire. If it angers participants or disturbs them in some way, they have already completed the most important part of your survey.
Ch 4: Writing Items to Measure Attitudes
Attit is a general predisposition toward people, ethnic groups, organizations, etc.
To measure attits we ask q about feelings, actions, and potential actions
To get a good sample of student attitudes toward school we need a collection of items that sample feelings,
actions and potential actions. To get student attitudes school we need to write objectives for feelings, actions (behaviors) or potential actions. Objectives sample the three categories. Then we write attitudes that reflect the objectives.
Attitudes are different from opinions which are usually a person's belief about ...Guideline 4.1
Examine attitude scales that have already been developed. Sometimes hard to do because they often become proprietary products.
Search the ERIC data base for attitude scales and other search engines, e.g., Google.Guideline 4.2
Use Likert-type items.
The basic scale is: SA A N D SD
If appropriate to your research, keep those standard choices.
5 point scale is the standard. You can extend it to 7 if necessary. You would use a 3 point scale for young childrenGuideline 4.3
The statement should deal with only one point. Do not ask about 2 separate points in a single item. See example.Guideline 4.4
Distinguish between Neutral or Undecided items. Necessary but not helpful to attitude scale.
It can be used as a way of not declaring their position or attitude on a controversial issue.Guideline 4.5
Use Don't Know sparingly. It is different from Neutral or Undecided. It is not on the same continuum as SA .. N... SD.
Not knowing about a topic is not the same as being neutral about it.*Guideline 4.6
Use multiple items in an attitude scale.
Most attitudes are complex constructs consisting of many elements.
A respondent may have positive feelings toward some elements and negative ones toward others.
Asking only a global question, such as the one in Example 4.6.1 I am satisfied with my job,
to measure attitude toward work, oversimplifies the construct. A better measure of attitude toward work
can be obtained by writing a series of tiems about the desire to be employed, the need to be productive,
the willingness to follow directions on the job, internal and external rewards of working,
coming on time, being prepared for the work task, and so on ...How many items you write depends on the number of salient charaacteristics people associate with the construct.
You have an overall construct, then you have salient characteristics and then you write items for each specific salient
characteristic which will then make up the attitude scale.Guideline 4.7
Prepare a list of the broad components of an attitude, and use it as the basis for writing an attitude scale.
How would you write a questionnaire to meaure the attitudes of public school teachers toward the teaching profession.
First break the teaching profession into components (salient characteristics):
1. wages and benefits,
2. psychological rewards of teaching,
3. status of teaching in society,
4. interactions with children,
5. interactions with administrators,
6. interactions with parents,
7. interactions with other teachers,
8. financial/physical constraints on being an effective teacher,
9. educational requirements to enter the teaching profession,
10. opportunities for advancement...
You could list many more, depending on your interest and experience.
Here are two more:
a. enjoy reading new materials
b. enjoy teaching new materials to students
You see writing an attitude scale involves
identifying a broad concept, then breaking down the concept into many component
parts, then writing attitude items for each specific component.
Check with teachers on most salient characteristics.
Use components to guide writing of items.
Have a small group answer items and give feedback for clarity, grammar, salience, etc.
Use feedback to fine tune items.
Write new attitude scale.
Test on group.
Guideline 4.8
Make some items favorable and some unfavorable.Guideline 4.9
Consider asking about reactions to hypothetical situations when necessary.Guideline 4.10
Only use items that are clearly indicative of a favorable or unfavorable attitude.
Questionnaire Research, Chapter 6: Conducting Item Tryouts and an Item Analysis
Guideline 6.1 Have your items reviewed by others
Do not be defensive; be receptive
Feedback on items
Improve quality
Reliability and validityGuideline 6.2 Conduct "think-alouds" with several people
Have colleagues answer items while thinking aloud
Identify items that are ambiguous
Revise items to be clearer, simpler...
Use feedback to improveGuideline 6.3 Carefully select individuals for think-alouds
Guideline 6.4 Consider asking about 10 individuals to write detailed responses on a draft of your questionnaire
Will give feedback that you can use to revise selected items.
Guideline 6.5 Ask 25 or more respondents to respond to the questionnaire for an item analysis
Sample group of subjects answer all items in questionnaire
This provides the basis for an item analysisGuideline 6.6 In the first stage of an item analysis, tally the number of respondents who selected each choice
You can convert the tallies into percentages. For example, if 4 out of 25 respondents mark "Excellent",
the percentage is 4/25 = .16 x 100 = 16%. You want to identify choices marked by few and items marked by many.
Example 6.6.1 shows percentages for an item designed for use with high school students. Choices do not reflect actual use and therefore give an inaccurate picture. This information calls for changing the type of choices. See Example 6.6.1 and compare to Improved Version of 6.6.1. By changing the categories the researcher obtained more information about student drug use.Consider Example 6.6.2. What is the basic outcome of this item? Almost all respondents (95%) endorsed the item. Therefore the item is not providing any new information. Consider dropping that item from the survey. Sometimes when respondents endorse an item you still want to keep it, if the information is important.
Guideline 6.7 In the second stage of an item analysis, compare the responses of high and low groups on individual items.
First you need to identify respondents who are hi and lo on a attitude scale. Consider the survey attitudes toward statistics. See Example 6.7.1. 4 items from the scale are presented. The scale is a 5 point Likert scale: SA A N D SD. Each choice has a numerical value, 5,4,3,2,1 or 4,3,2,1,0. Notice that positive feelings are scored 4,3,2,1,0 and negative feelings are scored in reverse. See text for exact score. How do you get a full score for the scale? In this example we have 4 items, each item is scored from 0 to 4, therefore a 4 item scale can have a maximum score of 16. A 5 item scale would have a maximum score of 20.Now we will review how an item analysis helps you evaluate an attitude scale. Consider Example 6.7.2. Item: I enjoy doing my statistics homework. Suppose we had 36 respondants. For this item analysis we divide the respondents into thirds. We take the respondents with the highest score and compare them to the respondents with the lowest 1/3 total score. This is illustrated in Example 6.7.2. The highest third also score the highest on the selected item. The same is true for the lowest third. This means that the item differentiates between those who have a high score and those who have a low score.
Now let us look at an item that is not discriminating properly. See Example 6.7.3. The pattern of response for the top scoring respondents is similar to the pattern for the lowest; those with the most favorable attitudes toward statistics had a similar response pattern to the item as those who had the least favorable attitudes. Thus, the validity of the item is called into question. The perception that statistics has many details seems to be common among all respondents low scorers and hi scorers. Thus, the item does not contribute to the overall measure of attitude toward statistics.
Questionnaire Research, Chapter 7: Preparing a Questionnaire for Administration
Guideline 7.1 Write a descriptive title for the Q. See two examples in the workbook. Which are most descriptive?
How does this help the attitude survey?
Guideline 7.2 Write an introduction to the Q.
Guideline 7.3 Group the items by content, and provide a subtitle for each group. See ex. 7.3.1
Questionnaire Research, Chapter 8: Selecting a Sample of Respondents
Questionnaire Research, Chapter 9: Preparing Statistical Tables and Figures
Need to analyze data. Analyze data in tables and figures.Guideline 9.1 Table of Frequencies
Report separately on each item. Determine how many respondents marked each choice.
Example 9.1.1 shows the numbers that marked each choice in two of the items. Since the values in it are organized and ordered, it is called a statistical table.
Notice for Item 1: Encourages stu partici, the item can be rated from Very Poor to Excellent in 5 equal steps. Excellent gets a 5, Very Good = 4, and so on until Very Poor which gets a 1.
To get the mean or average ratings of the class for that item, you need to multiply the selected value by the number of students that selected it. So, if 5 stud rated the item as Excellent then 5 x 5 = 25 for that choice. You continue with the rest of the choices for that item and you get 25+8+36+8+2 = 76/25. For item 2 you have 65+36+6+1=108/25=4.32. By comparing the frequencies of response to the two items lets you see the overall ratings of the two items.The data in 9.1.1 are called frequency distributions. For item 1 the distribution is roughly symmetrical. For item 2 the distrib. is skewed. It is important to know if a distribution is symmet. or skewed.
Example 9.1.2 Distribution of Attitude Scores. The total score indicates the degree to which a respondent has a positive attitude. Determine how many respondents earned each score. Table 9.1.2 shows the number of respondents who scored at each interval. This table can be put into a graph.
Guideline 9.2 Consider calculating percentages and arranging them in a table with the frequencies. Take a look of Table 0.2.1. How can you calculate %? If 5 students out of 25 rate student participation as excellent, then to obtain the % you take 5/25 = .2 x 100 = 20%. Calculate the other percentages in the table.
A major advantage of percentages over freque4ncies is that they make the results comparable across two or more groups of unequal size. Consider Example 9.2.2. Since the group of women has 40 respondents, while the group of men has 80 respondents, comparison of the groups using frequencies can be confusing. However, percentages are not confusing since a percentage indicate the number of respondents per 100.
Questionnaire Research, Chapter 10: Averages and Variability
Use the median as the average for ordinal data. What are ordinal data? Rank order data. Organize results into a rank order list. Then select the number in the middle of the list. It is the median. The median has half the cases above and half below.10.2 Consider using the mean as the average for equal interval data. An equal interval scale would be a ruler. Same distance between 1 and 2 as between 4 and 5. Consider a test from 0 to 100 an equal interval scale. Also, a Likert scale of 5 points is considered an equal interval scale.
Example 10.2.1 Compute the mean from a table. See book.
Example 10.2.2 See table to organize items and mean and sd
1. Use the median as the avg for ordinal data.
2. Use the mean as the avg for equal interval data.
3. Use the median as the avg for equal interval datra that are highly skewed.
Questionnaire Research, Chapter 11: Describing Relationships
...statistical methods to measure relationships between two variables. For example: is there a relationship between gender and political affiliation? or a relationship between attitudes toward reading and attitudes toward school ( and achievement ).
Text, Ch 9 School-based Action Research
Overview
Teachers focus on their own practice observing what they and their students actually do and fail to do.Develop a deeper understanding of teaching and the learning process; develop an awareness of their students behavior and awareness.
Focus is on the classroom and the school but can also impact on the district.
Can involve a pressing problem or a new opportunity to improve morale and learning.
... "action research involves teachers identifying a school-based topic or problem to study, collecting and analyzing information to solve or inderstand a teaching problem, or helping teachers understand aspects of their practice. Action research is educative, focuses on teachers and shcools, focuses on problems of practice, and aims at improving practice." p. 262.
Identify the Problem
Conduct a Literature Review
Develop Research Questions
Collect the Data
Data Sources -- Qualitative and Quantitative
Observation
Interviews
Questionnaires and Attitude Scales
Readily available Data
Collect and Analyze Data
Share the Findings
Implement the Findings in School
Give an example from class
How do you observe your students?
What behaviors do you notice?
List some of your student behaviors which are important
How do these behaviors reflect student involvement or achievement?
Consider doing a behavior observation of some students in class!
Text, Ch 13 Experimental Research...
Pre-Experimental Designs
The One-Shot Case STudy X O
The One-Group Pretest-Posttest Design O X O
Static Group Comparison X1 O
X2 O
Quasi Experimental Designs
Non Equivalent Control Group Designs
O X1 O
O X2 O
Experimental Designs
R O X1 O
R O X2 O
or
R X1 O
R X2 O
Questionnaire Research: Ch 6 Conducting Item Tryouts and Item Analyses, Exercises, 1 - 9. Important aspect of research. Focus on items. Pilot test and improve your items. Revise and improve. Focus on what question you want to ask.
<>
Method and Procedure: Review Components
Participants
Identify the number and characteristics of the sample, age, grade, ethnicity
In what neighborhood is the school
Characteristics of the neighborhood
Instruments
List tests used:
Achievement, Affective, Behavioral
Describe and give examples of each
Short description in method section.
List complete test in Appendix
Design:
Combine Ch 9 and Ch 13 for a good description of your Field Project Design
See example
Procedure
Describe all the steps that will be followed in conducting the study - from beginning to end.
Cite any pretests, procedures of administration
How do you meet with the participants
Where, when, how long
Describe nature of treatment
Be precise in your description
Data Analysis or Results
All data collected must be tabulated and presented.
Use text and workbook as guide for presenting tables and figures.
Data analysis flows from the hypoth. or, the hy: det. the res design which det data analysis.
Distinguish bet. qualitative and quantitative results. They are analyzed and presented differently.
Descriptive Statistics/ Text, Ch 14
First step in research is to describe or summarize . Descriptive statistics summarizes the performance of a sample.
Major types of descriptive statistics are:
Measures of central tendency: mean, median, mode
Measures of variability: spread of scores from the mean.
Measures of relative position: z, T scores,
Measures of relationship: degree to which 2 sets of scores are related: correlation.
Graphing Data
Summarize data in columns, then can graph.
Table 14.2, P. 413
Scores Frequency of Score
76 1
79 4
80 5
81 5
From this table you can create graphs in Fig. 14.1.
Frequency Polygon is a line curve which in this case follows a normal curve.
You can also graph a pie chart which expresses the data in %.
For achievement data (test scores) you can do a Frequency Polygon for whole class or 2 groups.
Other descriptive measures are: Mean and SD, z and T scores.
Standard scores permit valid math operations such as averages of different groups to be compared. Cannot do this with percentiles. See p.422-23 for z scores.
Correlation is a measure of degree of relationship. P. 426 shows aPearson r and Spearman rho.
Survey Data -- can present in a table with items and mean, sd and %.
Behavioral Data: need a method to count or tabulate particular variable, such as, present or absent, late, leave room in middle of class, talking out behavior, out of seat behavior, aggressive behavior, and so on, ...
Think of other behaviors that you can count and tabulate: positive participation, completes homework, or, perhaps social behaviors such as, seeks help, follows direction, handles criticism, initiates and or completes activity.
Include a description of techniques that will be used to analyze data. What kinds of analyses are you going to perform?
Are you using achievement tests? What kind? M-C? Essay? Will you obtain a mean and standard deviation? Will you convert the scores to z scores to better compare student progress?Do you want to describe your test results as a frequency polgygon? Use grade score and number of students who scored each grade. Use Excel and chart.
See Chapters 9, 10, and 11 in Questionnaire Research for ideas for presenting data.