Q: Why do researchers do experiments on university students?
A: Because rats are expensive and you grow attached to them.
Since 2014 I have helped to teach two astronomy units at the University of Western Australia, Our Universe and Our Solar System. These are relatively basic first-year astronomy units designed to be taken by non-science students (art, law, business, etc) as part of their wider general education. “Astro101” classes like these have proved to be not only extremely popular with students from a wide variety of backgrounds, but also extremely important. These classes are often the last formal opportunity to influence the science knowledge, literacy, and interest of many non-STEM students (and future voters).
Research has shown that students’ understanding of key phenomena, such as the cause of seasons and phases of the moon, are fundamental to understanding other scientific concepts, and making sense of new research findings reported in the media. However, research also shows that the traditional methods of teaching (i.e. lectures and exams) are nowhere near as effective as we would like for conveying new concepts, and are particularly ineffective when students come to a class with prior, inaccurate, conceptions about how their reality works.
Some types of assessments and exam questions also fail to show students’ misunderstanding. For example, in a lecture we can tell our students “The sun appears to move across the sky because the earth is rotating.” (And show them diagrams and animations.) Then, in the exam, if we ask a question like “What causes the day-night cycle?” Nearly all of the students will correctly answer “The rotation of the earth causes the day-night cycle.“
Job done, right?
No.
One year, we wrote the questions differently. We gave the students an image like the one below and asked, “If you could see stars during the day, this is what the sky would look like at noon on a given day. The sun is near the constellation Gemini. Near which constellation would you expect the Sun to be located at sunset?” (We took this question from an astronomy concepts test developed at the University of Sydney.)
The answer is Gemini. (Because the earth is rotating, so the stars appear to move with the sun.)
If you got that wrong, you’re not alone. That year, 83% of our students got that question wrong, after studying astronomy for a whole semester. (When I presented this research at a teaching research conference, I asked the audience, of lecturers and professors, for a show of hands for what they thought the answer was. Most of them got it wrong too. Clearly, it’s not just our students that are having problems getting to grips with these concepts.)
We needed a better way to teach astronomy.
In other fields of science, such as biology and conservation, students are often made to keep “observation diaries” where they record, in detail, what they see in the field. These have proved to be very important for doing good science, and very useful educational tools. The idea of observing diaries is also steadily making its way into astronomy education.
The next year, we decided to set our students a new assignment. They were to keep an astronomy observation diary, and were to go out weekly and record what they saw in the sky.
We went through the previous research on observing diaries for astronomy (and other fields) to work out how best to implement this assessment. We showed our students example diaries, we set a small number of compulsory observations (such as noting the phase of the moon over 4 days) to help them get started, we let them choose how they wanted to do the diary (e.g. hand written, typed, online blog), gave them the marking rubric, and held a class to check and compare diaries a few weeks into the semester. In particular, we emphasized that we wanted our students to not just keep a record, but to compare their own observation, and look for patterns over the course of the days and weeks of the semester.
We used a suite of questions carefully developed by other education researchers to create a test to measure how our students’ understanding of astronomy concepts, and positive or negative attitudes towards astronomy and science in general, changed over the semester. We also used the same test in our other astronomy class, but without getting the students to keep observing diaries, as a comparison.
Our data show that the observing diary assignment did help our students gain a better understanding of the astronomy concepts we wanted them to learn, but the effect was small. For example, for the question shown above with the sun in Gemini, the fraction of students who got it right nearly doubled, but this still meant that the majority of our students got the question wrong. The data also show that our students in the class with the diaries gained a slightly more positive view of astronomy than our students in the class without diaries, but again, the overall effect was small.
We also compared what mark students got for their diary with the mark they got in the final exam. The scatter was very large. What this showed was that students who did well in one of the two assessments didn’t necessarily do well in the other. Observation diaries draw on a very different set of skills compared with a conventional exam, and the diaries seem to be a very positive experience for students who do not suit conventional study and assessment techniques.
Our research indicates that getting astronomy students to keep a diary of their own observation of the night sky does have a small but positive effect on both their learning of basic astronomy concepts, as well as their appreciation of astronomy. However, the gains are disappointingly small when considering the large amount of effort put into the assessment by both the students and the lecturers. This may be due to fact that the observing diary task is a very different form of assessment to conventional study practices the students might be used to, and may indicate that much more careful effort is needed to integrate observing diaries into the course and the concepts being taught in the course for the diaries to become a strongly effective educational tool.
Our paper Contribution of self-directed, naked-eye observations to students’ conceptual understanding and attitudes towards astronomy is available here.
David Gozzard 