In the fall of 2016, I implemented benchmark quizzes in my organic 1 classes, which I wrote about here.   The quizzes covered key learning objectives, they were pass-fail, and students had to be essentially 100% correct to pass.  Although things didn’t go entirely according to plan, the outcomes for the class were extraordinary. This past semester, I decided to modify this approach for my introductory chemistry classes.  I had two lecture sessions, with about 210 students total (70 in the morning session, 140 in the evening session).  I built the benchmarks around key learning outcomes for each chapter.  As before, the benchmark quizzes were pass-fail, and students had to answer each question correctly to pass.  Students had 3 opportunities to pass each quiz.  A sample quiz was posted beforehand.  Quizzes were worth 10 points each - 10% of their grade.  However, In order to unlock their full homework grade (also 10%), students were required to pass 6 of 10 benchmarks.  (See this article for more detail about this).    The Logistics Students took the quizzes each week at the beginning of their lab section.   I passed out the main quiz for that week to the entire section.  As students turned in their quiz, they could pick up re-quizzes from the previous two weeks to complete.  (For example, quiz 1 was available in weeks 1, 2, and 3 – but not afterward.)  Because each quiz was pass-fail, it didn’t take me long to grade (about an hour per week).  I simply sorted quizzes by pass-fail, then entered the passing grades.  I kept hard-copy grade sheets for each benchmark.  I recorded a “1” for students who passed the first week, a “2” for the second week, and a “3” for the third week.  This allowed me to track performance week to week.  One of the biggest challenges was staying up-to-date:  I had to post grades each week, so students knew if they needed to re-take the quizzes.  With 210 students, this seemed like the most labor-intensive part of the experiment. The Results Ultimately, I didn’t make it through every benchmark.  I made it completely through 6, and gave a 7 th twice, but at the high-stress end of the semester, I decided to give full credit for everyone.  Here are the results: In general, most students who were going to pass did so in the first week, or in the second.  Very few students passed the third week.  A fourth week clearly was not justified.  The one glaring exception to this was week 4 - naming ions and compounds.  I think that in this case, it took most students longer to really master the ion names and the nuance of naming the compounds, so even the above-average students needed a couple extra weeks before they could get the quiz 100% correct. To be candid, I found these results discouraging:  I designed these quizzes to keep students on-track with lower-stakes weekly quizzes, rather than waiting for the exams to realize they were unprepared.  I also wanted students to have the opportunity to correct their mistakes if they failed the first time.  But some students simply didn't put any effort into preparing for these.  Perhaps because it is a freshman level class, and the only chemistry requirement for several majors.  Or perhaps the stakes were too low, or the grading scheme too complex. Compared with previous semesters, my retention rates remained strong (>90%).  Scores on the ACS standardized final were slightly lower.  Despite the considerable effort, I did not see the effects I had hoped. But there were also bright spots:  While the benchmarks didn’t motivate every student, there were students who came to me to figure out what they missed, and to practice with me until they could work the problems correctly.  This was what I was after.   Conclusions So what do I make of this?  Using benchmark quizzes had a profound positive effect in my majors organic class, but a negligible effect in my non-majors class.  I suspect that a lot of this has to do with the psychology of the majors versus the non-majors.  For highly motivated pre-med students, it was a challenge to meet.  For the non-majors, the effect was different. I’m not ready to give up on the model yet.  I think it can still work in the non-majors classes, but I’ve got to tweak it for that group.  This fall, I’m planning to make two adjustments:  Simplify and raise the stakes: Spring 2017 Fall 2017 10 quizzes @ 10 pts.each 5 quizzes @ 20 pts. each Pass/Fail Pass/Fail 3 attempts 3 attempts 6/10 required for full homework credit No connection to homework My hope is that these changes will make it easier for intro students to see the importance of the quizzes, and give them more priority.  We’ll see how it goes. ... View more

Last fall, I implemented a set of benchmark quizzes in my organic chemistry classes.  These quizzes arose from a simple question:  “What should every student who passes my class be able to do?”  Adapting the approach of Joshua Ring, the quizzes were pass-fail, with no partial credit.  Students had more than one attempt to pass the quizzes, but they had to get the quiz completely correct in order to pass.  Further, I tethered the quizzes to students’ homework grades:  In order to receive credit for homework, students had to pass 5 of the 6 benchmarks by the end of the semester. Not everything went as planned.  My benchmarks were compressed toward the end of the semester, leading me to trim the number of quizzes and removing the restriction on the homework grade.  Nonetheless, the results were remarkable:  Rather than limping through exams with partial credit and moving on to other topics, the multi-attempt, pass-fail approach drove students to analyze their knowledge gaps and hone their understanding.  After two or three failed attempts, students often made their way to my office to figure out what they did wrong.  It clearly made them stronger. Quantitatively, I was able to compare my class to the previous two semesters, using the ACS standardized exam.  The only major pedagogical change was the introduction of the benchmarks.  Here are the exciting results: Semester Students Average Percentile Fall 2015 58 56 Spring 2016 40 45 Fall 2017 46 73 I don’t think the Benchmark quizzes can be exclusively credited with these outcomes.  Our fall classes are usually stronger, and this was an exceptional bunch.  However, they definitely contributed.  I was especially interested to see the effect on those students at the bottom of the class, who often have the most pronounced knowledge gaps.  Here is the change limited to those students who finished in the bottom quartile of the class on the ACS exams: Semester Students Average Percentile Fall 2015 15 17 Spring 2016 10 11 Fall 2017 12 23 This semester, I've moved out of the organic sequence and into Introductory Chemistry.  I'm teaching large daytime and evening sections, and using an adapted benchmark scheme for both.  More to come about this in an upcoming article. ... View more

What should every student who passes my organic class be able to do? Joshua Ring is an associate professor of organic chemistry at Lenoir-Rhyne University.  This past summer, I had the opportunity to hear Joshua give a talk at BCCE.  He posed this question, and offered an innovative solution, recently featured in C&E News.   In essence, Joshua flips his class, then uses a series of benchmark quizzes built around the learning objectives.  The quizzes are pass-fail.  He gives no partial credit.  However, students have multiple attempts to demonstrate mastery.   In his class, there are about 7 essential benchmarks that students must pass in order to pass.  There are 14 additional benchmarks – the number of these that the student is able to pass determines the final grade. The Modification Although the class structure at my school is much different than at Lenoir-Rhyne, I was intrigued.  I therefore designed a modified benchmark system for my class.  We began with six benchmarks: Structure – Draw Lewis structures with correct formal charge and electron counts; identify electronic & molecular geometries & hybridization Nomenclature – Correctly name key molecules, including alkanes/alkenes/alkynes Function – identify acidic and basic sites, nucleophiles, electrophiles S N 1, S N 2, E1, E2 - Show complete arrow-pushing mechanisms for these four fundamental processes. Reactions of Alkenes - Show products from addition reactions to alkenes. Spectroscopy - Identify major features from NMR/IR/MS The benchmarks were pass-fail, but students had more than one attempt to take them.   The class also included an online homework grade.  However, in order to “unlock” their homework grade, they had to pass five of the six benchmarks. Implementation  Here is one version of the first benchmark quiz: Students had to be essentially 100% right to pass.  The first time, only about 44% passed.  The next time it was offered, another 30% passed.  Eventually, nearly everyone in the class passed the exam. The thing I liked about this was that it stressed to students the importance of learning the fundamentals.   And it seemed to pay off as the class wore on – I saw fewer problems with identifying electronic geometry or goofy formal charges on mechanism questions.  Unfortunately, I could not fully implement the benchmark scheme.  Because of the coverage of our text, too many of my benchmarks landed later in the semester, and, pressed for time, I was forced to cut the benchmark quizzes short, and also to remove the tether to the homework grade.  Despite the fizzle at the end, I was pleased with this first attempt.  I feel that my students are better prepared heading into organic 2 than they have been in the past.  The pass-fail approach drove students to practice the material until they got it down.  Outlook I am not teaching organic next semester, and so won’t have an immediate opportunity to improve on the approach with that course.  However, I plan to institute benchmarks for introductory chemistry.  I am planning on 12-14 benchmarks (essentially one per chapter), closely tied to the chapter learning objectives.  Students will take benchmarks at the beginning of lab each week.  I’m excited to see how it goes! ... View more

[Originally published on September 1, 2015] Cathy Welder at Dartmouth just sent me a link with a brief review: I thought the Flipped Chemistry crowd might want to know about this book. I'm halfway through a hard copy and LOVE it. Great info on many types of active learning. The book, entitled, Reaching Students: What Research Says About Effective Instruction in Undergraduate Science and Engineering, is available for purchase as a hard copy, but also as a free PDF through the National Academy Press website: http://www.nap.edu/catalog.php?record_id=18687 Thanks so much for the link - I'm looking forward to diving into this one also. ... View more

[originally posted 12/2/2014] It's that time of the year again where the turkey has been roasted, eaten, sandwiched, casseroled, and sworn off till next year (or next month for some); and if you are in academia you are now preparing for final exams and course/instructor assessment. It was just earlier this week that I passed out our standard institutional assessment forms to my first flipped biochemistry course and because I wanted to gauge the various blended learning techniques I employed over the semester, I added my own additional assessment. In this survey, I asked my students to rate the various tools I utilized in order to "help deepen student understanding on a variety of biochemistry related topics and to facilitate the development of critical thinking skills". These tools included: video lectures, 10 minute "muddiest point" lectures in class, Sapling Online Homework, lecture powerpoints, Facebook "Journal Club" Discussions, Facebook Course Management, exam study guides, exam study sessions, POGIL workbook activities, case studies, metabolic pathway posters, and the Moodle course management page. Students were asked to rate these tools on a scale of 1 to 10 (1 being not beneficial at all and 10 being very beneficial) and then comment on the tool they found most beneficial and least beneficial. The average ratings for these tools can be seen in the graph below. The standard deviations for these responses where quite large, ranging from 1.8 to 3.0, which to me indicated that these students varied greatly on the tools they appreciated, which should be expected since these tools target a wide range of learning styles. However, I will note that I inadvertently left out one of the most important tools - the textbook. It would have been great to see if there was a correlation between students who valued the lecture videos as little to no benefit but highly valued the textbook, and would be something I look into next year. And while the overall averages may seem disheartening at first glance (particularly the POGIL workbook and video lectures), reading the students comments have been very reassuring; so I wanted to take a moment and discuss some of the common disparities between student perceived gains/values and instructor perceived gains/values that I noticed as a result of this assessment.  In their comments students asked me to: Improve lecture videos (so that they are) strong enough to stand on their own w/o needing text" So okay, yes I admit, I took the "good enough" approach this semester because I had over 25 video lectures to make in less than 13 weeks, I do see reflection of that approach in the student evaluations and am planning to revamp a number of the lecture videos this summer and next fall to incorporate interactive features such as questions, polling, and feedback, but I do not want my lectures to ever be good enough to stand alone. I want to encourage my students to continually seek out a variety of resources and to never be quite satisfied so that they keep trying to learn more. As an instructor whose primary goal is to create life-long learners, I am actually encouraged with the student quote above because it indicates that he or she wanted to learn more. That being said, I am learning that it is important in the flipped classroom to constantly remind my students that the lecture videos (as they currently are now) would have been identical to me lecturing at a podium for an hour in a traditional lecture, but now they can pause, rewind, and fast forward as often as they wish, and access these videos throughout the entire semester; and that in traditional lectures, the classroom lecture is meant to help clarify and supplement the required text reading. And, as I said, there was a wide range of disparity in the student valuation of the videos which was reflected in their comments from "The lecture powerpoints and videos were beneficial because it helped identify the most important points from the chapter" to "More in class lecture, flipped classes are very confusing and do not allow the professor to lecture to the students the knowledge they need".  "POGIL - some concepts, actually most, were too complicated for the scope of this course" The POGIL workbook, surprisingly to me, was one of the least valued tools by the students. Based on the comments, the students were frustrated both at the level of challenge and that there was no answer key given at the end.  Again, I believe I need to be clearer about the overall purpose of utilizing POGIL activities in the classroom; however, I believe students will always be frustrated and uncomfortable when challenged and it is important to teach them (especially a class made up of 99% pre-med majors) that they will not be given an answer key on the job. As I was reading over the POGIL "How-To"  I ran across this statement :  Students are missing the experience of science as the exchange and evolution of ideas, and gender and ethnic issues are being ignored in the design of courses. Poor performers withdraw from learning, and even the best performers may disengage because they are not challenged. The results are low levels of learning and high levels of attrition... To address this situation and to help students become better learners in our courses, it is essential to recognize that education has two components, content and process, and that the process component often is not given adequate attention. Science education needs to be concerned equally with both the structure of knowledge, which is the content component, and with the development of the skills for acquiring, applying, and generating knowledge, which is the process component."  I believe to help our students become better at processing the knowledge they gain from lecture, we have to push our students hard, make them uncomfortable,  and challenge them. Therefore, when I was reading these comments such as "they weren't beneficial because there weren't clear answers, and they were pretty extreme cases, some really hard to understand", actually pleases me as an instructor to no end. But I will agree with the students in that it is important to incorporate some sort of post-activity reflection/discussion, and I will be spending time during the Holiday break figuring out how to do this given the in-class time constraints (and, as always, am open to suggestions from my colleagues). "I think the class should be more lecture based. While the flipped idea is fun, I think that for a class with this much information, we need a lecture" This is actually one of my favorite quotes. While I couldn't agree more that an undergraduate biochemistry course meant to prepare both pre-med and pre-graduate students for their post-baccalaureate careers does cover an immense amount of material, I believe this, in particular, is one of the biggest reasons this course begs to be flipped. By incorporating engaging lecture videos (yes something I need to work on), text readings, and challenging/provoking in class activities, we as flipped instructors, can encourage  our students not only to gain fundamental knowledge they need to "make the grade" but to develop the skills they need in order to apply that knowledge critically allowing them to derive new connections and new ideas in their future careers.  At BSC, our mission statement specifically says "Birmingham-Southern College prepares men and women for lives of significance. The College fosters intellectual and personal development through excellence in teaching and scholarship and by challenging students to engage their community and the greater world, to examine diverse perspectives, and to live with integrity."  If I were to simply "concentrate on the things we absolutely have to know"  as some of my students have request, I would not be living up to the expectations of the college, and even to the students themselves. While flipping the class, such as a biochemistry course, may not seem to have instant gratification for the instructor, I do believe my students will (eventually) see the benefit of this course. In the meantime, let us continue to teach, assess, reflect, and modify!! ... View more

[Originally published July 10, 2015] Doug Schirch teaches chemistry at Goshen College in Indiana. Recently, he wrote to me:   I've been following your helpful website since about the beginning of the year, when I started flipping the second half our GOB course for nursing majors. After only a couple weeks into the course I could see that this really worked better, especially for this course. That held true to the end of the semester when the students took the standardized ACS exam and responded to questions I asked them on their course evaluations. I then gave a presentation to the faculty at my college, which was well-received. I've made a video-Power Point presentation of why I made the switch, what I did, and how it worked out for me. Perhaps this will be of help to others. Doug's video (22 min) is available below: Video Link : 1829 ... View more

[originally posted fall 2014] About one year ago, I heard Gabriela Weaver present the results of a detailed study on the use of the flipped classroom in the majors chemistry course at Purdue. The course had been taught in the traditional format in fall, moving to the flipped format in spring with a different instructor, and the results overall were impressive. Her excellent seminar left me with two thoughts – first, that we should implement the flipped class at Marquette, and second, that it would be possible (and seemed best) to test the efficacy of the flipped class in a side-by-side comparison of lecture and flipped courses in our General Chemistry program. Thus began what I call the Flipped Classroom Project at Marquette. And so, in spring 2015 I will be teaching two sections of our General Chemistry 2 course, one in a traditional format (to some 200 students at 8 am) and a second in a flipped format (to around 120 students). The students will be given common exams, and while students are self-selecting into the courses as I write, all of the entering students will have taken the first-semester ACS exam, therefore benchmarking their entry point. Already there is a bit of a “buzz” around the flipped class, as I piloted this concept in our off-semester Gen Chem 2 course (around 70 students) this fall, and the two courses seem about equal in popularity in early enrollment. In upcoming posts I will share my experiences this semester, and what I’ve learned in implementing the flipped classroom concept. I would be thrilled to hear your suggestions on ways to implement this project so that the most meaningful data can be obtained, and any questions you might have. Thanks for reading! ... View more

[originally published 11/20/2015] [Originally published November 20, 2015] For the past year, we've been following the flipped classroom project at Marquette. The study used parallel classes taught by the same instructor, with students self-selecting into the traditional or flipped sections. The results of this study are now available electronically on the JCE site:   http://pubs.acs.org/doi/abs/10.1021/acs.jchemed.5b00717 Their study showed very little difference in higher-level students, but a marked decrease in the DFW rates between the two classes. This seems consistent with other studies as well as personal observation: Some students can perform well in chemistry no matter how the course is taught, but lower-level students benefit most profoundly from structured, active-learning environments.   Abstract Despite much recent interest in the flipped classroom, quantitative studies are slowly emerging, particularly in the sciences. We report a year-long parallel controlled study of the flipped classroom in a second-term general chemistry course. The flipped course was piloted in the off-semester course in Fall 2014, and the availability of the flipped section in Spring 2015 was broadly advertised prior to registration. Students self-selected into the control and flipped sections, which were taught in parallel by the same instructor; initial populations were 206 in the control section, 117 in the flipped. As a pretest, we used the ACS first-term general chemistry exam (form 2005), given as the final exam across all sections of the first-term course. Analysis of pretest scores, student percentile rankings in the first-term course, and population demographics indicated very similar populations in the two sections. The course designs required comparable student effort, and five common exams were administered, including as a final the ACS second-term general chemistry exam (form 2010). Exam items were validated using classical test theory and Rasch analysis. We find that exam performance in the two sections is statistically different only for the bottom third, as measured by pretest score or percentile rank; here improvement was seen in the flipped class across all five exams. Following this trend was a significant (56%) decrease in DFW percentage (Ds, Fs, withdrawals) in the flipped courses as compared with the control. While both courses incorporated online homework/assessments, the correlation of this indicator with exam performance was stronger in the flipped section, particularly among the bottom demographic. We reflect on the origin and implication of these trends, using data also from student evaluations. Congratulations to Michael Ryan and Scott Reid on the completion of this study. ... View more

I have been waiting to do this post so that I could look at my student evaluations in order to formulate my newest plan of action. For those of you who have been following this journey with me through my previous posts: Flippn' Biochemistry, In Defense of the Flip, and Flippn' Biochemistry Part II, I hope that you have discovered that completely flipping a course successfully can not be done in just one iteration. Instead, I have learned that it is a constant cycle of implementation, surveying and reflection, and adjustments.; and I am hoping by keeping a record of the challenges and strategies I have ran across in the last couple years, I can help encourage my colleagues to incorporate more active learning into their own classrooms while alleviating some of the stress. As I mentioned in my previous post, I implemented a number of changes the second time around including finding a TA, moving to a classroom that facilitated group work versus a fixed chair lecture hall, embedding quizzes into the lecture videos forgoing the online homework, emphasizing the learning objectives both in the videos and in class and directly correlating those objectives to the in-class activities; and redeveloping the exams such that they reflected more of the activities and less rote memorization. My main objective for this term was to overcome the student perception that a course "lecture" must be a lecture by stripping away the typical lecture environment and embracing the components of a laboratory that emphasis problem based learning. I was also fortunate this year that two colleagues (Dr. Pete Van Zandt and Dr. Melanie Styers) and I were awarded a grant from the Associated Colleges of the South to assess the impact of blended learning and flipped teaching in our student's ability to think critically. For this study we utilized the Critical Thinking Assessment Test (CAT) developed and validated by Tennessee Tech in a pre/post test format in three of our classes that to some degree utilize flipped teaching. We also employed pre/post SALG surveys to assess student perceived gains in hopes of discovering correlations between categories in which the students believed they gained and in what the CAT test measured. While we are still waiting on the results from the CAT test, both my student evaluations, the technology survey that I utilize each year and the SALG responses indicate that we are headed in the right direction. Technology Survey: In 2014 (light blue) and 2015 (dark blue) students were surveyed about halfway through the term with the following prompt: Throughout the term we have utilized a variety of tools to help deepen student understanding on a variety of biochemistry related topics and to facilitate the development of critical thinking skills. To help further develop this course, please rate the tools that we utilized on a scale of 1 to 10 (1: you found this tool to not beneficial at all or even distracting and 10: you found this tool to be very beneficial). The results of that survey are found below (note that in 2014 I goofed and forgot to ask students what they thought of the textbook... so that result was not included for 2014). As you can see, students' evaluations of each tool (except for the previous year's exams) all increased or stayed the same from 2014 (light blue) to 2015 (dark blue). Of particular note are the large gains in appreciation of the POGIL activities, the video lectures, and the lecture quizzes (which was compared to the online homework assignments through sapling last year). I believe these gains in appreciation are due in large part to stripping away the lecture environment to facilitate group work and communication while decreasing the expectation that lectures are for passive learning at the onset of the term (such as we do in the laboratory). My other goal in the course this term was to redesign the in-class exams such that they better reflect the POGIL activities which is why we see a significant decreases in appreciation for the previous year's exam, because they were dramatically different in structure with short answer versus fill in the blank and multiple choice. But still we see that, the highest rated tool for "promoting critical thinking" again this year was the course management page on Moodle... which again makes me question whether or not students truly understood the question "Which tools did you find were beneficial in facilitating critical thinking". But I was happy to see that the POGIL activities and case studies came in close. Student SALG Survey Results In addition to the comparison of technology between the class in 2014 and 2015, I also looked at how the class of 2015 perceived their changes in the various skills tested by the CAT: (pre=blue and post=orange)       As we can see in the results above, all of the students felt that they made gains in all of the various skills and based on a two sample with unequal variance T-test, we found the majority of the above gains to be significant. Again, I am anxious to see if and how these results correlate with the CAT test findings! Student Evaluations In comparing my student evaluations between 2014 and 2015, I also have found significant improvements particularly in the categories such as "The course improved my ability to think critically and reason effectively", "The course was organized in a way that enhanced my learning", and "The instructor's overall teaching effectiveness"; which I also hope is a reflection of the adjustments I made in approaching the flipped class between 2014 and 2015. Below are some of the comments and feedback provided by the students: Comments that made me smile: "Pretty well organized for how much stuff was needed. Lots of thinking by the students that was then reinforced by the teacher". (2015) vs "I think the class should be more lecture based. While the flipped idea is fun, I think that for a class with this much information, we need a lecture." (2014) "She has made students think critically every class period. She created a new spin to the science department at BSC". (2015) VS "A better focus on making sure students are learning rather than memorizing metabolic pathways" (2014) "I liked how she supplement the videos with some in-class explanations. The activities were pretty solid too; very helpful. The objectives were AWESOME". (2015) VS "POGIL activities - some concepts, actually most, were too complicated for the score of this course" (2014) "Forced us to reason through problems rather than simply memorizing facts"(2015) vs "More teaching in class" (2014) Comments that demonstrate challenges still exist: (besides the "I learn better with straight lecture" comments) "Narrow the learning objectives to better match the exams, make the exams have stuff on them that we learned in class before taking it, link the activities in class with the material more" "I also never felt prepared for test despite strenuously studying" "What was on the test always took me by surprise" So while the above comments lead me to believe that the students do realize that the flipped model is improving their ability to reason and think critically, I think they are still very unsure of themselves when it comes to the exam and believe that they should still rely on rote memorization. Now I do have to defend myself, because in re-creating the exams, I pulled questions directly (and sometimes literally) out of their in-class POGIL activities. And from the first exams where the average test grade was a D, to the third and even the cumulative final exam grades averaged around a high B; i'd have to say that the students improved DRAMATICALLY on what are very challenging exams! Changes for next year In order to continue improving this course, I have came up with three modifications to test next year: In the first week of class try to better model how the group should work together and the pattern of the activities. I think if we walk through the first two activities as a larger group step by step, it may help alleviate some of the stress and give them a rhythm to work with the rest of the term. Consistently remind them that the videos are there to help introduce or explain course content while the in-class meetings are designed to help them see how that content is applied. (And they need not rely on just the videos, they have a text book and internet resources at their disposal as well). Instead of requiring students to complete the video quizzes for credit, I will now have students write out their muddiest points for each video lecture/topic and submit them online (either through facebook or moodle) for a muddiest point lecture at the start of each class. While I consistently tell my students that I will address questions in my "muddiest point" mini-lectures every class meeting, I rarely actually get questions. Then at the end of the term, my students always ask that I do this more. So this way, by requiring the questions for credit, hopefully I can increase the engagement and help better address their needs for clarity. In terms of the exams, it is my hope that now, with the redesigned exam format from last year freely available to the students to analyze and study, some of the frustrations the students voiced will decrease. I have also implemented an objective alignment activity between the video learning objectives and the in-class learning objectives to help them see how they build and grow with each other to lead them to higher order thinking and then how those higher order critical thinking skills are what I test for in the exam. Overall, I am really pleased with how that class has progressed and am excited to see how it goes next year!! Also, stay tuned for the CAT results soon to come! ... View more

[Originally published by John Osterhout on August 17, 2015]   I polled the students in my flipped General Chemistry II class to see what they found useful for their studies. At the beginning of the Spring 2015 semester, we made a list of things they could do to help themselves learn chemistry. At the end of the semester I asked them to rate the things that we discussed. Here are some of the results. This post is a follow up to my earlier post The Flipped Classroom: To Video or Not to Video. My Flipped Class After a class, I post a handout on Blackboard that contains the reading assignments and the learning objectives for the next class. The handout also contains a brief introduction (which few student read) and a list of vocabulary words. When I can find appropriate content, I provide links to videos. The videos that I have used include ChemTours (Norton), ThinkWell videos (Cengage) and Khan Academy videos (more about the videos later). Online homework covering the material in the reading assignment is due the night before each class. I assign on average about six problems, although the number varies depending upon the material. In class, the first exercise is a quiz that has five questions about the assigned material, two questions over the previous day's material and a bonus question that is extra credit. The students have been told that the first quiz question covers the first learning objective and so on down the list. The quizzes consist of the simplest possible questions that relate to the day's learning objectives. The role of the quiz is mostly to see if the students made any effort to internalize the learning objectives. The students work in groups of four for the quizzes and I observe that this engenders some lively discussions. After the quiz, I give the students a worksheet that contains more complicated problems. The worksheet is also done in groups of four. I spend the class time walking around the classroom helping the groups as requested. I post the answers to the quiz and worksheet questions on Blackboard. These became available a few minutes after the end of class. The next homework assignment contains problems from the previous class's material and from the new material for the next class. All of the sections of General Chemistry take group exams so the students are tested over the material on the same day using the same exam. The Poll Since there were no lectures or required video lectures, the emphasis was on the learning objectives. I tried to make it clear to the students in the beginning that it was up to them to figure out how to learn. At the beginning of the class, we listed off a set of resources that the student had at their disposal. The poll concerns which of these that the students used and which were useful. The question on the poll was: What did you do to help yourself? I asked them to rate each of the resources that we had discussed. There were five possible answers for each resource: A = Did not use, score = 0, B = Not useful, score = 1, C = A little useful, score = 2, D = Moderately useful, score = 3, and E = Most useful, score = 4. The average score = (0*A+1*B+2*C+3*D+4*E)/100. Forty-one students from my two sections of General Chemistry II took the poll. Not all of the students who were registered in the two classes took the poll. The Answers Rank Resources Score Use by Students(%) 1 Worksheets & keys 3.8 100 2 Daily quizzes 3.3 100 3 Study with other students 2.8 90 4 Internet 2.8 98 5 Online homework 2.7 100 6 Hard copy textbook 2.3 88 7 Chem tours 2.0 80 8 ACS Study Guide 1.8 59 9 Kahn Academy videos 1.8 71 10 Evening tutorial sessions 1.8 63 11 Self-identified videos 1.4 56 12 Office hours 1.4 49 13 Training Center 1.2 44 14 UC Davis Wiki 1.2 49 15 Thinkwell Videos 1.1 46 16 Online textbook 1.0 49 17 Office appointments 0.9 34 Notes on the resources: “Internet” was not further defined – it was meant as a catch-all for things internet but presumably not the tools named directly, such as Khan Academy Videos. The online homework was delivered the SmartWorks system from Norton. The textbook was Chemistry: The Science in Context, 4th Edition, Gilbert, Kirss, Foster, Daves, W. W. Norton & Compan, New York/London. The online textbook was delivered though the SmartWorks login. ChemTours were short (less than 10 minutes usually) animated tutorials on chemical topics assessed through the SmartWorks system. ThinkWell videos are longer, lecture-like videos from Cengage. Khan Academy videos were either assigned or found by the students. I gave two tutorial sessions per week for one hour from 5 pm to 6 pm on Monday and Wednesday and had four scheduled office hours 1-2 pm (before the chem labs) on Monday through Thursday. Office appointments were tutoring outside my regular office hours scheduled in advance by the students. The UC Davis ChemWiki was listed in the beginning as a resource. The Tutoring Center is run by Angelo State University, and usually has a chemistry tutor present. The Fate of Videos In my previous post, I asked “Do the students need to have knowledge spoken to them in order to learn?” During the Spring semester the students could use four different types of videos: ThinkWell, ChemTours, Khan Academy and other, self-identified videos. The videos were far down in the rankings: ChemTours (7th), Khan academy (9th), Self-identified (11th), and ThinkWell (15th). There seems to be a correlation here with length: the ChemTours were the shortest, the Khan Academy generally longer and the ThinkWell videos were forty-five minutes plus, covered big chunks of content, and were made to replace classroom lectures, which they strongly resembled. In talking with colleagues, I find that they echo this result: students won't willingly watch long videos outside class. In the beginning of my flipping experiments, my students complained bitterly that I didn't lecture. When provided with lecture-like videos, they won't use them.   The Most Useful – The Top Six Worksheets and Keys. I posted the keys to the daily worksheets immediately after the class ended. The keys provided examples of five or six worked out problems per day for the students. Many students made notebooks of the keys, some simply corrected their own worksheets. Daily quizzes. The keys for these were posted along with those for the worksheets. These also provided sources of problems to study. Study with other students. I meant this to mean “out of class” study with other students but many may have construed this as “group work in class” as well. Next time I'll make the choices more explicit. Internet. Where would we be without the internet? I know from talking with the students that they used the internet to search for solutions for their online homework, for explanations of the learning objectives, and for clarification of the textbook. Online homework. Talking to the students, you would think that the online homework was the work of the devil. Then it winds up in the top five—go figure. I use the homework to trap the students into engaging the material before they have the quizzes and worksheets in class. If nothing else, they must do some kind of preparation in order to be able to do the homework problems. The hard-copy textbook. Most of the students bought a hard-copy book. They all had access to the ebook through their online homework system. I was pleased to see that the textbook scored as highly as it did. As time stumbles on, I feel that the students are becoming less capable of or at least less willing to use books as learning tool.   The Least Useful – The Bottom Five The tutoring center. I was surprised that the utilization of the tutoring center (44%) was as high as it was, but a utilization of 44% had the effect of lowering the score, since "did not use" = 0 score. The UC Davis Chem Wiki. The students had a hard time navigating the wiki. They couldn't find the appropriate information by searching the wiki and when they did, the answers were too involved or the nomenclature was too different from the textbook to be useful. ThinkWell videos. As a professor, I think these are great. The students, not so much. The knock on these videos is that they are too long (45 minutes plus). Why forgo lectures in a flipped classroom if you just have to sit through them outside class? Online textbook. About four students brought laptops to class and used the online textbook in class. The rest brought or shared a hard copy book. There were a few students who tried to use their cell phones to access their textbooks, which proved difficult since the screen is so small and since I outlaw cell phone use in class. The big loser, office appointments. Two students used office appointments regularly because my regular office hours conflicted with their classes. The others would mainly drift in before the exams for a quick tune-up. Note on office hours in general: only one male came to my office hours during the entire semester. This has been my experience over a number of years and seems to the be the experience of others as well. The ratio of women to men in my evening tutorials was about 6:1.   Moving Forward I can correlate these data with the grades and with scores on the American Chemical Society final exams. I am sorting through the data now. I can tell you this: there is not a simple correlation between the number of resources used and the final grade. I will look at what the A and B students found useful and compare that to the student will lower grades. I'll let you know how it turns out. ... View more

[Originally published fall 2015]   Last (academic) year, I wrote about our controlled comparison of lecture and flipped courses in our non-majors General Chemistry program, which I called the Flipped Classroom Project at Marquette. Our goal was to test the flipped classroom in a large enrollment first year chemistry course. Our approach pushed lecture content outside of the class using short (13 min on avg.) videos, and one 75 min face-to-face discussion was held each week, led by the instructor and TA. The results show that performance on 5 common exams was statistically similar (p < 0.05) in the two courses across every grade demographic save one – the bottom group of students, as measured either by pretest or by percentile ranking in the prior course. Following this trend, a significant reduction in the DFW (Ds, Fs, withdrawals) % was observed in the flipped course, as compared to the lecture-based course and historical data in the course. Where do we go from here? Given our interest in applying the flipped approach to large enrollment, non-majors chemistry courses, we are interested in scaling up the size of the sections. In our pilot, we kept the flipped discussion enrollment at 30/section, to match the discussion size of the control. This spring, we are piloting a scale-up, using a new approach. As shown in the figure below, our laboratory floor features three adjoining labs with space for 24 students each. We plan to offer a 50 minute flipped discussion immediately prior to the lab, to be held in the same room. A teaching assistant in each room will lead discussion, with the professor floating between the 3 rooms. By holding two of these each week, we envision that 144 students can be accommodated with two discussion meetings. Moreover, we anticipate that the timing of discussion and lab affords a greater opportunity for coordination of activities. 20 Views Comments: 0 Permalink ... View more

[originally published September 2014] Marlyn Newhouse teaches Fundamentals of Chemistry and Physical Science at Union University in Jackson, TN. I admire Union for their academic rigor, the strength of their undergraduate research program, and their clarity of purpose. In February 2008, Union was hit by a devastating tornado that injured 86 people and caused major damage to 19 campus buildings. Amazingly, there were no fatalities. Recently, in a discussion of flipped- and active-learning styles, Marlyn shared this story with me: The most stark of the changes in the course has been the use of technology. I was very old-fashioned about the necessity of the teacher being physically in front of the student, proclaiming the essentials of the course. That is until the tornado of 2008. One of my students was one of the ones trapped under the rubble. When he was recuperating at home, a web cam was installed in the lecture room. This allowed the student to view the MWF noon lectures in their home in Memphis. They could follow my movements from the lecture desk to the Periodic Table on the side wall as I moved around the room. (Exerpt: Newhouse, Marlyn “Adapting Elementary Teaching Methods to College Students”, Journal of the Union Faculty Forum, Fall 2010, vol. 30 pp 39-41.) These days, Marlyn is flipping her class using the World of Chemistry series, available from Annenburg Media. Coming from an elementary school teaching background, she has always used videos in classes. Some students are visual learners. The surprise is that some students are not. A worksheet almost always accompanies the assignment to watch a video. At the next class day the questions on the worksheet are discussed and emphasis made on specific points. This frees up class time to do demos, show how to do homework problems on Sapling or explain concepts in further detail. Strategies for Intro and Gen Chem In addition to the flipped Fundamentals course (CHE105), Dr. Randy Johnston at Union has developed a new version. It is a hybrid course using the materials from the flipped course in a purely online component but also containing "face to face" lab component. It was originally designed to accommodate the adult students in the Bachelor of Science in Organizational Leadership (BSOL) program. The lecture portion is online. Most of the videos used are available from Union's Emma Summer Library media service , "Videos on demand". The lab portion is accomplished in two Saturdays with 4 or more experiments each. Since the course begins in October it has the added benefit to serve as a “parachute course” for a few traditional students who struggle with General Chemistry. Instead of dropping 4 credit hours or staying in a course in which they are struggling, students can add the Fundamentals course in the second half of the semester and get the support they need to be successful. This option is limited because the original purpose was for adult students. Finally, staying on the theme of helping students be successful, Union has developed a one-hour math review related to General Chemistry ( CHE 111). In this model, students with low math ACT scores are advised to enroll in the 8 week (1 credit hour) course, which starts in August. The course is designed to build problem solving skills and strengthen algebra skills. The course is being taught by one of the instructors who regularly teaches General Chemistry. A second 1 credit hour course can be taken the second half of the semester for students to receive more practice at working chemistry problems. In order to make these new courses more effective, the General Chemistry course is being taught using atom first approach. This allows a student enrolled in the first 8 week course to complete it before they are required to do too many calculations in the General Chemistry course. Both of these courses are being offered on a trial basis for the first time this semester. 
 ... View more

[Originally Posted Fall 2014] Over the years that my children were making their way through elementary school, I became the go to science volunteer. Times were tough for the teachers as they were adapting to new science achievement testing. Many of my elementary school teacher friends were feeling the pain of not only having to beef up their own science content knowledge, but to also learn to teach science using active, inquiry methods. The days of opening the book and defining science vocabulary words were over! At about the time my youngest son was attending kindergarten (2007), I began to think - “What do we do at my own institution to prepare elementary teachers to teach science?” I learned that our elementary teachers were herded through all the regular non-majors, large lecture courses and were required to take a total of 12 hours of science. They then had a “science methods” course their senior year.  Those large lecture courses were very much using the telling is teaching model – I know because I taught the non-science majors chemistry sequence for 6 years.  As conventional wisdom will attest, those who raise a concern end up being tapped to solve the problem! I ended up co-leading an interdisciplinary group of science faculty to develop a new four semester sequence of inquiry based science courses for pre-service elementary teachers.  I had been teaching chemistry in the lecture format for over ten years. However, the process of learning how people learn science changed my own personal teaching philosophy. Telling people what I took years to figure out was not an acceptable way to facilitate the construction of solid science principles for future teachers. By working outside my comfort level with other science faculty, I had authentic learning experiences in physics and geology using constructivist, inquiry methods. As a result of this experience - I knew I couldn’t go back to my “old” tell them and drill them style of teaching. High teaching evaluations and litany of teaching awards aside – I knew I wasn’t really advancing learning – I was just facilitating fact gathering. What does this have to do with the flipped classroom? Fast-forward four years – the course transformation program was launched on our campus. Chemistry was selected as a “gate keeper course to be transformed” and I was asked by our chair to help transform our large enrollment general chemistry course. The goal was to increase learning outcomes and reduce the Drop/Fail rates. I just had to share what I had learned about constructivist theory, active learning, process learning and all the other evidence based teaching practices I had picked up during my sojourn into the course development of the pre-service elementary science curriculum.  I convinced my chemistry colleagues that student centered active learning was the way to go to help uncover and disrupt misconceptions such that we could lay a solid foundation in chemical principles for all our STEM majors. The problem is that when you move to an active learning model – you have to give up some class time to let the students make mistakes and mess about with the activity to find their way to developing their own concept map. Our solution was to take some of the direct teach material and skill building training out of the face to face time of class and chunk and package it into learning modules delivered by a home grown homework service called Quest. Since we started this project, the university has changed learning management systems, and we are now delivering out outside of class content via Canvas. Hence we were able to free valuable class time for more engaging, thought provoking, guided inquiry type activities. Today I’m known on campus as the woman who flipped her class. When we started down this road, I didn’t even know what the word flipped meant! Rather, we were just trying to use widely available technology to enhance our student’s experience. I have sense learned a lot about the subtleties of the different types of technology-enhanced teaching. I would refer to our class more as a blended learning environment than a flipped class. Regardless of what you call what we do, our focus is on creating a supported student centered learning environment - both inside and outside the traditional face to face class room. ... View more

To be the most effective in teaching chemistry to the present generation, an educator is best served by providing the resources learners need in the manner that is most familiar to them.  For the chemistry course that I teach, Allied Health General, Organic, and Biochemistry, I am quite convinced that the flipped format serves the students best.  Today’s students have been raised in and surrounded by technology that permeates EVERY aspect of their lives. This, I believe is where educators will most easily connect with students. Furthermore, I know this can be done without sacrificing rigor or lowering expectations.   Having said this, there is so much educational technology out there now from a variety of sources and in a range of quality.  These sources have not, and will not, standardize their language (terminology) and can be contradictory. To simply gather the existing technology and provide those links in a central location would likely be counterproductive to the students’ learning without actually integrating and standardizing the content.   Over the last 15 years I have invested considerable time developing and creating effective, engaging, and often-fun content for the Allied Health Chemistry Course (General, Organic, and Biochemistry).    As a teacher I believe instructors should be responsible for providing learning solutions to meet the needs of all types of learners so that they can succeed in the class.  I believe the role of a teacher is one of facilitator; we facilitate learners in building their own understanding of the subject.   Learning must be done for one’s self, and “educators" facilitate that learning by giving them the best set of tools to do so.    There are many "digital resources," but as one who has spent hundreds of hours searching what is out there, there is a huge gap in consistent and integrated quality material. However, not all learners use the same tools.  While a textbook and the primarily monologue style of lecture where the instructor "covers material" during class works fine for some learners, the data that I will discuss in this blog shows an overwhelming student preference for the video lecture format and the flipped classroom.  The use of quality technology in education will bring the entire population of learners an increased level of achievement.    This generation of students is not served well using the same methods as was done in the past.  In my course, I provide a completely integrated video and text-based presentation of the content that is assigned as homework.  The in-class work includes worksheets, and graded online.  The flipped format class is flexible, student-centered, instructor-powered and adaptive. Students have multiple resources to engage with based on their own learning preference (reading text, watching video, working problems, tutorials, etc.).  The full suite of content and pedagogy enables students from being the product of teaching to the owners of their own learning.  By having the lectures available online, content is accessible whenever convenient for learners and can be started, stopped and repeated as necessary.  Students have varying attention spans.  Some learners are fine with 1-2 hour lectures, other do much better taking it in in small portions.  Our product allows the learner to choose their own learning times and durations!   Times are changing.   The best teaching tool is the one that provides learners with what they need in order to achieve their educational goals and accomplish the learning outcomes the instructor sets for the course!  It is very important for us to acknowledge that what was in the past defined as being literate is changing.  Being literate and perhaps specifically learning now includes the ability to find and process information in various formats beyond just the written word.  We are in the age of the Internet, video, and other forms of digital information and to be literate in today's society includes more than just the ability to read, but also to find and process information from multiple modalities.  The reason that I am a huge believer in the flipped class, in large part, come from the data below.   In an anonymous survey of students from classes that I taught using video lectures, the students were asked, "Which of the following choices was the better resource for you in this class?".  Here are the results: When students were asked to respond to the statement: "The video lectures made this course less stressful. Circle one:" the response was as follows: Students also perceived the lectures as advantages in their exam achievement by responding to the statement, "The video lectures helped me to perform better on the exams. Circle one:" as shown below:     I asked my most recent class, Spring 2014, the following question in an anonymous survey:   In traditional classes, the students listen to live lectures in class and do the assigned problems at home.  Our CHEM 108 course was presented in the "flipped" format; you had the lectures available to be watched at home and you did the assigned problem sets in class.   Which course format do you think you prefer for CHEM 108,  CIRCLE ONE: I prefer the flipped format as we did in CHEM 108 (watch video lectures at home and do the assigned problem sets in class)  I would have preferred the traditional class style (have live lectures in class and do the  assigned problem sets at home)  No preference, both are equal for me.  The student response data for this question is shown below.   I will conclude with a cautionary note.  We all know how the use of technology, for example PowerPoint, can be done well or can be done poorly.  The same is true for the flipped class.   One of the most important components of the flipped class is the video lectures.  I strongly advise against voice-behind-PowerPoint slides, these are very low on the visual engagement scale.  If you plan on making video by recording your in-class lectures, I suggest making several videos of the same lecture and then using video production software to cut and paste the raw video into an engaging, high quality production.  An example of a typical video lecture that I use can be viewed from the following link.here.  You can take a look at my entire curriculum at: http://www.saddleback.edu/faculty/jzoval   If you have any questions or comments about this blog, it is best to email me directly at: jzoval@saddleback.edu   Cheers, Jim Zoval ... View more

[Originally Posted August 2014] Last week, I had the opportunity to attend a one-day symposium put on by the Yale Center for Scientific Teaching.  The event was fantastic, and essentially the entire morning was dedicated to flipped and active-learning classrooms.  I especially enjoyed listening to Jim Rolf, a calculus teacher at Yale who has adopted the flipped classroom.  I appreciated his stories about putting content online in video format, because his experiences so closely mirrored my own.  He also offered some ideas and frameworks which I found very useful.  Here are a few tidbits I brought away from his talk, and a few meandering reflections:  Jim cited a 2012 DOE study which showed no difference in outcomes in purely online versus face-to-face teaching.  However, the study found that students did perform better when the course was presented in a hybrid format.  I’m not sure if it was original, but Jim described his pedagogy using an I.C.E. framework: Inform - prior to class (video) Confirm - linked quizzes related to the material.  I know from my own classes that this is a critical component; it apparently holds true even at Yale. Extend - During class, offer just-in-time-teaching, peer-instruction, etc., to build on the ideas. In my classes, I have migrated essentially all of my previous lecture content to video, including big ideas and detailed sample problems.  This means that in class, I can be almost completely focused on small-group problem solving.  By contrast, Jim limits his pre-class videos to the big story - he talked about developing a narrative tension in his videos - framing the “why” of the idea, and the high-level concept.  (He hopes to add a library of solved problems in the future.)  I’ve been thinking about the pros and cons of the two approaches, and don’t have a strong opinion yet.  Do any of our readers have reflections on this?  I’d love to hear from you in the comments.  This past year, Rolf did a study on two sections - one using the pre-class videos, and one not.  While he saw slight improvement in exam scores in the class with preliminary videos, he actually saw a slight drop in reading, problem-solving, and peer-teaching activities that took place after class.  This seems like a negative, but I suspect it means that students are more comfortable with the material when they leave the classroom, and therefore perhaps don’t need these activities to the extent they would otherwise.   There were several other very enjoyable sessions from the meeting, but perhaps the very best one came at the end of the day - a trip all by myself to Modern Apizza, a venerated brick-oven pizza place a mile or so from Yale’s main campus.  Delicious. ... View more