Imagine yourself at the helm of a large freighter in the middle of the Atlantic on a dark, starless night at the turn of the 20th century. You don’t have GPS (you don’t even know what that is in 1900 but you get to sport a cool monocle) which is okay because you have a compass, maps, star charts, weather reports, last known position at sunset, and you’ve got years of sailing experience. The star charts aren’t much help at the moment as it is quite cloudy but no bother, all was right with the world and you were on course only a few hours ago when the sun dipped below the horizon. Steady as she goes, right?
At long last morning comes. The sun is rising and the clouds are burning away….but wait. Isn’t the sun about 15 degrees south of where it should be in the sky? Or have you been heading 15 degrees north of where you were supposed to be? That’s more likely, but for how long? How many hours were you off course? How far away from your destination did you drift between sunset and sunrise? If only you could have seen the stars overnight!
No time to dwell on it now. We need a plan to get back on course, and quickly.
KPREP scores seem to come like the sunrise sometimes, burning off the fog off a long dark night. If you’ll grant me that analogy then humor me a little more when I say that MAP Test scores are a little like the stars that can keep us pointed to the right spot on the horizon when we can’t see the sun. Just like fog and clouds can obscure the stars, sometimes the MAP data can be a little foggy.
When KPREP scores were released in September, district leaders realized just how far off course we Shelby County Public Schools were. MAP data had indicated that things were off but not to the degree that was uncovered in September. MAP data was good enough that course corrections were underway before school started this fall. The KPREP scores sparked a greater effort to turn the ship to a destination we had been so close to just last year.
I have heard the plans from each of our schools. I’ve heard what was working, what wasn’t, and what will be done about it. Before I talk about the tools that were used to diagnose and plan from I have to say that data is only as good as what you do with it.
To make this real in the world of business I’d like to give you an example of some of the work I’ve been doing for my employer.
For several years my employer has been giving technicians a randomized technical assessment for knowledge prior to offering them a job. We knew what we were getting when we hired a technician and we knew that they needed training but there was no structured plan to deliver that training and no tools in place to measure the effectiveness of training.
With my first class of apprentice technicians I helped develop a program of training that built on each prior module and concluded with the most complicated concepts and courses. Furthermore, I asked for the randomized assessment that was being used prior to hiring to be given again after 18 months of training. Now the results from these assessments were not used to calculate a grade or score for the technicians taking the tests but rather they were used to tell me how effective I was at teaching. Passage of a given module is based on demonstrated ability to achieve success criteria for each module.
For example, in the first chart below, technicians showed improvement in all areas from pre-hire testing to post-training testing but I wasn’t happy with the results in electrical knowledge.
Based on the data from these results I shifted my approach and spread electrical instruction out over more weeks and increased the depth in which I covered the topic.
In the second chart you’ll see that electrical showed greater gains as a result and without much loss anywhere else except in air conditioning. I didn’t anticipate that drop but I have a pretty good idea of why it happened and how to adjust for another group of apprentices that are still a few weeks away from that module.
So from the data I had a good indication of where I was, what was working, what wasn’t and I started making adjustments immediately that manifested in the test results.
Well that’s exactly what our school administrators, instructional coaches, and teachers are doing in the classroom on a far more sophisticated level than I am.
Just like our imaginary sailor, our schools are looking at data and applying the tools they have to adjust course. Over the first two weeks of November, the Board of Education sat in round table discussions to hear what kinds of methods are being applied to stay on track and get back on track where needed. Ultimately it comes down to solid and consistent data collection, acting on that data rapidly, and targeting the right approach and instruction for the right student.
MAP Test scores, collected three times a year, are primarily used as a diagnostic tool for identifying standards or components of standards where a student may be struggling. These tests are helpful in that they target individual pieces that may be lacking in a student’s understanding. These results combined with the results daily or weekly formative assessments or checks for knowledge are particularly powerful when discussed in data teams.
Data teams are groups of teachers that are familiar with the students they collect and analyze data on and they work together, daily in many cases, to tailor instruction for each student based on needs. In lean manufacturing you would call it “Just in Time” instruction. Instructional coaches are often instrumental in helping data teams understand gaps in understanding for both the student and teacher. I know that my data tells me as much about my effectiveness in instruction as it tells me about gaps in my technicians’ learning. What if I don’t know a better method to help my students? An instructional coach could help me for sure and they help teachers regardless of experience with a fresh set of eyes and new or unfamiliar approaches.
In addition to targeted and individualized instruction, there are broad approaches to affecting what is gained from instruction. If you haven’t already heard the terms “learning target” and “criteria for success” from your students yet, ask them. Learning targets are showing up in nearly every classroom in the district. In some classes students are developing their own targets, in others the targets are set for them but in all cases, they are posted or communicated so that the student understands them. Not only that, but often, students are asked to describe the criteria for success in reaching those targets.
It took me years in management, and parenting, and teaching to understand the power of getting the words to come out of their mouth. Questions are a powerful way to create ideas in people’s minds. Imagine a problem being presented to you and you can immediately identify a lack in your knowledge that would help you solve this problem. You can likely identify that gap in your knowledge and determine what you need to learn to solve the problem. You’ll also know what success looks like for you. Our students can identify those things when given the chance.
One last one I’ll touch on is that in addition to data collection, planning action, and implementing a plan there is follow-up. Administrators, instructional coaches, and teachers participate in school walkthroughs and classroom observations to look for all the elements in this process and give feedback to teachers. Yes, it adds a level of accountability but more importantly it adds a level of feedback that is needed to make sure that students have what they need to learn effectively and that teachers have what they need to instruct effectively.
After the round table discussions I have been part of and the answers I got to some of the questions I think a lot of parents have, I can say with some confidence that the ship is turning quickly to get back on course. The data teams and MAP scores are the stars and compass that will guide our students to that point on the horizon we’re aiming for. When students take KPREP tests in the spring I feel like they will find that the sun rises exactly where expected.
Our school administrators and teachers are taking the targeted actions to benefit our students in a more focused and intense way than I could have imagined. Don’t take my word for it though, ask them and ask how you can help your students at home.
As always, feel free to drop me a line if you have thought or questions at email@example.com