Using developmental thinking to improve test construction

Developmental rubrics are not always the most appropriate way to assess something. When you are marking something that is coded as just right/wrong, a rubric might not be the way to go.

However, developmental thinking still has something helpful to say about how to write tests that are marked this way.

We found that when we started using developmental rubrics for some assessment tasks, we wondered how to incorporate the best things about them into our tests as well. Using rubrics to mark tests is often not the best way to go about it – but could they still be improved using developmental principles? We think the answer is yes!

There are a few considerations though, at the outset:

  1. Asssessing single-level difficulty content knowledge
    • Perhaps you only want to know how much simple knowledge a person has, and not what they can do with it. For example, maybe you taught students simple geography and want to see how many capital cities they can remember. It could be that the most appropriate way to assess this is with a content knowledge test with X number of questions, and you give feedback saying they got X questions right out of Y total. Nothing wrong with that.
  2. Assessing multi-level difficulty content knowledge
    • Perhaps instead though, you realise that the content knowledge you want to test is not all as easy to remember. Say maybe some capital cities are in major well-known countries and some are in far flung nations. So surely they’re not as easy to remember? Do you want to give feedback on that? i.e. tell students they got 80% of the well-known cities correct but only 40% of the far-flung ones? Well, there is a benefit to that because you’re giving students more specific feedback and therefore more information they can use to improve. If they just had gotten a 60% overall, then that wouldn’t have given them as much information for future study. Once you’ve divided it into well-known and far-flung, then the student would know they had to do more study of far-flung cities – that being the kind of knowledge they were weakest at. So the more finely grained the divisions in the test, the more finely grained the feedback you can give students, and then the more information they have to regulate their own learning.
  3. Assessing both skills and content knowledge
    • this is the kind of test that we’re mostly referring to here. Most tests don’t just assess knowledge, but what you can do with that knowledge. So, as above, would it be useful to have the skills used separated out so that you could give feedback on student achievement in different skills? We think yes.

Here is a video explaining how developmental thinking can improve test construction:

The video referenced an excel file that can automatically generate a class set of PDFs that produce disaggregated test results in graphs, which is here:

That spreadsheet is copyright Adele Hudson.