We have three main options:
- The ETS ICT test.
- The Project SAILS test.
- A home grown test of our own.
Since we will need to start working on the logistics of giving the assessment almost as soon as this semester ends, we need your input soon.
What do you think? Which of these options should we choose? What do you see as the advantages or disadvantages of the different options? Let me know if want some additional information.
4 comments:
When are we looking at implementing an assessment instrument? Who are we testing? Will it just be INTD 105 students or is it going to include the students in Crosby & Kintz' classes who have had instruction from Sue Ann & Kim?
The ETS' ICT looks the most useful, since it also incorporates assessment of their technology skills, as well as library skills. However, you did mention a concern about the overemphasis of technology skills questions rather than information literacy as a whole, but I liked the idea of students being given scenario-based tasks to accomplish, rather than multiple choice questions. The ICT looks like it might glean more useful data than the Project SAILS assessment.
However...what are our cost restrictions? Do we have a budget to contract and purchase such an expensive test? What would be the costs of creating and implementing our own assessment tool based on the ICT instrument (if that was one we decided to model)? This might be ideal, since we could control the test, but might mean more work...
Kate asked some good questions that I would like to address.
First, we would like to ask students to take this assessment sometime during the middle of the spring semester.
Second, we plan to test a wide group of students from the college as a whole. This particular assessment will be aimed at the general college population. We are looking at other methods (bibliographies, the INTD 105 assessment, etc.) to test the INTD 105 students and the students in Crosby and Kintz's classes.
Third, I don't think we have the technical ability to design a test like the one from ETS. We could design a test to assess technical skills and library skills, but the simulated software, databases and scoring techniques used by the ICT exam are beyond our skills and timeline.
I like the interactivity of the ICT product. but I have trouble with a test that has questions like how to purge email (which the sample test has), since we do not teach that. If the scores on this test are going to be specifically used to assess our program, it has to only have questions that test what we are trying to teach in our program. Don't get me wrong, I do think it would be interesting to see how Geneseo students stand on technology skills, but I don't think our library instruction progam should be held responsible for things CIT should be teaching. I asked Bonnie if we can eliminate certain irrelevant questions, and the answer was no. My next question was, can we see the scores for individual questions? If so, then we can calculate a score for the library skills alone, and use that to assess our teaching.
I do think it is extremely expensive, and I suppose that means we only test a few students, rendering the results less useful. I don't think the SAILS product is very valuable.
I know that there is software that keeps track of what people are clicking on. Can we have focus groups where librarians are watching (and taking notes) while students work on a research task (and while the software is keeping track of their web whereabouts)?
If we agree on how to score this, and each librarian takes on 10 students a semester, we'll have a small but relevant sampling.
I like Sue Ann's idea of conducting closely monitored focus groups or smaller sessions with librarians. What she wrote reminds me of what we would still like to do with the website -- what Sue Ann is describing sounds more like a usability test, but monitoring how the students' are accomplishing research tasks through the website, rather than just observing HOW they are getting from point A to point B within a website.
There is software available to track and record your movements throughout a website (Camtasia is one such sfotware).
Post a Comment