Why the SBAC Speaking Assessment is Off the Mark

Think of sometime in your life when you heard a great speaker, someone who really impressed you.  Now imagine that someone asked you right after the speech what you thought.  Imagine the conversation.

That was amazing.  That speaker is one of the best I have ever heard.

Why do you say that?  What was so good about him?

His grammar!  He had amazing grammar.

Probably not, right?  What was it then?  Was it carefully selected supporting details?  Was it the use and citation of resources?  Was it smooth, well-designed transitions?  Was it impressive figurative language or precise use of technical language?  I ask these questions because they are from the SBAC speaking scoring guide.  Don’t misunderstand: I am not saying these are unimportant.  But are any of them the reason you declared the speaker to be one of the best you have ever heard?

            If you have read my earlier posts, you realize that all of the items being measured in the questions above are part of what I call “building a speech.”  They are things we do before we ever open our mouths.  As we write the message, we worry about grammar, supporting details, figurative or technical language, and transitions.  In fact, these are all good measures for writing assignments.  But we are supposed to be measuring speaking.  Should you be able to be called proficient as a speaker if you write a good essay?  Again, what was it that made you say the speaker was one of the best?

            I’m guessing that the largest part of what impressed you was what the speaker did as he was speaking, what I have referred to as “performing a speech” in earlier posts.  The performing talent is very different than the building talent.  That is why we have actors to say the lines the screenwriters compose.  We do not give an actress an Academy Award because the lines she delivers were so well written but rather because they were so well said.  Do you ever watch TED talks?  All of speakers have great and interesting ideas—that’s why they were chosen to give a TED talk.  You won’t find better built speeches anywhere.  Few of them, however, would cause you to say, “Wow, that was a great speaker!”  Of course the lines and the message are important.  But in the world of oral communication, the speaker who impresses does so by performance more than by words.  To measure speaking, then, what percent should be based on how well the speech was built and what percent on well it was delivered?

            An extreme example: a teacher assigns an essay; the student is told to read it aloud; the grade is based 75% on how well the essay the written and 25% on eye contact, enunciation, good pace, “fluent delivery”, and gestures as it was read.  We could probably all agree that that would be a poor assessment of speaking skill.  Three examples will prove that.  Speaker A writes a great essay (75 points).  As he reads it, he wiggles and fidgets which is fine because there is no score for that.  Though monotonous, very fast, and without gestures or eye contact, he pronounces well and doesn’t stumble over words.  The teacher gives him 12 out of 25 points for a combined score of 87.  He is declared to be a “proficient speaker.”  Speaker B also has a fine essay (75 points).  She is constantly flipping hair out of her eyes and rocking back and forth which is not a problem according to the scoring guide.  She is monotonous, has no gestures and no eye contact but pronounces well, doesn’t stumble over words, and speaks slowly.  The teacher gives her 16 out of 25 for a combined 91, an “advanced speaker.”  Speaker C did not have a superior essay.  There were a few grammatical errors, good but not exceptional word choice, some disorganization, and not enough details so he received 54 out of 75 points.  As he reads, the class is amazed.  He is poised and commands the stage, but that gets no points.  He is passionate, varies speeds for effect, has beautiful gestures, looks at each audience member, and speaks clearly as the words flow out of his mouth.  Easily 25 out of 25 for a combined score of 79.  He is dubbed a “pre-proficient speaker.” 

In my opinion and I bet yours, all three speakers were scored incorrectly.  If you were in the class listening, you would have said, “Wow, Speaker C is really good!” and if we are scoring speaking skill, his score should reflect that.  I believe that when we score speakers, we should give more weight to the performance and less to the writing because that is how we instinctively react to speakers.  I won’t argue with you if you prefer a 50/50 split.  But I strongly disagree with giving three-fourths of the weight to the writing as the teacher in my example did. And as the SBAC scoring guide does.  Yes, the SBAC uses the same scoring guide as the inept teacher.  (You can see it here: http://www.ode.state.or.us/wma/teachlearn/testing/scoring/guides/2011-12/spkingscorguides1112.pdf.)  

The PARCC speaking assessment, which will be locally scored, will be out in the fall.  Local scoring brings up several other issues: 1) Your building does not have a consistent way to evaluate speaking; 2) No two teachers in your building use the same rubric or scoring guide; 3) No teachers in your building have ever been trained about how to evaluate (or teach) speaking.  If it is locally scored, at least you have a chance to create a better guide than the one the SBAC offers. 

If you want to prepare students to pass the SBAC speaking assessment, teach essay writing.  If you want students to be college and career ready, teach oral communication skills and don’t let a proficient rating according to the SBAC make you think you have done that.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s