At the Drop of a Hat
Thursday 12 August 2010
Math, Moodle and Costing Choice
As is usually the case when an great event like this is brought to my attention I've sadly already agreed to be somewhere else. The seminar I would very much have liked to have joined is being presented by the University of Birmingham's Chris Sangwin: Automatic e-Assessment of Mathematical Word Problems (STACK). Chris has been doing some great work with computer aided assessment (CAA) systems for mathematics. The result of this work is STACK - System for Teaching and Assessment using a Computer Algebra Kernel.
And what's great about STACK is that it fully integrates into Moodle. Click here for details.
How to make marking algebra homework a nightmare for teacher
Imagine you've completed the first question of your math homework and the answer is 3x+4. Part of the reason your math teacher wants you to write down your answer as 3x+4 (i.e. in that order) is that if all the class has written their answer with the term in x first then you don't have to confirm the math, you can just visually check that the answers are laid out correctly. But say a student has given their answer as 4+3x? It's still mathematically correct (albeit against the usual convention of writing linear expressions in the form something times x plus a constant) but, because we aren't really attempting to check the math - just the way the answer is laid out (lazy, I know) - then it's all to easy to accidentally mark an answer as incorrect when it is, in fact, right.
My example of 3x+4 is a fairly trivial linear expression. What about a quadratic? If the answer is x2+4x+4 then my student could have written that... or (x+2)(x+2), or (x+2)2... you get the idea. All are mathematically/algebraically equivalent but having to check the math for every question you're marking can become a bit of a chore.
Which is why, in exams, questions are usually very specific about how you should answer a question. For example "factor/factorize completely" or "write your answer in its simplest terms" or words along those lines.
I always get asked what the right way is of answering such-and-such a question and, to be honest, outside of an exam, I'm not that precious about it. If I'm giving one-to-one tuition in their home my usual answer to a student will be "Well, I could have come to your house by car, jet pack, helicopter. I could have come straight here, gone via Scotland (I live a long way from Scotland)." In fact I could have gone round and round the world three times. As long as I get there, and as far as the student is concerned, it really doesn't matter.
So what decides which route I take? What decides which mathematical method we should use? The answer is cost. Cost (in the accounting/economics/project management sense of the word, i.e. in the "cost benefit analysis" sense) is all about taking the most effective choice within known constraints. I tell my students to chose a method (i.e. make a choice) that they feel most comfortable with. A choice that, when they are actually doing the math, they feel they are less likely to make a mistake. That, for them, will be the most cost effective route through the problem. It's a very common project management approach (effectively the one we're using for the international Moodle rollout I'm currently working on) and it can work wonders in math teaching.
A worked example
In the best traditions of math textbooks, let's explore what I mean about choices having a cost with a worked example. Chris included this poser in his abstract:
In a railway journey of 90km an increase of 5 kilometers per hour in the velocity decreases the time taken by 15 minutes. What is the velocity?
I thought it might be interesting to work through this problem but to also explain, in terms of cost, why I personally would use particular math techniques.
I know this is probably going to sound geeky but the elegance of algebra really appeals to me. Expressing a problem in algebraic terms is going to be, for me, the most cost effective. Once I've translated the problem into algebra I'm going to continue with the algebraic approach and solve any equations I can construct to find the unknowns.
What about the less algebraically inclined? Well, if you got as far as coming up with equations you could plot them and see where the lines or curves (or whatever - I don't know yet) cross. That, of course, depends if you're more graphically motivated.
If you happen to know that speed/velocity (there's more on the difference further on in this post) is given by dividing the distance travelled by the time taken to travel that distance then there is yet another good way... but, unless you're lucky, it could take you some time. That is the way of trial and improvement (basically trail and error): take a guess and work in small steps from there until you get at least a good approximation to (or at best the exact value of) the answer.
For those that are interested here are the steps I took:
1) The units aren't SI units. Convert distances into metres and time into seconds.
90km is equivalent to 90000m
15 minutes is 900 seconds
5 kilometers per hour is 5000m in 3600 seconds. That simplifies to 25 eighteenths metres per second.
Reason: well, basically that's the professional physicist in me. If you want to know what happens when people who really should know better don't convert their measurements to standard units then read this.
2) Use speed equals distance divided by time relationship...
We rote learned this one when I was a budding young physicist in science lessons at school - so one could argue that either you know this or you don't.
Reason: two aspects here to consider. One is that experience tells me because this is a "speed/distance/time" question so use the "speed equals distance..." relationship. Second is that I also rote learned two unknowns takes two equations.
This is knowledge in my head. How it got there was, as I say, and rightly or wrongly, rote learning.
3) ... to construct two equations
Firstly... assign some letters: 'v' for velocity and 't' for time. My two equations are
t=90000/v
and
(v+25/18)(t-900)=90000
Reason: That takes an understanding of rearranging equations, for example v=d/t can be written as vt=d. To do that we just need to understand that '=' means everything on one side is the same as everything on the other and, because of that, everything you do to one side you've got to do to the other. Emphasis of this last point is criminally (I think) missing from a lot of basic math teaching in the UK.
3) Substitute the first equation into the second
Remember that, although I have two equations with two unknowns, what ever the values of v and t in the first equation they have exactly the same value in the second equation. They are, in the trade, simultaneous.
Now I have just one equation... with no t in it, just v:
(v+25/18)(90000/v-900)=90000
Reason: I'm in my algebraic comfort zone.
4) Multiply out the brackets
How long have you got? There are lots of different ways of multiplying out brackets. There's FOIL (first, outer, inner, last), the "box" method - where v+25/18 is the length of one side and 90000/v-900 is the length of the other and the area of the box is the area of four smaller boxes added together (if you're familiar with it you'll know what I mean). My method of choice is "happy smiley face"... I guess because *cough* I'm a happy, smiley kind of guy...
I end up with
90000-1250+125000/v-900v=90000
which, with a bit of algebraic tomfoolery, I can manipulate that into
900v2+1250v-12500=0
Why write it in that form remembering that I can write it however I like?
Reason: now it's a quadratic I can solve it using the quadratic formula.
5) Put the numbers through the quadratic formula
In fact I end up with two numbers, one positive and one negative. Can I can ignore the negative velocity? (Your author puts on his best James Bond 'Q' immitation) Now pay attention 007...
A negative speed doesn't make much sense but a negative velocity might. Speed is a scalar quantity but velocity is a vector. A negative velocity means you're going the opposite direction. However, in the context of this question the negative solution can quite happily be ignored.
So...
If you substitute a=900, b=1250 and c=-125000 into the quadratic formula you end up with a positive answer of 11 and 1/9 metres per second. That's the equivalent of 100/9 times 3600 metres every hour, or... to cut a long story short... 40km per hour.
6) Test your answer
Don't just give the answer. If you have time (and more so if you're not feeling very confident of your result) justify your answer.
How long will it take to travel the 90km at 40km per hour? The answer is 8100 seconds.
How long at 45km per hour? That works out to be 7200 seconds.
That is a difference of 900 seconds, or 15 minutes. That gives us good justification that our answer is correct*
Conclusions
How did you get on with Chris' problem? Which method works best for you? How do you measure the cost of a given mathematical process? I'd be interested to know what you think.
Is measuring the cost of choices before you in a project your method of managing a project? I'd be interested to hear about your experiences.
*Note that I'm not going to claim I was 100% correct. I might have made a mistake in my justification that wrongly proves an incorrect result. Again, it's the scientist in me: by showing you my working I'm hoping that any mistakes will be picked up by you - my peer reviewer, as it were. Don't hesitate to let me know if you think I'm wrong.
Friday 6 August 2010
Target Your Audience
He was outlining how they were monitoring web traffic so that sales could target specific clients with specific information. For example, imagine I'm trying to sell you Moodle services. I want to target you with information on how I can help your school. So I create a web page with links to examples on using Moodle for teaching different subject areas. You click on the links to see what I'm offering. As a supplier, once I know which subject areas seem to interest you, I could target my sales pitch accordingly.
For instance, I've included an image of a 3D representation of a molecule that's taken your interest and you've clicked on it. Are you interested in multimedia? We can create another page containing more information on including multimedia content in Moodle. In subsequent interactions we can find out if it is actually science you are interested in by monitoring what pages you have visited, what links you have clicked on, even how long you have spent on a particular page.
Of course, why not just ask the question "are you interested in teaching science with Moodle?" Well, there are a number of problems with this approach. For instance, I might inadvertantly offend cultural sensibilities - especially when dealing with a worldwide audience. I might come across as being too blunt in my approach (it's easy for your motives to be misunderstood at the best of times but seemingly more so when the interaction is online). More importantly from a sales point of view, however, is that our potential client who's being asked that question might not actually be aware that they are interested in teaching science with Moodle. In other words: if you were that potential client then you might well be interested in the tools to teach science with Moodle... you just didn't realise it yet.
For example, I'm a theology instructor (which I'm not - I was just trying to think of a discipline that wasn't science but it probably isn't a good example) teaching about evolution: I dim the lights slightly and use the Moodle Jmol plugin to display a large, slowly revolving 3D representation of a water molecule on the interactive whiteboard. Then I start to teach...
"What is truely remarkable about that molecule on the board is that the angle between the hydrogen atoms is exactly 104.5 degrees. That means that when water molecules join together they form a solid (ice) that is less dense than the liquid. This means that if water freezes then it doesn't sink: it floats. If ice sank then everything living under it would die. Do you think the water molecule looks like that by chance [I point back to the large, slowly revolving water molecule on the board] or by design?"
See what I mean?
If you and I were trying to sell Moodle services then you can see how this approach to designing and managing our online presence is going to be far more successful than just putting up a few web pages telling our site visitors how great Moodle is and then crossing our fingers.
To discuss the care with which this particular client studies, and follows up, the interactions that take place between their online presence and their (potential and current) clients was extremely informative (in all honesty, it's probably that level of attention to detail and presence of thought that might help explain why they lead the world in what they do).
Are you a teacher running an online course in Moodle? If so check out the Reports link in your course administration block.
In exactly the same way my marketing colleagues would want to drill down to see how clients interact with the online collateral and target their sales pitch accordingly, you too can see how your pupils are interacting with your teaching materials and provide just the right kind of support and guidance.
Stick with me a few moments more because another fascinating point came out of this discussion: I was thinking about @Moodleman Julian Ridden mentioning in the excellent Moodle Mayhem podcasts (here) about tackling a teaching colleague on having lots of animated GIF images in her Moodle course (if you haven't heard those podcasts then you really must: there are two and they're excellent).
In fact what the guys in marketing have also been investigating is how their clients interact with static images and animations on web pages. For instance, they've found that if you have a static image of a book, clients are far less likely to click on that than if the icon was animated - e.g. the pages of the book start to flick open by themselves. They've also been looking at delaying the animation, e.g. for a second and a half or two seconds after the page has loaded, before the book flicks open. Then users are much more likely to click on the book.
Can we likewise take our students on a particular journey through our courses by the clever use of "visual cues"? Would we want to?
Do we, by using visual or audio cues, inadvertantly take our learners to specific resources without realising it? This is just one of the questions that has come out of the work of Dr Richard Clark (learn more here).
Have you tried investigating how your students interact with your courses using the Reports function in Moodle? Have you tried using visual cues to guide your students through a Moodle course? If you have then I'm interested to hear your experiences.
Thursday 5 August 2010
Moodle Chemistry
@lasic retweeted a question from @betchaboy:
You would be forgiven for thinking it looking at the relatively small edit box you use to specify your multichoice answers but (as the developer responsible for Moodle's quiz engine @tim_hunt later confirmed) you can actually type HTML in there (the only reason why the HTML Editor isn't displayed is that a pretty long page web page would be rendered even longer - the multi-choice scroll of death becoming even more deadly). The image below demonstrates what I mean about the small edit box...
My solution, rather than using HTML, would be to use LaTeX.
Those of us of a certain age (*cough*) predate these new-fangled "what you see is what you get/WYSIWYG" text editor whatsits. We had to use a typesetting langauge. The language of choice - certainly in the science and math community - was (and in some instances, thinking of University of Birmingham here in the UK, still is) LaTeX.
LaTeX is pronounced "lah-tek" or "lay-tek", depending on who you speak to (more details here).
Speak to your server/Moodle administrator about getting LaTeX installed on the server and the Moodle TeX filter enabled (see http://docs.moodle.org/en/TeX_filter for details). Make sure you have TeX properly configured. Check that your LaTeX isn't being rendered using the Mimetex filter - for no better reason than Mimetex only recognises a subset of TeX and the output can look a little jagged. See this discussion thead on in the Mathematics Tools forum on Moodle.org for some tests you can try: http://moodle.org/mod/forum/discuss.php?d=120418. For the more technical readers, you'll also witness me having a strange configuration error on one of my servers.
Here is a multi-choice question on the Haber Process:
Well, the correct answer...
...is actually written in the answer edit box like so:
$$\textrm{N}_{2(g)} + 3\textrm{H}_{2(g)} \leftrightharpoons 2\textrm{NH}_{3(g)}$$
That may seem horribly complicated (and I'm challenging you with a technically difficult example) but, rest assured, once you learn some of the basic typesetting commands it can become extremely powerful way of arranging/typesetting your text. Check out http://docs.moodle.org/en/Using_TeX_Notation for more details.
For more details on LaTeX, how to set it up in your Moodle, and how to typeset complex notation using LaTeX then definitely visit the Mathematics Tools forum on Moodle.org (http://moodle.org/mod/forum/view.php?id=752).
Are you a LaTeX user? Do you find it easy or difficult to typeset mathematical and/or chemical notation using LaTeX? I'm interested to hear from you.
PS I don't want to be accused of plugging one of my books but I go into detail about configuring Moodle to support complex notation in Moodle 1.9 Math - including how to integrate the excellent DragMath drag-and-drop LaTeX editor into Moodle.
Tuesday 3 August 2010
Smile... Changing Moodle's Default Emoticons
A few years back I had to give some pastoral support to one of my students who was in tears because of something that was said by one of her friends on her Facebook wall. My immediate reaction was to tell my student to give her friend a call on her mobile. "I can't do that - not after what she's said" was the response. The situation quickly resolved itself when we could bring the two girls face to face. I can't even remember what the Facebook conversation was about but what caused all the upset was just two words: "yeah right".
Try saying "yeah right" to yourself positively - with a sing-song, shrill intonation.
Now try saying "yeah right" negatively - with a deep, sarcastic intonation.
Very different.
Do the same but now also imagine looking at a happy, smiling face (say a close friend) happily saying "yeah right" and a dark, angry face (perhaps someone you don't particularly get on with) for the slow, sarcastic, negative "yeah right".
The contrast between the two is even more pronounced: it's very, very different
One way of overcoming this problem is to use 'smilies', or 'emoticons', in your text :)
The only issue I have... and I guess this is personal preference... is that Moodle's default smiley set is not very expressive. I have a set of animated (but not too busy) emoticons I use in my Moodles and I'll describe now how you can switch over to using them - or, of course, a set of your own.
- Locate the folder containing the smilies Moodle is currently using. This is usually the /pix/s folder. Note that custom themes can also contain custom smilies so you might need to also look for /pix/s in the theme folder for your custom theme. Make a backup copy of the 's' folder (e.g. 's_back') and replace the emoticons with your set.
- You might have guessed that the emoticon file names are used to identify what the icon represents so the next step is to rename the files accordingly (you can specify the names in Moodle's source code but this way is easier).
You might need to resize the Insert Smiley dialog. Here's how:
- Look in /lib/editor/htmlarea/ for the file dialog.js and make a backup of this file.
- Search for the line beginning case "dlg_ins_smile": x = and alter the x and y numbers accordingly (check out the copy of dialog.js in the sample files and see how this differs from your original dialog.js
Do you want to include extra smilies in the Insert Smiley dialog?
- ...if you do then it is best to copy your new emoticons to /pix/s. We'll definitely need to make changes to Moodle's core code (don't worry: it's not complicated) and that will pick up files from /pix/s - as well as in custom themes - if you switch to using one of Moodle's standard themes, e.g. standardwhite.
- Look in /lib/editor/htmlarea/popups for the file dlg_ins_smile.php and make a backup copy of this file.
- Search down dlg_ins_smile.php for the line $emoticons = array ( 'smiley' => ':-)',
At the bottom of the $emoticons array add references to the extra emoticons you want to include (again, see the example file for details).
- Look in /lang/*your language*/ - for example /lang/en_utf8 - for the file pix.php. This contains the emoticon "tooltips" - the helpful text that appears above an emoticon when you hover the mouse over it. Make a backup copy of this file.
If you already have the HTML editor open in your browser then refresh the page, press the insert smiley button and your new emoticons will be displayed. Try inserting them into some text:
Reconfigure the HTML Editor
The editor can recognise ASCII emoticons and replace them with GIF images. We can, if we wish, include our newly specified smilies.
- Ensure you are logged in as a Moodle administrator. From the Site Administration block click on Appearance HTML Editor.
- Add your new emoticons to the list. The desired ASCII smiley is specified on the left and the name of the GIF image (not including ".GIF") is specified on the right
- Press the Save Changes button at the bottom of the page and you're done!
More ideas
In December - in the run up to the festive season - I switch to using a "Santa" smiley set - bascially the emoticons you can see in the previous screen captures but all wearing Santa hats.
Further ThoughtsHave you ever had experience of your students reading a comment online and taking it the wrong way? How was the misunderstanding resolved?
I'm Listening... Creating a Rapport
I found the issues raised by Dr Clark absolutely fascinating. Two items caught my eye. Take a look at the list of 11 tips for increasing the effectiveness of online learning. Item 10, "human voice is always better than mechanical narration" is one. But the statement I've been reflecting on is:
adjusting instruction for different learning styles does not increase learning (accommodation = fail)
Tomaz Lasic has just today asked this question on Twitter:
If there were a 'scientifically proven' best way to teach but clashed with your style/values, would you change? Why (not)?
To which I answered that it basically depends on my audience. I'll change my manner of teaching depending on the rapport I have with the class. What do I mean by that? I'll try and explain...
I was involved in some sales training when I was younger and I found it interesting that salespeople (at least those in the UK) tend to have a standard approach when meeting new clients. Here are the standard steps:
- Full-on stare your new client in the face and say "Pleased to meet you I'm XXXX [complete as appropriate]"
- Quickly follow up wide-eyed stare with (usually over-) firm handshake
- Thrust business card over to client
- Immediately proceed with demonstration of product (either virtual - read "PowerPoint" - or actual)
It's not particularly slick and it rarely works. So how should you do it?
As any successful salesman will tell you, the very first step is engage your potential client in an interaction, usually a conversation. If it's a conversation you've started then you can then look for the predicates in the sentences they use - specifically looking out for visual, auditory or kinethesthetic (emotive/emotional) representations. Don't worry: this isn't Neural Linguistic Programming, but the deep underlying principle is roughly the same (based on work by Chomsky, Jung et al.). Let's see how this might work in practise.
Imagine the following interaction:
Pupil: Sir, I was listening to what you were saying earlier but I didn't really understand what you meant about denominators. What do you have to do with them?
Teacher: Have a look at the board. Can you see how I've multiplied them together?
Pupil: Erm... not really. I couldn't quite hear at the time so I didn't know what you meant.
Teacher: Look at the numbers on the bottom. Can you see that you need to multiply them?
Pupil: No. It doesn't matter. I'll have another go. Thanks anyway.
The pupil feels let down by the teacher because they don't feel like they've been understood. The teacher is left feeling frustrated that they didn't get their point across. What went wrong in this interaction?
If we were a salesperson then we'd be thinking about the language the pupil and teacher used. Have you noticed that they are, at this moment, using completely different representations - the pupil is hearing what the teacher is saying, the teacher is looking towards what's represented on the board. Personal chemistry between the pupil and teacher aside, there is very little "linguistic rapport" in the language the teacher and pupil are using to interact with each other. If the teacher had used auditory words rather than visual ones would the teacher have been better able to put their point across?
It seems to work for me but I'm not sure how you'd test such a pragmatic approach scientifically.
The best salespeople will sell you something you didn't know you wanted at a time you don't need it.
To my mind that very much applies to being a teacher.
So... have you been paying attention at the back? Here's a test:
What kind of language have I used in this short piece? Is it visual, auditory or emotive? What would be the best way of attempting to sell me your idea? As ever: I'm interested to know what you think. I'll post a followup on this later in the day.
Monday 2 August 2010
In Pursuit of Happiness: Open Learning in Moodle
If you've read Dave's post then Hypothesis 7 I found particularly interesting: "Print controls our learning". The assertion is that the "historical roots and technologies of print have a profound and controlling influence on how we see education".
What intruiged me about this assertion (putting my Chomsky hat on for a moment) is the visual nature of the language used to express it. To the central point: is print a barrier to learning? As ever, that depends. As anyone reading this or Dave's blog using screen access software (e.g. Windows Narrator, JAWS, Supernova, etc) will tell you: you don't need to be able to see print to gain access to information. You need access to the technology. The printed word, after all, is a technology. Yes, there are ways of representing language in touch (Braille, blister paper) but - certainly as far as the UK is concerned - Braille readers represent a small fraction of the visually impaired population. Those visually impaired students I work with learn by being spoken to and from speaking to each other. The only communication medium available to many is language, thoughts, truth and logic represented in speech (often US English speech to boot). To the use of lanugage to express the sentiment: I'm wondering if it would be better using the word "experience" rather than "see".
Can there be such a thing as an "open" learning platform? How vital is an instructor to the learning process? This had me thinking about...
The Three Steps to Open Learning and Eternal Happiness
Step 1: Yay! My college runs a dedicated happiness course...
I'll click on that now.
Step 2: The secret to eternal happiness is only a click away...
Nirvana here I come...
Step 3: Oh hum...
How do you resolve the contention between providing a truly open learning platform and wanting to manage users in your Moodle? What do you think? Have you managed to create a fully open learning platform? I'm always interested to hear your thoughts.
Friday 30 July 2010
Can't See the Wood for the Trees
And we searched and searched the page and couldn't find mention of WordPress anywhere.
We both looked at each other convinced that this must be a WordPress blog. When we looked back at the screen there it was: a large WordPress icon in a box about half an inch high by an inch and a half long - containing "WordPress" in big letters.
How did we manage to miss that?
Have you ever lost a piece of paperwork on your desk only to have a colleague come along and point to it right there in front of you? I don't mind admitting that it happened to me the other day. How did the paperwork come to be invisible (at least to me) until someone pointed their finger at it?
I've had a long interest in cognitive psychology - and especially the design of user interfaces both screen-based and otherwise.
If you haven't listened to the interviews with @moodleman Julian Ridden on Moodle Mayhem then a) you should because Julian knows his stuff and they are fascinating listening and b) Julian talks in detail about the design of the Quantum Riverview E-Learning Portal. Umm... should that be a), b) and c)?
What interested me about the Moodle Mayhem discussion is that Julian mentioned he'd put all the blocks on the right and the main content on the left because English readers have a left-to-right reading order (as above). That set me thinking: would this affect the way a user would interact with the site?
I'm a big fan of the research carried out by Donald Norman - author of one of my favourite books The Design of Everyday Things. Basically Norman asserts, based on his research, that users seek "cues" (typically "visual cues" but they could be audio cues) based on the task they want to carry out.
Then there is the notion of actually reading from left to right...
Those who know me will be aware that I worked for a time with the blind and the visually impaired. Braille readers are actually few and far between but the remarkable thing is that in just the same way we read ahead as we read a sentence (less confident readers don't manage this as well which is why children's reading sounds... so... stilted) so Braille readers will actually read with two hands - the leading hand reading ahead. I guess the same is true with music sight readers (hats off to my mother-in-law who can play the piano like this). Check out the interesting work done by Thomas Wolf in the 1970s here. Wolf suggested there are (I guess obviously) lots of cognitive processes going on - perhaps the most powerful being pattern recognition, interestingly. Though again, with music, this is left-to-right reading order.
Have you tried speed reading? If not then the idea is that you open the page of a book, place your finger in the very middle of the page under the first line and focus your sight in the middle of the page as you drag your finger down it (your eyes following your finger down the page). The idea is (and I'm not sure how efficacious this actually is) that your peripheral vision will recognise word patterns - or the sense of the patterns - and you'll be able to get a feel for the content without having to read it properly. That doesn't involve scanning from the left to the right.
But there is a deeper level at which Julian's very interesting point about left-to-right reading order works extremely well - and that is the relationship between the visual and a description of the visual, read from left to right. I'll try and explain what I mean...
I don't know how many readers are familiar with UK television "celebrities" Ant and Dec?
Ant and Dec are both very popular here in the UK but, with the best will in the world, neither Ant nor Dec individually have too recognisable TV personalities (I'm sure they're wonderful company in real life). To overcome that tricky problem all their publicity photos have them stood, from left to right, in the order Ant... then Dec.
If you don't believe me then do a quick Google image search for "Ant and Dec" to see what I mean.
Are you a Moodle theme designer? What metrics do you use to judge usability? Have you carried out any usability studies? For example, there's a great thread on Moodle.org here.