Technology Independent Assignments – Why You Should Mandate The Outcome, Not The Method

For the benefit of the two-and-a-half people who usually read this blog, you can switch off – it’s time to rant about work again! This is in two parts in one inconveniently sized post, because I’m a total hypocrite. Skip to Part 2 for the final point.

Part 1 – YABSS – Yet Another Bloody System Syndrome

1.1 The Powers Send Their Regards

Recently, I received a rather annoying email from the Powers That Be. According to this email, it is now compulsory for students I work with to use some software called PebblePad. They have to use this to do… well, things. I’ll get onto that.

The email ended with the following:

I really hope you will find this a easy system to use and I will be happy to answer any questions and offer help where I can.

Emphasis added. Because I have such a question:

What the fuck does Pebble Pad actually do?

No, it doesn’t do that, either.

I mean, really, what does it do? What is its purpose? Why are we using it? What can I do with it? Why is it better than its competitors? What the hell even are its competitors given we don’t know what it is?

Because at this juncture I have no idea. And it’s not for lack of people telling me it exists.

I first came across PebblePad being touted about by the aforementioned Powers That Be a few months ago. They’ve already launched it for compulsory use by undergraduates for some strange compulsory side-project (less said about how fuckawful that’s turned out to be, the better), and so I checked out the website during this lecture/meeting. Supposedly it was a good idea, but I wasn’t so sure, as our Glorious Leaders of the University didn’t seem to go into much detail about what it did.

So, thanks to 4G technology, I managed to bring up the website and understand absolutely everything about this software while in the meeting… hashtag sarcasm…

1.2 How Not to Market Your Software

Edit: I’ve checked the website since this was written, and the blurb and description of the software is a little better than it was. Not by a huge margin, but enough to make some of the sarcasm here redundant. 

I got the website up, but the website tells me nothing. I don’t quite know what it’s about, or what it’s for. What is this mysterious “PebblePad” that I’ve heard about once at a conference and twice in staff meetings?

I’ve looked at the website a few times in the last few months, and I scrubbed through it especially more thoroughly after seeing that it was now compulsory for postgraduates to use it for… for whatever its mysterious and enigmatic purpose actually is.

It’s probably not just me. I have some good evidence to suggest that I’m not a total idiot. I’ve done some stupid things in my time, but in my defence I was drunk – or in one lone, awkward case, slightly high – but here I am as a sober adult completely confused as to what is even the basic premise of this software.

It’s also definitely not just me because three junior academics, two senior academics and half-a-dozen postgraduates voiced the same concern to my knowledge. And something must definitely be up because, proportionally, those people constitute 100% of the sample size of potential end-uses who have ever mentioned it. No one quite knows what it’s for, just that they (not yet we, as a junior academic I’m safe for now) have to use it.

If you look at most websites or software you’ll find out very quickly what it is they do. For instance, Wikipedia, in nice clear text at the top, says “the free encyclopedia that anyone can edit”. Well, that’s pretty cool, it’s obvious. Microsoft Word’s main site says “Polished documents, anytime, anywhere, on all of your devices”. Straightforward, a little marketing mumbo-jumbo but generally self-explanatory;  “this software will let you type shit up and make it look pretty”.

This is not an advertisement, this is just proving a point.

Pebblepad, not so much, as here is the first piece of text that you come across on the homepage:

Pebble Learning is an award-winning company made up of the clever folk who design, build, and support our learning technology, PebblePad. What started life 10 years ago as an eportfolio system has now developed into a leading learning technology used in organisations across the globe to support learning, teaching, assessment, career advancement, and professional development.

On our journey we’ve had input and feedback from the people who really count – our customers; it’s a big part of why we now offer a proven technology that looks great, works brilliantly, delivers a positive return on investment, and provides a better user experience along the way. As you explore our website you’ll see there’s good reason why we’re so very proud of what we do.

Wait, what? This tells me nothing. “Learning, teaching, assessment, career advancement, and professional development” are not things. They are broad categories of work, but they are not specific things. My daily bus ride arguably very highly supports “learning, teaching, assessment, career advancement, and professional development”. It’s damn-well essential to my presence and ability to do my job. A regular intake of coffee certainly supports “learning, teaching, assessment, career advancement, and professional development” – indeed, Higher Education literally runs on the ability to inject caffeine at a faster rate than your body can metabolise it.

I’m sure you’re an uber cool company and all, but what does your product do? Anyone who is computer literate in the Western world should know what Word and Wikipedia are, but those products still see fit to describe what they do in a short declarative sentence at the top, and give more information when needed.

But PebblePad is a fairly obscure piece of  (apparently) educational software. If there was a definite situation for explaining as quickly as possible what your thing does, it’s this situation. Even “eportfolio” doesn’t really cut it. As a native English-speaker I can spot that this is probably better rendered as “e-portfolio” or “ePortfolio” to emphasise that it, probably, means “electronic portfolio”. But why do academics and grad students in chemistry need a portfolio? We aren’t artists. We don’t show off our work in a big book or web-page so that potential employers can gauge our work. We already have that sort of thing in terms of references, CVs, and publication records.

So what does an “eportfolio” even do?

How do we learn what it is and what it does if it doesn’t tell us?

I haven’t seen one before, no one has…

1.3 Don’t Let the Technology do the Talking if the Talking Lasts Forever

Oh, turns out that this confusion is sort of intentional:

We’re proud of PebblePad and what it can do. We’ve got a bunch of clever and passionate folk designing stuff that looks great and works brilliantly – so we’d be silly not to let our technology do the talking. It’s why you won’t find our website cluttered with wordy descriptions of our technology and why instead you’ll find lots of rich media showing PebblePad in action and the amazing stuff it can do.

No.

Just no.

Nope, nope, nope, nope.

No. And more no.

Even Diaspora’s “privacy aware, personally controlled, do-it-all, open source social network” utter-bollocks-jargon was better than this. At least Diaspora’s tagline translates to “Facebook, but not”, which is all you needed to know.

Coming out and saying you’re not going to say what you actually do is the opposite to how it works.

Suppose I’m on a weak internet connection and can’t download your video. Suppose I’m on a computer with no speakers attached so can only watch and not listen? Or, suppose I have a hearing difficulty?

So, what does the advertising video look like without sound? Well, it explains nothing. It’s silly icons, flashing pictures, it shows nothing.  I can’t figure out what you’re selling, I’ll move on rather than waste time.

Three minutes in and I am bored. And still clueless. There was something about trains. And a guy with a beard, I think. I can share it to Facebook. I can share it to Twitter. I can tell you all about the third-party non-Flash, non-YouTube video player they’re using, but if I have hearing difficulties I am fucked as a potential customer. I try it again from my laptop, which has speakers, and the voice-over makes me cringe faster than the thought of Rebecca Black releasing another album. A minute of that and I’m still clueless, but instead of bored and clueless, I’m vomiting, bored and clueless.

Explaining things simply is easy. Although I have recently discovered that explaining them simply  immediately after the question has been sprung on you in a tutorial and you have to do it in two minutes with zero preparation time is something else entirely…

1.4 Take the Plunge Anyway

So I gave up trying to figure it out and took the opportunity to actually try the software. After all, the decision has been made by the Powers That Be; we’re using this thing now. I have access to it for “free”. So I log into the University’s website and spend a little while browsing through the apps they have for us to use. It opens, and…

…and…

…meh.

I finally figure it out. In less than 30 seconds I learn more than the five minutes of video and ten minutes of website browsing tells me. Which is nice if your resident sociopathic faculty directors have already paid for the software and got the IT guys to configure it, but not so much if you’re figuring it out from scratch and wondering “do I want to use this?”.

The punchline is this: It’s basically OneNote, but online.

Write stuff, drop some pictures in, that’s it. Couple of templates for staff development reviews that look exactly the same as their PDF and Word counterparts we’ve used since forever. Fill in boxes. Add another picture. Try to make a new customised theme and see it crash your browser, you know, the usual. That is what it does. My gods, how revolutionary. I totally need this in my life [insert-sarcasm-punctuation-here].

It’s not like we don’t already have a million of those already.

1.5 If it Ain’t Broke, Break it…

Now, unlike 99.5% of the population, I actually have a use-case for OneNote. I know, weird, right? I won’t go into it, but anyway, it means I have it. I have it set up. I know how to use it. It’s pretty intuitive to use, actually, it’s just a case of whether you need it. It is downloaded onto my (various) computers and they’re all synced up nicely. It’s compatible with the two graphics tablets I use, all the operating systems I use and I’m familiar with its layout.

OneNote works for me when making notes. Word works for me when writing. WordPress works for me when blogging. Google Drive works for me when sharing. Facebook works for me for spouting utter bollocks to the world.

Which brings me from this specific rant about something I really shouldn’t bother getting angry about to the more general point, which is something I should be getting angry about, because so few of the Powers That Be understand this: that is, how to go about actually implementing software in education.

Part 2 – Instructions For Growing A Good Moustache

2.1 Use-cases and work-flow

I’ve pointed out – extensively – above what is wrong with PebblePad, currently, as a thing.

This in itself isn’t a problem. It’s my opinion. I struggled to figure it out, others might not (even if 0% of my current end-user sample have done).

I don’t like inefficient browser-based copies of standalone software I already have working. I don’t like seeing Flash hang and say it’s working for 30 seconds when my existing work-flow does the same task in, literally, milliseconds. I have a CV, I have a blog, I have a note-taking application, I have a sharing and communications protocol in place. I don’t have an “e-portfolio” but I have all the tools to make one already under my belt (which is a metaphor for “installed” plus “know how to use”).

But others might not have such a work-flow in place. Others might find PebblePad useful. And when it gets around to being overhauled by real programmers who know what they’re doing, it might fit the use-case of fledgling users better, without wasting any of their time adapting.

Badly designed software exists. We know that. Decently designed, but badly implemented, software also exists. This is a given. And software that doesn’t fit our use-case and doesn’t fit into our work-flow, but is pushed on us from above anyway exists in absolute abundance.

We need the ability to find out what works and choose appropriately, and we do that entirely because whether something is “bad” is opinion. Occasionally what is “bad” for one person will actually work very well for others. I get a kick out of using MyPaint, but I can’t stand Krita or GIMP. I find Lightwave’s interface to be the most intuitive and easy thanks to its text-based buttons and reliance on customisable short-cuts – others find the nested menus infuriating. I like WordPress, only the gods know why, others prefer BlogSpot. Etc. etc. etc…

No one mandates the software to be used to create either 2D images or 3D models – particularly with respect to a non-professional hobbyist. If you produce a decent image in the end, that’s fine.

It also doesn’t have a the concept of “new file”. You open it, and you have a functionally-infinite whitespace to draw on. Compared to Photoshop where you need to File -> New, set the size, colour profile, dpi and so on… I like MyPaint precisely because it strips out a lot of crap that is superfluous to digital painting. But I don’t use it for image manipulation or photo-resizing for the exact same reason I don’t use a coffee cup, paper clip and a piece of string to find the Higgs boson.

2.2 Software Selection (Like SAS Selection, But With More Stress Positions and Violence)

Software selection is entirely about choosing what is right for the job. The selection criteria that informs this can be split into two core issues:

1. How well does the software produce the results that you need in the time-frame that you need?
2. How easy is it for the end-user to get to grips with it, adapt to it, and integrate it into their existing work flow?

For instance, it’s almost certainly true that if you want to do heavy and serious statistical analysis with complex output, you should use R. If you want to write mathematical equations and format them right, use LaTeX. Those two pieces of kit satisfy issue no.1 perfectly. They are considered somewhat standard for serious work and for good reason.

2.3 The Barrier-to-Entry Goes Way Beyond Clicking “DOWNLOAD”

However, issue no. 2 – how easy is it for the user to get to grips with it – is a lot more important than anyone might give it credit for. This is particularly true of nerds who menacingly preach their preferred method at the expense of anything else.

This is because barrier-to-entry to use standards like R and LaTeX, mentioned above, is significant. You need to do a lot of learning to get to grips with these things. But it’s not just that R and LaTeX are powerful, and quite nerdy in their own way, it’s that if you are a new user there is a learning curve you must face. Often, you need to go back and think of very basic computing concepts to understand them. This has its uses, but it creates a big wad of effort for Generation Facebook – this isn’t a disparaging comment, it’s simply the case that computers are now so user-friendly that going back to non-WYSIWYG editing and command-line programming seems like a step backwards. Probably because it really is.

Do I want to go through all that?

The answer depends on whether I think it is worth my time. If my use-case is working out an average and a standard deviation, then no, R is not worth my time. Do I need to write 2 mathematically equations for a twice-yearly report? No, LaTeX is not worth my time. Do I want to do a multiple regression analysis on a convoluted set of 20,000 data points and output the confidence intervals of the various hypotheses? I might want to consider learning R. Do I need to write up 18 pages of mathematics lecture notes a day? I better grab LaTeX and learn to type fast.

But those are my decisions. I know my prior experience. I know my work-flow. I know my use-case.

Anyone who is an end-user knows this.

Anyone who is a quarterly-decent programmer worth her salt knows this.

Upper management that make the decisions about what software to use do not know this.

The upper management and Powers That Be who make the decisions are more likely to be swayed by the marketing jargon. They’ll see the flashy videos and gobble it up. They’ll sign the cheques to say “we’ll take it!” whether the people using the software at the end of the day like it or not – mostly because they’ll do it without even asking if they like it or not in the first place.

Because more often than not, they aren’t the people who will be using that system.

2.4 An educational perspective on choosing the right software

From an educational perspective, mandating the precise method that people must use to achieve a particular result is not helpful.

There’s an argument that it’s experience, or that it’s training – but, personally, I don’t teach software, I teach chemistry. In fact, I don’t think anyone actually “teaches software” as a primary goal. I really have to wonder what the hell they are doing if that is even remotely close to how they’d describe their primary goal.

I want students to understand the chemistry and get there faster and easier and better than everyone else. To give them thorough experience and training in particular software approaches would detract from that, and eventually we’d have no choice but to switch our teaching time away from chemistry and towards software training. Discussions of sp³ hybridisation would convert towards how to calculate a 95% confidence interval using R and spit out a box-plot. Clearing up how to calculate Pauling electronegativities from bond enthalpies would be replaced with how to write a blog in PebblePad.

Hi kids, I thought today instead of teaching you about what a cool organometallic reagent methyl iodide is, I thought I’d teach you how to make it look pretty in POV-Ray. What? Even though 3DS Max with Mental Ray will produce a better result, Lightwave is cheaper and easier to learn, Blender is free and free and open source, DS Visualiser and ChemCraft are designed to make molecules while QuteMol will make ’em look pretty… No, you will do my method and my method only! Now, mush!

There are limited, precious hours in the day, and diverting those away from what we actually need to teach is always detrimental. This is why I don’t care about software, or the method people use to get a final product.

Do I practice this? Here are my three main use-cases for mandating the outcome, not the method:

• I assess undergraduate lab reports. They get printed out, or converted to PDFs and emailed to me, and that’s all I see. That’s all I need to see, as I am assessing the scientific content and writing style. I want to see if they can communicate well. I don’t care if they use Word, or OpenOffice or Libre Office, or even TeX or HTML. Care-factor: zero. The method is not part of the assessment. It would distract from my ability to tell students they can’t write an abstract for shit if I had to give step-by-step instructions for how to install one of a dozen TeX editors. All I say is “it’s easier to use the tools the software has to insert references and figure numbers, learn how to do that in your software of choice”. I get the end result, that’s all I care about. If they don’t want to learn how to “insert caption” in MS Word, that is not my problem.
• I also run a video project, but the main aim, the goal and the ambition of it is entirely student-led. They set what they aim to produce, and I see if they deliver. If they want to make a high-production-values movie, they can; if they want to do something gonzo-style with a camera phone, they can. I don’t sit them down and say “you must use a Go-Pro camera and Final-Cut Pro”. I don’t tell them “You must film it with a Canon DSLR camera then upload the final video in .flv format to BlackBoard using its wiki feature from an iPhone connected via the VPN…” because that would be insane. And, not least, purely nonsensical. What hardware do you have access to? What software do you know how to use? If “none”, I’ll offer an opinion, otherwise go for it with what you can do.
• The only time I “mandate” a particular piece of software is that I collaborate lab marking between about half-a-dozen postgrads via Google Sheets. BUT, this isn’t “mandate” as in no other option is available – and it isn’t “mandate” as in forcing people to use a system they have zero prior experience in. If they want to download the sheet and use Excel, they are free to. If they want to print it out and do it manually, they are free to. Though if you want to do this without a spreadsheet or at least some sort of database, I do question your sanity. Providing I get a list of names with some numbers after them by [DATE and TIME] I am happy. I care about the result – that is, the work is marked fairly and I get the final grades in my office by the deadline that’s given to me from above. And similarly, those above me don’t tell me how to go about it; they just care that I get them a list of [NAMES and NUMBERS] by [DATE and TIME].

Part 3 – The Final Metaphor – How to Grow a Proper Moustache

Doin’ it right.

The US Army provides extreme detail on its grooming standards for moustaches, sideburns and haircuts. But you know what it doesn’t do? It doesn’t mandate that you must use this brand of razor or that brand of hedge trimmer to get there. Those regulations may well be very in-depth, but they are in-depth with respect to the end result.

They are technology-independent.

And technology-independent is what our educational assessments should be, particularly for anything compulsory. Compulsory means we subject every student to it. And every student will have a different established work-flow. Particularly postgraduate students that do their own research will have completely different use-cases for the software they want. One-size does not fit all in research.

A lucky few might have no method established, in which case there is a bounty of stuff for them to work with.

By all means, we should tell postgraduates and undergraduates to assess their training needs. We should tell them to keep a portfolio of work that they do. We should tell them to update their CV. We should give them strict guidelines on how the end-result should be readable for themselves and others, and how it should be written to be useful. Yet we should allow them the freedom to figure out what works best to generate that result and learn the software at their own pace – to become advanced users in their own time at their own volition. That’s where power-users come from; not from those who have a system foisted on them, but those who have a passion for producing the right end result and seek out the method to do so.

But what we shouldn’t do is tell them to use this piece of software and this piece of software only.

To do so undermines the entire point of what they’re doing. It focuses our attention away from the end result, and towards how to use the thing we give them. It focuses our attention away from why they want to be doing it, and towards troubleshooting a myriad of software problems. We would replace chemistry lectures with training workshops for the Virtual Learning Environment. We would replace lessons in good scientific writing with lessons in LaTeX.

Because if we do that, the end result suffers. If we mandate the method and not the outcome, we get a load of moustaches back that look unkempt, untidy, irregular and terrible – but hey, at least they were all cut with the same model of beard-trimmer.