The Lost Dimension of Ethics in Design Experiments

I graduated in Psychology from Nottingham University in 1992, and there is one experiment which I took part in which I still remember to this very day. The reason is simple – embarrassment.

We were discussing animal cognition in one of our classes, and we carried out a simple test as an icebreaker (experiment would be too strong a word). We placed five different coins in a number of felt bags, and and goal was to pick out one specific coin which we were shown and able to play with before dipping our hands in the bags.

The vast majority of my fellow students were able to do this without issue, but for some reason I was really having an off day. In the five attempts I was given, I could not pull out the target coin, I kept pulling out the one most similar in size and weight.

As I mentioned, this was an ice-breaker with no defined methodological procedure. This is a wordy way of saying that when I failed the test, those around me laughed at me, with one pointing out that even a chimp would have had a statistically better performance than me. At the time I remember feeling really stupid for failing.

Credit: Pixabay

This was an extremely important experience for me to have, since after graduating I joined BT as a professional psychologists in their Human Factors department. There we carried out a wide range of usability tests and trials with members of the public, working with prototype telecoms products and services to see how easy of difficult, or indeed frustrating they were to use.

As a professional and member of the British Psychological Association all the tests I designed and ran fully complied with the BPS Standards and Guidelines. These are all available to read online, and are divided into a number of sections, the main ones of interest in relation to design tests being:

BPS Code of Ethics and Conduct (2009)

BPS Practice Guidelines

Data Protection Act 1998 – Guidelines for Psychologists (2009)

Ethical Enquiries FAQ

The main Code of Ethics and Conduct provides a simple framework for understanding the professional practice of psychologists. The practice is underpinned by four key ethical values – Respect, Competence, Responsibility and Integrity – and these five core skills:

  • assessment and establishment of agreements with the client;
  • formulation of client needs and problems;
  • intervention or implementation of solutions;
  • evaluation of outcomes; and
  • communication through reporting and reflecting on outcomes.

These are shown in the figure below.

Source: The British Psychological Society, Practice Guidelines, Third Edition

As the guidelines state:

“Assessment of psychological processes and behaviour is derived from the theory and practice of both academic and applied psychology. It includes both assessing change and stability, and comparison with others. Assessment procedures used will depend heavily on the practice context and may include:

  • the development and use of psychometric tests following best practice;
  • the application of systematic observation and measurement of behaviour in a range of contexts and settings;
  • devising structured assessment strategies for individual clients, teams and organisations; and
  • the use of a range of interview processes with clients, carers, other stakeholders and other professionals.”

The list above describes the range of structured methodologies used in the practice of psychology as a science. When carrying out any type of experiment or trial, including usability testing, the general principle is that people must leave the trial in at least the same sound frame of mind as when they entered it.

If we think back to the simple test of selecting a coin, I did not finish the test in the same state of mind; I felt embarrassment. In order to ensure that this principle is always in operation in any test or trial, the psychologist will draw up a full briefing and de-briefing procedure. So for example, when asking users to play with a new interface, I always used to emphasise that if any parts were difficult to use, this was a fault of the design and the person should not leave thinking they were somehow lacking.

As industry trends shifted from the scientific multi-disciplinary Human Factors to frameworks such as Design Thinking, non-qualified professionals are now being encouraged to carry out rapid-prototype testing as well as many other trials and customer research, such as empathy mapping. In general, this is positive, but I have seen trials carried out with no regard for any formal structure or procedures which acknowledge the psychological impact of being tested upon.

Just as a historic side note, it’s interesting for me to see people nowadays talk about lean startups and agility and the need for minimal viable products. In this Channel 4 programme for which I was interviewed in 2000, I discuss the fact that technology is changing was (and is) changing so rapidly, that knowledge can soon be out of date.

There is huge pressure for those in the design space to trial as fast as possible with the minimum of checks and balances. These pressures though are no excuse not to follow ethical guidelines and in fact it was great to see IDEO recently publishing their own book on ethics in design.

In the field of personality psychology there is the concept of locus of control. This refers to a person’s belief system in relation to the degree to which they ascribe external or internal events and factors to their success in life. If someone has an internal locus of control, they see their own personal efforts as contributing to their success, but someone with an external locus of control places more emphasis on factors external to them.

These effects are always taken into account by psychologists when designing experiments, as are many other forms of cognitive bias. (The BPS guidelines, pp 11-12, list many of these if you are interested in finding out more). But to what extent are these guidelines and factors taken into account in design experiments?

This question is the reason I am writing this article. I have seen extremely unprofessional practices which have the potential to be damaging to a lesser or greater degree on those volunteers agreeing to take part in tests. No matter how informal or innocent a test may appear, there are potential risks of harm, or, at the very least, accidentally inducing feelings of embarrassment and shame.

When experiments and trials are carried out with the correct guidelines and procedures, these risks are reduced immensely. Given that many different people are moving into and doing Design Thinking from a wide variety of backgrounds, and who do not necessarily have much in the way of professional training in the design and running of consumer experiments and trials, I did feel the need to write this article to alert people to the need for vigilance in this area.

I know other people are looking and writing about design ethics, and I just wanted to add my voice and provide some references for where to look further. In addition to the British Psychological Association guidelines, Richard Gross has written one of the definitive student textbooks on psychology, Psychology: The Science of Mind and Behaviour, and this has an extensive final section on ethics, issues and debates for psychologists. Although it is written for degree-level students, it is a very readable introduction for those looking for further guidance.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s