An emotion recognition test I designed using Pygame

Here's a stream of consciousness review. I basically started at the top an jotted down things as I saw them. Don't be discouraged by the number of remarks. If the code works well enough, then fix up the comments and add some doc strings or other documentation and call it a day. Make sure and have a backup before making any changes. Here we go:

Add a module level docstring describing what the code does and any information that is needed to use the code. For example, someone would need to change the base directory, provide practice and actual images in the proper subdirectories. The image files need to have the emotion encoded in the filename. Etc.

import pygame
import os
import random
import time
from enum import Enum
import csv
import sys

Comments that just parrot the code just add noise. Useful comments provide information that isn't apparent from reading the code. It's rather obvious what the line participantId == input(...) does; the comment #get participant ID number adds nothing and should be removed. Most of the comments in the code are similarly useless.

This should go with the other top level code below.

#get participant ID number
participantId = input("Please enter the participant ID number: n")

Rather than using comments to explain what the colors are, use better variable names, like buttonNormalColor and buttonHoverColor`.

PEP 8 and suggests using camelcase variable names: button_normal_color. Things that are meant to be constants, like button color or a list of all possible emotions, are generally written in all caps, like BUTTON_NORMAL_COLOR or EMOTION_TYPES. There are other python styles. It's your project; pick a style, but be consistent.

Most Python programmers seem to dislike comments on the same line after the code

#defining button colours
buttonColour1 = (20, 40, 170)  #button colour when cursor over button
buttonColour2 = (19, 47, 230)  #button colour when cursor not over button

#ms for displaying images
displayTime = 1000    # display time for images in ms

Avoid global variables like answer.

#generate empty list to store answers
answers = []    #this list stores all answers
numberofQuestions = 4    #number of non-practice questions
pnumberofQuestions = 4    #number of practice questions

This is the first worthwhile comment

#all possible answers - modify if you have more/less emotions
emotionTypes = ["Happy", "Sad", "Neutral", "Afraid", "Angry"] 

#display screen
display_width = 1200
display_height = 800

We can see that it is a dict of rects. A useful comment would be that there should be a button for each emotion type, that the top of all the rects should be below 700 so they don't interfere with the image display.

Why are the emotions capitalized in emotionTypes, but lowercase here?

#dictionary containing rects of button positions
rectDict = {
"sad": pygame.Rect(0, 700, 200, 100),
"happy": pygame.Rect(240, 700, 200, 100),
"neutral": pygame.Rect(480, 700, 200, 100),
"angry": pygame.Rect(720, 700, 200, 100),
"afraid": pygame.Rect(960, 700, 200, 100),
}

Under DRY (Don't Repeat Yourself) principle, it might be better to extract the emotion types from this dict like you do for the rectList: (emotionTypes = list(rectDict.keys()). Or just use rectDict.keys() or .values() when needed.

Could put this information in a sequence of tuples, so you don't have to type in pygame.Rect so many times. Then code could build the dict.

Given a list of button names the program could calculate where the buttons go. rectDict could be build from just the button names.

rectList = list(rectDict.values())

# Define class to keep track of states
class States(Enum):
    VIEWING = 1
    ANSWERING = 2

The name of the function is misleading. It doesn't ask a question, it displays an image. Instead of passing in a list of images and an index, just pass in the image to display, it simplifies the function signature.

#display width minus half the image width is a long way to say it is centered in the display.

Use sequence unpacking to make code clearer. image_width, image_height = image.get_size()' then xCoord = (display_width - image_width) / 2`. (Note inconsistent naming style)

For a function, a doc string is better than a comment. The audience of a doc string is a user of the function. In most REPLs or IDEs they can query the doc string to see how to use the function. Comments are generally to help someone to understand the code to better debug/modify/enhance/... it.

#this function displays an image to the screen
def askQuestion(images, imageNumber):
    """displays an image centered on the display."""
    
    xCoord = (display_width/2) - (images[imageNumber].get_size()[0]/2) #display width minus half the image width
    yCoord = (display_height/2) - (images[imageNumber].get_size()[1]/2) #display height minus half the image height
    screen.blit(images[imageNumber], ((xCoord, yCoord))) #blit image to screen

It only displays one button, so displayButton().

The calls to displayButtons() below hard code the button name and coords/width. But that data is already in rectDict above; pass in the name and rect from the dict.

I suspect it would be possible to premake a surface for each button for each color. Then simply blit the correct surface to the screen.

#define a function to display answer buttons
def displayButtons(buttonText, x, y, width, height):
    mouse = pygame.mouse.get_pos()
    buttonText_rect = pygame.Rect(x, y, width, height)
    if buttonText_rect.collidepoint(mouse):   #if mouse is hovering over button
        pygame.draw.rect(screen, buttonColour1, (buttonText_rect)) #display button colour 1
    else:
        pygame.draw.rect(screen, buttonColour2, (buttonText_rect)) #otherwise, display button colour 2

    text = pygame.font.SysFont('Arial', 22)     #add text to buttons
    textSurf, textRect = text_objects(buttonText, text)
    textRect.center = ((x+(width/2)), (y+(height/2)))  #center text within buttons
    screen.blit(textSurf, textRect)

I don't see a value in this next function. I find it clearer to call font.render() and surface.get_rect().

Generally, functions do something, so their names are often a verb: display(), sort(), etc. Variables are often things so they tend to have nouns for names: screen, button, etc.

#define functions to display text
def text_objects(text, font):
    textSurface = font.render(text, True, (0, 0, 0))
    return textSurface, textSurface.get_rect()

def displayText(someText, xpos, ypos): #use this function to display text
    text = pygame.font.SysFont('Arial', 22)
    textSurf, textRect = text_objects(someText, text)
    textRect.center = xpos, ypos
    screen.blit(textSurf, textRect)

Kinda odd having this module-level code here. Suggest moving it down with the rest, or putting it into a function.

#initialising pygame
pygame.init()

#create the screen
screen = pygame.display.set_mode((display_width, display_height))

# Caption
pygame.display.set_caption("Emotion Recognition Test")

Loading images should be a function. It doesn't hurt anything to shuffle the practice images. Pass in the directory as an argument, and it returns a list of the (imagename, image) tuples (or use a namedtuple or dataclass). If there are many images and memory is an issue, return a list of the image paths and load them when needed.

Recommend using pathlib.

Rather than hard coding the paths to the image file directory, define a base BASE_DIR = pathlib.Path("C:/Python 3.8/Projects/Emotion Recognition Test)". And make other directories relative to it: `image_dir = BASE_DIR / "All images". If the project is moved around, or someone downloads it from github, it only needs to be changed in one spot.

Use random.sample(os.listdir(), numberOfQuestions) and only load those image files. No need to shuffle, because they would already be in random order.

Should the code make sure there are a minimum number images for each emotion type?

Consider pathlib.Path().glob() instead of os.listdir()

# Load images and randomise their order
imageList = []
questionOrder = []
practiceImages = []
os.chdir(r"C:Python 3.8ProjectsEmotion Recognition TestAll images") #change cwd to folder containing non-practice images
for image in os.listdir("C:Python 3.8ProjectsEmotion Recognition TestAll images"):
    imageList.append(pygame.image.load(image).convert_alpha()) #this creates a list of surface values
    questionOrder.append(image) #this contains the image names in order

combined = list(zip(imageList, questionOrder)) #this combines image names and associated surface values
random.shuffle(combined) #randomises the combined list
imageList[:], questionOrder[:] = zip(*combined) #separates the combined list, image name and surface values are still associated


os.chdir(r"C:Python 3.8ProjectsEmotion Recognition TestPractice images") #change cwd to folder containing practice images
for image in os.listdir(r"C:Python 3.8ProjectsEmotion Recognition TestPractice images"): #for every image in the practice images folder
    practiceImages.append(pygame.image.load(image).convert_alpha()) #append to practiceImages and load in the game
random.shuffle(practiceImages) #randomise the order of practice images

Something like show_instructions() might be a better name.

Processing the event queue in multiple places seems like it is asking for trouble.

def instructions(instructionsText):      #define a function to display an instruction screen
    running = True
    while running:
        screen.fill((255, 255, 255))
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
                running = False
            elif event.type == pygame.QUIT:
                running = False
                pygame.quit()
                sys.exit()
             
        displayText(instructionsText, (display_width/2), (display_height/2)) #displays text to centre of screen
        pygame.display.update()

Perhaps init_display().

Loop over rectDict.items() and call displayButton() with the button name and Rect (the key and value in the dict) for name, rect in rectDict.items():n displayButton(name, rect)

def mainDisplay(): #this function displays fixation circle + buttons
    pygame.draw.circle(screen, (0,0,0), (int(display_width/2), int(display_height/2)), 10, 2)
    displayButtons("sad", 0, 700, 200, 100)
    displayButtons("happy", 240, 700, 200, 100)
    displayButtons("neutral", 480, 700, 200, 100)
    displayButtons("angry", 720, 700, 200, 100)
    displayButtons("afraid", 960, 700, 200, 100)

Instead of counting images and comparing it to numberofQuestions or pnumberofQuestions just loop over the sequence of images passed in. for image in images: gameState = States.VIEWING running = True while running: ...

Maybe run_experiment() for a name.

Add an argument for the instructions and state States.INSTRUCTING. Then the event loop can be taken out of instructions()

answers is a global variable, but without being declared global. It is better for it to be local to the function and then have the function return the list of answers.

def mainGame(images):
    dt = 0    #delta time is set to 0 to begin
    timer = displayTime #how long image is displayed for
    clock = pygame.time.Clock()
    imageNumber = 0
    gameState = States.VIEWING    #this is the game state where a participant views a question
    running = True
    while running:
        mouse = pygame.mouse.get_pos()
        screen.fill((255, 255, 255))
        mainDisplay()             #this displays the answer buttons
        dt = clock.tick_busy_loop(30)   #dt = number of milliseconds per frame
        
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                running = False
                pygame.quit()
            elif event.type == pygame.KEYDOWN:
                if event.key == pygame.K_ESCAPE:
                    running = False
                    pygame.quit()
                    sys.exit()


        if ( gameState == States.VIEWING):
            timer -= dt                     #minus dt from the timer, making timer count down
            if timer >= 0:                  #if the timer remains above 0, continue displaying image
                askQuestion(images, imageNumber)
            else:                           #when timer reaches 0, stop displaying image and move to answering state
                gameState = States.ANSWERING
                

I think you could use rect.collidedict(). Create a 1-pixel square rect at the mouse position: mouse = pygame.Rect(pygame.mouse.get_pos(), (1,1)) hit = mouse_rect.collidedict(rectDict, use_values=1) if hit: emotion, _ = hit answers.append(emotion) else: ... didn't click a button

        elif (gameState == States.ANSWERING):     #this is where participants select their answer
            timer = displayTime                   #reset the timer
            screen.fill((255, 255, 255))
            mainDisplay()                         #displays answer buttons
            for emotion, rects in rectDict.items(): #this detects whether a button has been clicked or not
                if event.type == pygame.MOUSEBUTTONDOWN:
                    if rects.collidepoint(mouse):            #if a button is clicked
                        gameState = States.VIEWING           #switch back to viewing state
                        answers.append(emotion)             #add answer to answer list
                        imageNumber += 1                    #move to the next image
                        break
                    elif not any(rect.collidepoint(mouse) for rect in rectList):   #if a button is not clicked
                        displayText("You did not click a button!", 600, 600)        #inform users they have not clicked a button
                        

These if statements constitute a "hidden interface" for this function, in that the way the function works depends on variables that aren't in the function's argument list. It makes it harder to test and debug.

        if practice == True:   #if in practice mood, stop when number of practice questions is reached
            if len(answers) == pnumberofQuestions:
                break
        elif practice == False: #if in main game loop, stop when number of non-practice questions is reached
            if len(answers) == numberofQuestions:
                break

        pygame.display.update()

The rest of the code drives the experiment and saves the results. It would be cleaner to put it in a main() function.

instructions("First instruction screen") #inform participants about the game

#Game Loop
practice = True

mainGame(practiceImages)   #this is the practice loop
answers = []                 #reset the answer list - since we don't track the practice trials

practice = False

instructions("Second instruction screen") #inform participants that practice trials are now over

mainGame(imageList) #non-practice round begins

instructions("Thank you for taking part") #final screen - thank participants for taking part

Here, image is a poor name. It misleads the reader into thinking it is an image, but it is actually an image name, i.e., a string. Call it imagename.

This code suggests that the emotion represented by an image is encoded in the name of the image file. The file name format should be documented.

emotions = []                                            #empty list to store the emotions in each question
for image in questionOrder[0:numberofQuestions + 1]:        #for loop to generate a list of the emotions presented in each face
    for emotion in emotionTypes:
        if emotion in image:                    #if emotion type is in the name of the image, append emotion type to emotions list
            emotions.append(emotion)

Try results = ["question", "emotion", "answer"] and results.extend(zip(questionOrder, emotions, answers))

results = list(zip(questionOrder, emotions, answers))       #combine all three lists into tuples within a single list
    
results.insert(0, ("question", "emotion", "answer"))    #top row for the CSV file

os.chdir(r"C:Python 3.8ProjectsEmotion Recognition Test") #change cwd to folder where results are stored

with open(f"{participantId}.csv", "w", newline='') as f:    #save results into a CSV file titled with participant ID no.
    writer = csv.writer(f)
    writer.writerows(results)