4.1 Tutorial: Multi-Participant Studies
This tutorial walks through a complete example of a multi-participant study that demonstrates how to:
Randomly assign participants to treatment or control groups
Match treatment participants into pairs
Have participants complete tasks and collect their responses
Calculate outcomes based on comparisons between matched participants
The study implements a simple experiment where participants complete arithmetic tasks. Control group participants receive payment based only on their own performance, while treatment group participants are matched with a partner, and their payment depends on whether they score higher than their partner.
This tutorial assumes you’re familiar with the basic concepts covered in Congame basics: An overview and Introduction: a quick tour using Conscript. We’ll focus on explaining the code itself, walking through each section to understand what it does and why.
To follow along: You can find the complete code for this study in the Multi-participant Study Recipe section of the Conscript cookbook.
4.1.1 Study Overview
Here’s what happens when someone participates in this study:
The participant is randomly assigned to either the treatment group or the control group, using balanced randomization (2 treatment, 2 control per group of 4 participants).
They see instructions explaining how payment works for their assigned group.
They complete 2 simple arithmetic tasks.
They see their score (number of correct answers).
Control group: They immediately see their payment, calculated as $1.00 base + $0.20 per correct task.
Treatment group: They wait to be matched with another treatment participant. Once matched, the system compares their scores. The participant with the higher score receives $3.40 ($1.00 + $2.40 bonus); the one with the lower score receives just $1.00. If scores are tied, one winner is chosen randomly.
Now let’s look at how each part of this study is implemented in code.
4.1.2 Preamble
#lang conscript (require conscript/form0 conscript/survey-tools racket/list racket/match) (provide simple-treatment-study)
As always we begin with #lang conscript, require the additional modules we’ll need, and provide the name of our study (which we’ll define at the very end). (To suppress errors in DrRacket while you’re writing the file, you can comment out the provide line with ; and then uncomment it when you’ve finished actually defining simple-treatment-study.)
4.1.3 Treatment Assignment
At the start of the study, we need to set up variables and assign each participant to a treatment condition. Here’s the code:
(defvar/instance treatments) (defvar is-treatment?) (defstep (assign-treatment) ; Balanced: out of every 4 participants, assign 2 to treatment, 2 to control (with-study-transaction (when (or (undefined? treatments) (null? treatments)) (set! treatments (shuffle '(#t #t #f #f)))) (set! is-treatment? (first treatments)) (set! treatments (rest treatments))) (skip))
4.1.3.1 Instance-scoped Variables
(defvar/instance treatments) creates a variable with instance scope. Unlike regular defvar variables (which store a separate value for each participant), instance-scoped variables are shared across all participants in a study instance.
Here, treatments holds a list of treatment assignments (#t for treatment, #f for control) that all participants will draw from. This is how we ensure balanced assignment: we create a list with exactly 2 #t and 2 #f values, so out of every 4 participants, 2 will be assigned to each condition.
is-treatment? is a regular (participant-scoped) variable that stores whether the current participant is in the treatment group.
4.1.3.2 The Assignment Step
The assign-treatment step does three things:
Checks if the treatments list is empty or undefined. If so, it creates a new shuffled list of assignments: '(#t #t #f #f). The shuffle function randomly reorders this list.
Takes the first assignment from the list and stores it in is-treatment? for the current participant.
Removes that assignment from the list (using rest) so the next participant gets the next assignment.
All of this happens inside with-study-transaction, which ensures that if two participants reach this step at exactly the same time, only one will access the treatments list at a time. This prevents both participants from accidentally getting the same assignment.
The step ends with (skip), which immediately moves the participant to the next step without displaying any page.
4.1.4 Instructions
After assignment, participants see instructions explaining the study. The instructions differ based on whether they’re in the treatment or control group:
(define instructions-control @md*{You are in the **control group**. You will complete 2 simple arithmetic tasks. Your payment will be: - $1 base payment - $0.20 for each task you get correct Good luck!}) (define instructions-treatment @md*{You are in the **treatment group**. You will complete 2 simple arithmetic tasks. After finishing, you will be matched with another participant in the treatment group. Your payment will depend on both your scores: - **Winner** (higher score): $1 + $2.40 = $3.40 - **Loser** (lower score): $1 - **Tie**: 50-50 chance of winning **Important:** After you complete the tasks, you may need to wait briefly while we match you with another participant. Please be patient and wait for another participant to complete their tasks so the two of you can be matched.}) (defstep (instructions) @md{# Instructions @(if is-treatment? instructions-treatment instructions-control) @button{Begin}})
This code defines two sets of instructions using md* (note the asterisk). The md* function is like md, but returns a value that can be reused within other page content, rather than as a page by itself.
In the instructions step, we use (if is-treatment? instructions-treatment instructions-control) to display the appropriate instructions based on the participant’s treatment assignment. This is a straightforward conditional: if is-treatment? is #t, show treatment instructions; otherwise show control instructions.
4.1.5 Tasks
Next, participants complete two arithmetic tasks. This section involves several pieces:
4.1.5.1 Setting Up Variables
(defvar task1-response) (defvar task2-response) (defvar score) (defvar payment) (defstep (init-tasks) (set! score 0) (set! payment 0) (skip))
We create separate variables for each task response, plus variables for score and payment that we’ll calculate later. The init-tasks step initializes score and payment to 0 and immediately skips to the next step.
4.1.5.2 Task 1: Using Forms
;; Task 1: What is 7 + 5? (define-values (task1-form task1-onsubmit) (form+submit [task1-response (ensure binding/number (required))])) (define (render-task1 rw) @md*{@rw["task1-response" @input-number{Your answer}] @|submit-button|}) (defstep (task1) @md{# Task 1 What is 7 + 5? @form[task1-form task1-onsubmit render-task1]})
This demonstrates the standard pattern for collecting input via forms, which you may have seen in Introduction: a quick tour using Conscript:
form+submit defines the structure of the form data and returns both a form object and a submission handler. Here we specify that task1-response must be a number (via binding/number) and is required.
render-task1 is a helper function that renders the form’s input widget. It takes one argument rw (a widget renderer) and returns the rendered form elements.
The task1 step displays the question and includes the form by passing the form object, submission handler, and renderer to form.
4.1.5.3 Task 2
;; Task 2: What is 15 - 6? ;; Lots of identical code from above is repeated. ;; See conscript-multi-example.rkt in the repo for an example of how this could ;; be simplified (define-values (task2-form task2-onsubmit) (form+submit [task2-response (ensure binding/number (required))])) (define (render-task2 rw) @md*{@rw["task2-response" @input-number{Your answer}] @|submit-button|}) (defstep (task2) @md{# Task 2 What is 15 - 6? @form[task2-form task2-onsubmit render-task2]})
Task 2 follows the exact same pattern as Task 1, just with a different question and variable name. As the comment notes, in a real study you might want to refactor this repeated code (see congame-example-study/conscript-multi-example.rkt for an example), but for this tutorial we keep it simple and explicit.
4.1.5.4 Calculating and Displaying the Score
(defstep (show-score) (set! score (+ (if (= task1-response 12) 1 0) (if (= task2-response 9) 1 0))) @md{# Your Score You answered **@(~a score) out of 2** tasks correctly. @button{Continue}})
After both tasks are completed, we calculate the score by checking each answer against the correct answer (12 for task 1, 9 for task 2). Each if expression returns 1 if the answer is correct and 0 if not; we add these values together to get the total score.
We then display the score to the participant using ~a to convert the number to a string for display in the Markdown text.
4.1.6 Control Group Payment
For control group participants, calculating payment is straightforward:
(defstep (control-payment) (set! payment (+ 1.00 (* score 0.20))) @md{# Your Payment Your payment is: - $1.00 base - @(~$ (* score 0.20)) for @(~a score) correct task(s) **Total: @(~$ payment)** Thank you for participating!})
We calculate payment as $1.00 plus $0.20 times their score. The ~$ function (provided by conscript/survey-tools) formats numbers as dollar amounts with two decimal places.
4.1.7 Treatment Group: Matchmaking and Results
The treatment group process is more complex because we need to match participants and compare their scores:
4.1.7.1 Variables and the Matchmaker
(defvar opponent-score) (defvar did-win?) (define matchmaker (make-matchmaker 2))
We create two more variables: opponent-score to store the matched partner’s score, and did-win? to store whether the current participant won the competition.
(make-matchmaker 2) creates a matchmaker function that groups participants into pairs (groups of 2). This function will handle the complex logic of matching participants as they arrive.
4.1.7.2 Waiting for a Match
;; Wait for a match using the matchmaking module (defstep (wait-for-match) @md{# Waiting for Match You have finished the tasks. We are now matching you with another participant... Please wait. @refresh-every[5]}) (defstep (pair-with-someone) (matchmaker wait-for-match))
The wait-for-match step shows a waiting page that refreshes every 5 seconds (using refresh-every).
The pair-with-someone step calls our matchmaker function, passing it the wait-for-match step. Here’s how this works:
When a participant reaches pair-with-someone, the matchmaker assigns them to a group.
If there’s not yet a partner available for their group, the matchmaker displays the wait-for-match page.
Each time the page refreshes (every 5 seconds), it checks again whether a partner has been found.
Once another treatment participant is matched into the same group, both participants automatically proceed to the next step.
4.1.7.3 Storing and Retrieving Group Results
(defstep (record-score-for-group) (store-my-result-in-group! 'score score) (skip)) (defstep (get-opponent-score) (define other-score (first (current-group-member-results 'score))) (cond [other-score (set! opponent-score other-score) ; Determine winner (set! did-win? (or (and (= score opponent-score) (> (random 2) 0)) (> score opponent-score))) (skip)] [else @md{# Please wait Waiting to learn your opponent's score… @refresh-every[2]}]))
These steps handle the coordination between matched participants:
record-score-for-group: Uses store-my-result-in-group! to save the current participant’s score in a way that other participants in the same group can access it. This function stores results in a shared data structure (similar to how defvar/instance creates shared variables). The step immediately skips to the next one.
get-opponent-score: Retrieves scores from all other group members using current-group-member-results. Since we’re in a group of 2, this returns a list with one element: our partner’s score.
Store it in opponent-score
Determine the winner: did-win? is #t if either (a) scores are tied and (random 2) returns 1, or (b) our score is higher than the opponent’s score
Skip to the next step
If the partner’s score isn’t available yet (because they haven’t finished their tasks), we display a waiting page that refreshes every 2 seconds.
4.1.7.4 Displaying Treatment Results
(defstep (treatment-results) (set! payment (+ 1.0 (if did-win? 2.4 0))) @md{# Match Results **Your score:** @(~a score) out of 2 **Opponent's score:** @(~a opponent-score) out of 2 **You @(if did-win? "🎉 WON!" "lost this round.")** Your payment is: - $1.00 base - @(if did-win? "$2.40 for winning" "$0.00 (you lost)") **Total: @(~$ payment)** Thank you for participating!})
Finally, we calculate payment based on whether they won ($3.40 total) or lost ($1.00 total), and display the results along with both participants’ scores.
4.1.8 Main Study Flow
The defstudy form ties all the steps together into the study’s transition graph:
(defstudy simple-treatment-study [assign-treatment --> instructions --> init-tasks --> task1 --> task2 --> show-score --> ,(lambda () (if is-treatment? 'pair-with-someone 'control-payment))] [control-payment --> control-payment] [pair-with-someone --> record-score-for-group --> get-opponent-score --> treatment-results] [treatment-results --> treatment-results])
This defines a study named simple-treatment-study with several paths:
Treatment participants go to 'pair-with-someone
Control participants go to 'control-payment
Control branch: The control-payment step transitions to itself, meaning it’s a terminal step (participants stay on this page once they reach it).
Treatment branch: Proceeds through the matchmaking and result steps (pair-with-someone → record-score-for-group → get-opponent-score → treatment-results), with treatment-results also transitioning to itself as a terminal step.
4.1.9 Key Concepts Recap
This tutorial introduced several important concepts for multi-participant studies:
Instance-scoped variables (defvar/instance): Variables shared across all participants in a study instance, useful for coordinating assignments and shared data.
Study transactions (with-study-transaction): Ensures that when multiple participants are accessing shared variables simultaneously, their operations don’t interfere with each other.
Matchmaking (make-matchmaker): Automatically groups participants and manages the waiting process until groups are filled.
Group result storage (store-my-result-in-group!, current-group-member-results): Functions that let participants in the same group share and access each other’s data.
Conditional transitions: Using lambda functions in the transition graph to send participants down different paths based on their data (like treatment assignment).
4.1.10 Next Steps
This study demonstrates the core mechanics of multi-participant research in Conscript. From here, you might want to:
Explore the more sophisticated version in congame-example-study/conscript-multi-example.rkt, which refactors the repeated task code using helper functions
Look at the Prisoner’s Dilemma tutorial for another example of participant matching and group decisions
Consult the Conscript Cookbook for more recipes and patterns
Review the reference documentation for functions like make-matchmaker, store-my-result-in-group!, and with-study-transaction to understand all their options and capabilities