# PD N-Person Iterated

Do you have questions or comments about this model? Ask them here! (You'll first need to log in.)

## WHAT IS IT?

This model is a multiplayer version of the iterated prisoner's dilemma. It is intended to explore the strategic implications that emerge when the world consists entirely of prisoner's dilemma like interactions. If you are unfamiliar with the basic concepts of the prisoner's dilemma or the iterated prisoner's dilemma, please refer to the PD BASIC and PD TWO PERSON ITERATED models found in the PRISONER'S DILEMMA suite.

## HOW IT WORKS

The PD TWO PERSON ITERATED model demonstrates an interesting concept: When interacting with someone over time in a prisoner's dilemma scenario, it is possible to tune your strategy to do well with theirs. Each possible strategy has unique strengths and weaknesses that appear through the course of the game. For instance, always defect does best of any against the random strategy, but poorly against itself. Tit-for-tat does poorly with the random strategy, but well with itself.

This makes it difficult to determine a single "best" strategy. One such approach to doing this is to create a world with multiple agents playing a variety of strategies in repeated prisoner's dilemma situations. This model does just that. The turtles with different strategies wander around randomly until they find another turtle to play with. (Note that each turtle remembers their last interaction with each other turtle. While some strategies don't make use of this information, other strategies do.)

Payoffs

When two turtles interact, they display their respective payoffs as labels.

Each turtle's payoff for each round will determined as follows:

```
| Partner's Action
Turtle's |
Action | C D
------------|-----------------
C | 3 0
------------|-----------------
D | 5 1
------------|-----------------
(C = Cooperate, D = Defect)
```

(Note: This way of determining payoff is the opposite of how it was done in the PD BASIC model. In PD BASIC, you were awarded something bad- jail time. In this model, something good is awarded- money.)

## HOW TO USE IT

### Buttons

SETUP: Setup the world to begin playing the multi-person iterated prisoner's dilemma. The number of turtles and their strategies are determined by the slider values.

GO: Have the turtles walk around the world and interact.

GO ONCE: Same as GO except the turtles only take one step.

### Sliders

N-STRATEGY: Multiple sliders exist with the prefix N- then a strategy name (e.g., n-cooperate). Each of these determines how many turtles will be created that use the STRATEGY. Strategy descriptions are found below:

### Strategies

RANDOM - randomly cooperate or defect

COOPERATE - always cooperate

DEFECT - always defect

TIT-FOR-TAT - If an opponent cooperates on this interaction cooperate on the next interaction with them. If an opponent defects on this interaction, defect on the next interaction with them. Initially cooperate.

UNFORGIVING - Cooperate until an opponent defects once, then always defect in each interaction with them.

UNKNOWN - This strategy is included to help you try your own strategies. It currently defaults to Tit-for-Tat.

### Plots

AVERAGE-PAYOFF - The average payoff of each strategy in an interaction vs. the number of iterations. This is a good indicator of how well a strategy is doing relative to the maximum possible average of 5 points per interaction.

## THINGS TO NOTICE

Set all the number of player for each strategy to be equal in distribution. For which strategy does the average-payoff seem to be highest? Do you think this strategy is always the best to use or will there be situations where other strategy will yield a higher average-payoff?

Set the number of n-cooperate to be high, n-defects to be equivalent to that of n-cooperate, and all other players to be 0. Which strategy will yield the higher average-payoff?

Set the number of n-tit-for-tat to be high, n-defects to be equivalent to that of n-tit-for-tat, and all other playerst to be 0. Which strategy will yield the higher average-payoff? What do you notice about the average-payoff for tit-for-tat players and defect players as the iterations increase? Why do you suppose this change occurs?

Set the number n-tit-for-tat to be equal to the number of n-cooperate. Set all other players to be 0. Which strategy will yield the higher average-payoff? Why do you suppose that one strategy will lead to higher or equal payoff?

## THINGS TO TRY

Observe the results of running the model with a variety of populations and population sizes. For example, can you get cooperate's average payoff to be higher than defect's? Can you get Tit-for-Tat's average payoff higher than cooperate's? What do these experiments suggest about an optimal strategy?

Currently the UNKNOWN strategy defaults to TIT-FOR-TAT. Modify the UNKOWN and UNKNOWN-HISTORY-UPDATE procedures to execute a strategy of your own creation. Test it in a variety of populations. Analyze its strengths and weaknesses. Keep trying to improve it.

Relate your observations from this model to real life events. Where might you find yourself in a similar situation? How might the knowledge obtained from the model influence your actions in such a situation? Why?

## EXTENDING THE MODEL

Relative payoff table - Create a table which displays the average payoff of each strategy when interacting with each of the other strategies.

Complex strategies using lists of lists - The strategies defined here are relatively simple, some would even say naive. Create a strategy that uses the PARTNER-HISTORY variable to store a list of history information pertaining to past interactions with each turtle.

Evolution - Create a version of this model that rewards successful strategies by allowing them to reproduce and punishes unsuccessful strategies by allowing them to die off.

Noise - Add noise that changes the action perceived from a partner with some probability, causing misperception.

Spatial Relations - Allow turtles to choose not to interact with a partner. Allow turtles to choose to stay with a partner.

Environmental resources - include an environmental (patch) resource and incorporate it into the interactions.

## NETLOGO FEATURES

Note the use of the `to-report`

keyword in the `calc-score`

procedure to report a number.

Note the use of lists and turtle ID's to keep a running history of interactions in the `partner-history`

turtle variable.

Note how agentsets that will be used repeatedly are stored when created and reused to increase speed.

## RELATED MODELS

PD Basic, PD Two Person Iterated, PD Basic Evolutionary

## HOW TO CITE

If you mention this model in a publication, we ask that you include these citations for the model itself and for the NetLogo software:

- Wilensky, U. (2002). NetLogo PD N-Person Iterated model. http://ccl.northwestern.edu/netlogo/models/PDN-PersonIterated. Center for Connected Learning and Computer-Based Modeling, Northwestern Institute on Complex Systems, Northwestern University, Evanston, IL.
- Wilensky, U. (1999). NetLogo. http://ccl.northwestern.edu/netlogo/. Center for Connected Learning and Computer-Based Modeling, Northwestern Institute on Complex Systems, Northwestern University, Evanston, IL.

## COPYRIGHT AND LICENSE

Copyright 2002 Uri Wilensky.

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

Commercial licenses are also available. To inquire about commercial licenses, please contact Uri Wilensky at uri@northwestern.edu.

This model was created as part of the projects: PARTICIPATORY SIMULATIONS: NETWORK-BASED DESIGN FOR SYSTEMS LEARNING IN CLASSROOMS and/or INTEGRATED SIMULATION AND MODELING ENVIRONMENT. The project gratefully acknowledges the support of the National Science Foundation (REPP & ROLE programs) -- grant numbers REC #9814682 and REC-0126227.

## Comments and Questions

globals [ ;;number of turtles with each strategy num-random num-cooperate num-defect num-tit-for-tat num-unforgiving num-unknown ;;number of interactions by each strategy num-random-games num-cooperate-games num-defect-games num-tit-for-tat-games num-unforgiving-games num-unknown-games ;;total score of all turtles playing each strategy random-score cooperate-score defect-score tit-for-tat-score unforgiving-score unknown-score ] turtles-own [ score strategy defect-now? partner-defected? ;;action of the partner partnered? ;;am I partnered? partner ;;WHO of my partner (nobody if not partnered) partner-history ;;a list containing information about past interactions ;;with other turtles (indexed by WHO values) ] ;;;;;;;;;;;;;;;;;;;;;; ;;;Setup Procedures;;; ;;;;;;;;;;;;;;;;;;;;;; to setup clear-all store-initial-turtle-counts ;;record the number of turtles created for each strategy setup-turtles ;;setup the turtles and distribute them randomly reset-ticks end ;;record the number of turtles created for each strategy ;;The number of turtles of each strategy is used when calculating average payoffs. ;;Slider values might change over time, so we need to record their settings. ;;Counting the turtles would also work, but slows the model. to store-initial-turtle-counts set num-random n-random set num-cooperate n-cooperate set num-defect n-defect set num-tit-for-tat n-tit-for-tat set num-unforgiving n-unforgiving set num-unknown n-unknown end ;;setup the turtles and distribute them randomly to setup-turtles make-turtles ;;create the appropriate number of turtles playing each strategy setup-common-variables ;;sets the variables that all turtles share end ;;create the appropriate number of turtles playing each strategy to make-turtles crt num-random [ set strategy "random" set color gray - 1 ] crt num-cooperate [ set strategy "cooperate" set color red ] crt num-defect [ set strategy "defect" set color blue ] crt num-tit-for-tat [ set strategy "tit-for-tat" set color lime ] crt num-unforgiving [ set strategy "unforgiving" set color turquoise - 1 ] crt num-unknown [set strategy "unknown" set color magenta ] end ;;set the variables that all turtles share to setup-common-variables ask turtles [ set score 0 set partnered? false set partner nobody setxy random-xcor random-ycor ] setup-history-lists ;;initialize PARTNER-HISTORY list in all turtles end ;;initialize PARTNER-HISTORY list in all turtles to setup-history-lists let num-turtles count turtles let default-history [] ;;initialize the DEFAULT-HISTORY variable to be a list ;;create a list with NUM-TURTLE elements for storing partner histories repeat num-turtles [ set default-history (fput false default-history) ] ;;give each turtle a copy of this list for tracking partner histories ask turtles [ set partner-history default-history ] end ;;;;;;;;;;;;;;;;;;;;;;;; ;;;Runtime Procedures;;; ;;;;;;;;;;;;;;;;;;;;;;;; to go clear-last-round ask turtles [ partner-up ] ;;have turtles try to find a partner let partnered-turtles turtles with [ partnered? ] ask partnered-turtles [ select-action ] ;;all partnered turtles select action ask partnered-turtles [ play-a-round ] do-scoring tick end to clear-last-round let partnered-turtles turtles with [ partnered? ] ask partnered-turtles [ release-partners ] end ;;release partner and turn around to leave to release-partners set partnered? false set partner nobody rt 180 set label "" end ;;have turtles try to find a partner ;;Since other turtles that have already executed partner-up may have ;;caused the turtle executing partner-up to be partnered, ;;a check is needed to make sure the calling turtle isn't partnered. to partner-up ;;turtle procedure if (not partnered?) [ ;;make sure still not partnered rt (random-float 90 - random-float 90) fd 1 ;;move around randomly set partner one-of (turtles-at -1 0) with [ not partnered? ] if partner != nobody [ ;;if successful grabbing a partner, partner up set partnered? true set heading 270 ;;face partner ask partner [ set partnered? true set partner myself set heading 90 ] ] ] end ;;choose an action based upon the strategy being played to select-action ;;turtle procedure if strategy = "random" [ act-randomly ] if strategy = "cooperate" [ cooperate ] if strategy = "defect" [ defect ] if strategy = "tit-for-tat" [ tit-for-tat ] if strategy = "unforgiving" [ unforgiving ] if strategy = "unknown" [ unknown ] end to play-a-round ;;turtle procedure get-payoff ;;calculate the payoff for this round update-history ;;store the results for next time end ;;calculate the payoff for this round and ;;display a label with that payoff. to get-payoff set partner-defected? [defect-now?] of partner ifelse partner-defected? [ ifelse defect-now? [ set score (score + 1) set label 1 ] [ set score (score + 0) set label 0 ] ] [ ifelse defect-now? [ set score (score + 5) set label 5 ] [ set score (score + 3) set label 3 ] ] end ;;update PARTNER-HISTORY based upon the strategy being played to update-history if strategy = "random" [ act-randomly-history-update ] if strategy = "cooperate" [ cooperate-history-update ] if strategy = "defect" [ defect-history-update ] if strategy = "tit-for-tat" [ tit-for-tat-history-update ] if strategy = "unforgiving" [ unforgiving-history-update ] if strategy = "unknown" [ unknown-history-update ] end ;;;;;;;;;;;;;;;; ;;;Strategies;;; ;;;;;;;;;;;;;;;; ;;All the strategies are described in the Info tab. to act-randomly set num-random-games num-random-games + 1 ifelse (random-float 1.0 < 0.5) [ set defect-now? false ] [ set defect-now? true ] end to act-randomly-history-update ;;uses no history- this is just for similarity with the other strategies end to cooperate set num-cooperate-games num-cooperate-games + 1 set defect-now? false end to cooperate-history-update ;;uses no history- this is just for similarity with the other strategies end to defect set num-defect-games num-defect-games + 1 set defect-now? true end to defect-history-update ;;uses no history- this is just for similarity with the other strategies end to tit-for-tat set num-tit-for-tat-games num-tit-for-tat-games + 1 set partner-defected? item ([who] of partner) partner-history ifelse (partner-defected?) [ set defect-now? true ] [ set defect-now? false ] end to tit-for-tat-history-update set partner-history (replace-item ([who] of partner) partner-history partner-defected?) end to unforgiving set num-unforgiving-games num-unforgiving-games + 1 set partner-defected? item ([who] of partner) partner-history ifelse (partner-defected?) [set defect-now? true] [set defect-now? false] end to unforgiving-history-update if partner-defected? [ set partner-history (replace-item ([who] of partner) partner-history partner-defected?) ] end ;;defaults to tit-for-tat ;;can you do better? to unknown set num-unknown-games num-unknown-games + 1 set partner-defected? item ([who] of partner) partner-history ifelse (partner-defected?) [ set defect-now? true ] [ set defect-now? false ] end ;;defaults to tit-for-tat-history-update ;;can you do better? to unknown-history-update set partner-history (replace-item ([who] of partner) partner-history partner-defected?) end ;;;;;;;;;;;;;;;;;;;;;;;;; ;;;Plotting Procedures;;; ;;;;;;;;;;;;;;;;;;;;;;;;; ;;calculate the total scores of each strategy to do-scoring set random-score (calc-score "random" num-random) set cooperate-score (calc-score "cooperate" num-cooperate) set defect-score (calc-score "defect" num-defect) set tit-for-tat-score (calc-score "tit-for-tat" num-tit-for-tat) set unforgiving-score (calc-score "unforgiving" num-unforgiving) set unknown-score (calc-score "unknown" num-unknown) end ;; returns the total score for a strategy if any turtles exist that are playing it to-report calc-score [strategy-type num-with-strategy] ifelse num-with-strategy > 0 [ report (sum [ score ] of (turtles with [ strategy = strategy-type ])) ] [ report 0 ] end ; Copyright 2002 Uri Wilensky. ; See Info tab for full copyright and license.

There are 10 versions of this model.

## Attached files

File | Type | Description | Last updated | |
---|---|---|---|---|

PD N-Person Iterated.png | preview | Preview for 'PD N-Person Iterated' | almost 10 years ago, by Uri Wilensky | Download |

This model does not have any ancestors.

This model does not have any descendants.