Plan of the tutorial
• Introduction
• Normative systems
• The agent perspective: normative
multiagent systems
• The construction of social reality
website: http://normas.di.unito.it/iat04
Introduction
Social norms
In the MultiAgent Systems field, social
norms are perceived to help improve
coordination and cooperation (Shoham
& Tenneholz 1992; Jennings and
Mandami 1992; Conte & Catselfranchi
1995; Jennings 1994; Walker &
Wooldridge 1995).
Agents cannot be assumed to be
benevolent
Why norms?
(a) How to avoid interferences and collisions
among agents autonomously acting in a
common space?
(b) How to ensure that negotiations and
transactions fulfil the norm of reciprocity?
(c) How to obtain a robust performance in
teamworks?
(d) How to prevent agents from dropping their
commitments, or how to prevent agents from
disrupting the common activity ?
Shoham & Tenneholz 1992
In multiagent systems be they human societies or distributed
computing systems different agents, people or processes, aim to
achieve different goals and yet these agents must interact either
directly by sharing information and services or indirectly by
sharing system resources. In such distributed systems it is
crucial that the agents agree on certain rules in order to
decrease conflicts among them and promote cooperative
behavior. Without such rules even the simplest goals might
become unattainable by any of the agents or at least not
efficiently attainable. Just imagine driving in the absence of
traffic rules. These rules strike a balance between allowing
agents sufficient freedom to achieve their goals and restricting
them so that they do not interfere too much with one another
Shoham & Tenneholz 1992
They consider the possibility of limiting the
agents to a subset of the original strategies of
a given game thus inducing a subgame of the
original one. They call such a restriction a
social constraint if the restriction leaves only
one strategy to each agent. Some social
constraints are consistent with the principle of
individual rationality in the sense that it is
rational for agents to accept those assuming
all others do as well.
Hard or soft constraints?
The distinction between hard and soft
constraints corresponds to the distinction
between preventative and detective control
systems. In the former a system is built such
that violations are impossible (you cannot
enter metro station without a ticket) or that
violations can be detected (you can enter
train without a ticket but you may be checked
and sanctioned).
Autonomous agents
• Agents: systems oriented to achieve states in the world.
• Goal: an explicit representation of a world state which the agent
•
•
•
•
wants to be realised; agents with goals and beliefs are cognitive
agents.
Belief: a representation of the world that the agent holds true.
Norm: an obligation on a set of agents to accomplish/abstain
from a given action,
– external: no mental representation
– internal,
Institution: a supra-individual system deliberately designed or
spontaneously evolved to regulate agents’ behaviour.
Autonomy: an agent is autonomous wrt
– its physical environment or
– other agents in the same environment -> social autonomy.
• Goal-autonomy
• Norm-autonomy
Autonomy
"...to pose a goal to oneself is something
about which no external legislation can
interfere...". An agent: "cannot undergo any
obligation other than what he gives himself on
his own. (...) only by this means it is possible
to reconcile this obligation (even if it were an
external obligation) with our will".
Kant (Die Metaphysik der Sitten, 1794)
Normative systems
and
deontic logic
Normative systems
• “Sets of agents whose interactions are normgoverned; the norms prescribe how the
agents ideally should and should not behave.
[...]
Importantly, the norms allow for the possibility
that actual behavior may at times deviate
from the ideal, i.e., that violations of
obligations, or of agents’ rights, may occur.”
(Jones & Carmo 2001)
Deontic logic
• von Wright, 1951: formal study of ought
• Deontic modalities besides alethic ones
– “it is obligatory to see to it that x” inspired
to “it is necessary that x”
– “it is permitted to see to it that x” inspired to
“it is possible that x”
Modal operator
•
•
•
•
Op = it is obligatory that p
Pp = Op it is permitted that p
Fp = Op it is forbidden that p
Minimal system D:
– O(p  q)  (Op  Oq)
– O(p)  Op
(I.e. Pp, obligatory implies permitted)
– if |- α then |- Oα
(but not Op  p like for knowledge: ideal is
not real necessarily)
Conditionals and paradoxes
• Obligations are inherently conditional:
“when you… you have to…”
• Different possibilities to define O(y|x)
– x  O(y)
– O(x  y)
– NEC(x  O(y))
• They all have counterintuitive results:
paradoxes of deontic logic
e.g. contrary to duty
O(kill) but O(gently|kill)
Anderson’s reduction I
• Reduction of deontic logic to alethic
logic:
“the intimate connection between
obligations and sanctions in normative
systems suggests that we might
profitably begin by considering some
penalty or sanction S, and define
obligations as: p is obligatory if its falsity
entails the sanction S”.
Anderson’s reduction II
• Formalization
– O(p) = NEC(p  S)
– S
• Problem: not all violations are
sanctioned
• Reply of Anderson “S just means
something bad or violation”
Dynamic logic
• Meyer, 1988: Deontic logic viewed as a
variant of dynamic logic
• a is obligatory if the effect of not doing
action a is that there is a violation:
O(a) = [¬a] V
The agent perspective:
normative multiagent systems
Social order I
Castelfranchi 2000: Social orders are patterns
of interactions among interfering agents that
allow the satisfaction of the interests of
agents, such as values or shared goals that
are beneficial for most or all of the agents
Agents delegate to the normative system their
own shared goals which become the content
of the obligations regulating the system
Social order II
For example, if agents delegate the goal
to avoid accidents to the normative
system, then the system may adopt the
subgoal to drive on the right side of the
street. This subgoal is the content of the
obligation to regulate traffic. Agents
adopt this goal since they contribute to
the delegated goal, and they know other
agents will adopt it too
Social control
Castelfranchi 2000: Social control
“An incessant local (micro) activity
of its units aimed at restoring the
regularities prescribed by norms”.
Agents attribute to the normative
system the ability to autonomously
enforce the conformity of the
agents to the norms
Violating norms
Probably, when one thinks about multiagent
systems, one assumes that the agents stick
to the obligation posed by the system.
However, this assumption is not always
realistic, so we must consider what happens
with agents that must be motivated to respect
an obligation. See heterogeneous multiinstitutional agents, like the Grid.
Why violating norms
Nobody can avoid that norms - and in particular
their instances - might be incoherent. There
might be conflicts, and the agents should be
able to manage these conflicts. Norms also
cannot predict and successfully frame all
possible circumstances. There might be some
important event or fact to be handled, where
no norm applies or some norm applies with
bad results.
Instrumental norms
• Law scholars like Hart distinguish:
– primary norms: prescription of behavior
– instrumental norms: help the achievement
of primary norms. Directed towards the
juridical system: sanctions, procedures for
trials
–…
(deontic logic focussed on primary norms)
What do we learn from this?
• Mental attitudes like goals not only at
the individual level: delegation of goals
• Normative systems: not only
specification of ideal behavior of the
system, but also active role
• Normative system has goals and does
actions. Is it an agent?
The agent metaphor
• G.Lakoff: Role of metaphor in
cognition to conceptualize reality
which is not bodily grounded.
• An ontology of social reality should
disclose the metaphorical mapping we
use to understand social reality
• Can the agent metaphor be used for
understanding social reality?
Intentional stance
• Dennet: attitudes like belief and desire are
folk psychology concepts that can be fruitfully
used in explanations of rational human
behavior.
For an explanation of behavior it does not
matter whether one actually possesses
these mental attitudes: we describe the
behavior of an affectionate cat or an unwilling
screw in terms of mental attitudes.
Dennet calls treating a person or artifact as a
rational agent the ‘intentional stance’.
The importance of us
• “The possibility of ascribing goals, beliefs,
and actions to collectives relies on the idea
that collectives can be taken to resemble
persons. […] both factual and normative
beliefs can be ascribed (somewhat
metaphorically) to groups, both formal and
informal, structured and unstructured.”
Tuomela, 1995
Norms as mental attitudes
(Boella and van der Torre)
• If a normative system is described as
an agent with mental attitudes,
thus norms are defined in terms of the
conditional mental attitudes of the
normative agent
– obligations are goals (“ideal behavior”)
– what about beliefs?
Input/Output Logics
(Makinson & van der Torre)
• Let R  Rul: a,..,d→x or (a,…d,x)
• Outi(R) is closure under set of rules
– Out1:SI
– Out3:SI,CT
a,b→x a,b→x
a→x
a,b→x
Out2:SI,OR
Out4:SI,OR,CT
SI
a→x
•Outi+: Outi and ID
OR
a→a
a→b a,b→x CT
a→x
ID
Multiagent system
MAS=<A,X,G,E,>
– A: set of agents
– X: propositional variables
– G: goal rules a,..,d→x
– E: effect rules a,..,d→x
– : priority relation on goal rules
Xa: actions of agent a, Ga: goals of a
Normative MAS
NMAS=<A,X,G,E,,N,V,n>
– n  A the normative agent
– N: a set of norms
– V: norm description N x A → X
eg V(n,a)
Anderson’s reduction:
Obligation Oa(x,s|C) if
– C, x → V(n,a)  E
– V(n,a) → s  E
Alternative approach
• Violation is not an effect of the behavior,
but an action of the normative system
• Analogously, the sanction is an action of
the normative system (with a cost)
• Recognizing violations and sanctioning
violations are goals of the normative
system
Obligations Oa,NS(x,s|Y)
1.
2.
3.
4.
Y→x is goal of NS
Y,x → V(n,a) is goal of NS
Y,V(n,a) → s is goal of NS
Y → s is goal of agent a
Hart:
Instrumental
norms
precondition
Two actions:
V(n,a) = violation by agent a of norm n
s is a sanction
Michael Luck, Fabiola López y López
•
•
•
•
•
Societies and Autonomous Agents.
How can autonomous agents be integrated
into societies regulated by norms?
What does an agent need to deal with
norms?
What does an agent evaluate before
dismissing a norm?
How are the goals of an agent affected by
social regulations?
Michael Luck, Fabiola López y López
•
•
•
•
•
A formal structure of norms that includes the
different elements that must be taken into
account when reasoning about norms
A formal basic representation of norm-based
systems
An analysis and formalisations of the kinds of
norms that norm-based systems have
An analysis of the dynamics of norms
The set of normative relationships that might
emerge by adopting, complying and
dismissing norms
Norms dynamics
Issue
Spread
Modification
Adoption
Abolition
Activation
Compliance
Reward
Violation
Sanction
Dismissal
Non-sanction
Norms compliance
hindered
by
normative gs
goals
current
goals
normative
goals
benefited
from
rewards
hindered
by
punishments
Z specification
Z specification
Emergence of norms
Off-line design: In this approach, social laws are
designed off-line, and hard-wired into agents
(Shoham & Tennenholtz 1992b; Goldman &
Rosenschein 1993; Conte & Castelfranchi 1993).
Emergence from within the system: (Shoham &
Tennenholtz 1992a; Kittock 1993), a convention can
‘emerge’ from within a group of agents.
The first approach will often be simpler to implement,
with a greater degree of control over system
functionality. But not all the characteristics of a
system are known at design time; not suited for open
systems.
Conte, Castelfranchi,
Dignum, 1998
• Social science: norms are emergent
properties of utility driven behavior.
(Binmore 1994)
• They survive if associated with
monitoring and sanctioning (Axelrod
1987, Boyd 2003)
• Social science does not explain the
decision process of autonomous agents
Norm acceptance
• Norms would not be respected if there
were the sanction only:
90% of crimes are not punished
• Norms are respected since they are
accepted
• They derive from goals delegated to the
normative system
Autonomous norm acceptance
An agent is normautonomous if it can:
(a) recognise or not a norm as a norm
(normative belief formation);
(b) argue whether a given norm concerns or not
its case; decide to accept the norm or not;
(c) decide to comply or not with it
(obey or violate);
(d) take the initiative of reissuing (prescribing)
the norm, monitoring, evaluating and
sanctioning the others' behaviour relatively to
the norm.
Goal Acceptance
• Goal-acceptance= a special case of goal-generation:
social goal-filter.
–
IF x wants p, and
–
x believes that
IF y obtains q
–
THEN x obtains p
–
THEN x wants that y obtain q.
• Autonomous agents accept a new goal iff they believe that it is a
means for an old one.
• The value of a current goal p increases if agents (are led to)
believe that p is
– Instrumental to one more important (meta-)goal q, or more
(meta-)goals Q (instrumentality beliefs. These include beliefs about
achievement costs).
– Probability of instrumental connection is higher than expected
(probability beliefs, whose credibility increases as a function of
credibility of sources. These include a different evaluation of
feasibility).
– Endangered. Maintenance goals are more compelling than
achievement ones (emergency beliefs).
Norm acknowledgement
Input = a candidate norm (external norm). An obligation in the
form OyX( q), q = the norm, y = authority that issues the norm
and X = the set of the norm subjects.
Output = possibly a normative belief. Several tests:
evaluation of the c- norm: is it based on a recognised N?
evaluation of the source: Is agi entitled to issue N? This entails:
is q within the domain of y 's competence?
is the current context the proper context of q?
is X within the scope of y 's competence?
evaluation of the motives: is q issued for agi 's personal motives?
The evaluation process is formalised as follows:
BELx(OzU( r)) & BELx(OzU( r)  OyX( q))
(10)
(OyX( q) & BELx(auth(y,X,q,C)) & BELx(mot(y,OK)))  BELx(OyX( q)) (11)
Both lead to BELx(OyX( q))
The relation “auth”: y has authority to issue q on X in C.
The relation “mot”: y's motives are correct.
Acceptance (From Conte et al., 1998)
Is N-belief sufficient? No! Belief about instrumentality.
Normative corollary of social autonomy: x will form a N-goal q iff it
believes that q is instrumental to a further goal:
BELx(OyX( q)& INSTR(OBTX(q),p) & GOALx(p|r))  N-GOALx(OBTX(q)|GOALx(p|r) & r)
Important differences from the g-generation rule:
the existence of a N-belief.
created:
BELx(O(OyX( q))
&
But norms can be autonomously
INSTR(OBTX(q),p) & GOALx(p|r))
 N-GOALx(OBTX(q)|GOALx(p|r)
&
r)
the form of the instrumental belief. But x may have internalised
the norm:
BELx(OyX( q) & INSTR(q,p) & GOALx(p|r))  C-GOALx(q|GOALx(p|r)
No N-conformity. We need:
BELx(BELy(OzX( q)))  BELx(OzX( q))
BELx(N-GOALy(OBTX(q)| r)  INSTR(OBTX(q),be_like(x,y)))
plus
GOALx(be_like(x,y)|true)
&
r)
So far...
• Agents undergo social influence, that is they are
often implicitly or explicitly requested to accept
new goals.
• Institutional influence is a special case of social
influence.
• In both cases, autonomous agents accept new
goals (including normative ones) only as means
to achieve old ones.
• Questions
– But what are the specific motives for accepting
influence and forming new goals?
– What is their respective efficacy? Which type of
influence is more effective?
Motives for Acceptance
• Trust (probability/emergency
belief)
Goal (old)
Bel
(p of connection)
Goal
(execute action)
Emotions
• Acknowledgement
Norm
(acceptance)
Bel
(instrumentality)
• Social Responsibility
– Don’t harm
 Material (e.g., passive smoking)
 Symbolic harm (break
institutional authority)
- Don’t give a bad example
Bel
(emergency)
Norm (old)
Goal
(execute action)
Motives for Acceptance (cont’)
Incentives
• Negative
– penalty
– costs of action
– obstacles
• Positive
– side-goals
– meta-goals
Bel
(instrumentality)
Bel
(importance)
Bel
(importance)
Goal
(avoid penalty
Norm
(acceptance)
Goal (old) Side-Goal
Goal
(execute action)
Meta-Goal Goal (old)
Goal
(execute action)
Motives for Acceptance (cont’)
• Social Control
Goal (old)
– Image and reputation
» Responsible
» Rational, consistent
» Trustworthy
Bel
(instrumentality)
– Social isolation
– Social identity
• Sharing (new) social norms
& values
Goal
(accept influence)
Goal
(execute action)
Bel
(value or norm)
Bel
(instrumentality
Norm or Value
(shared)
Goal
(execute action
The construction
of social reality
John Searle
“Consider a simple scene like the following. I go into a café in Paris
and sit in a chair at a table. The waiter comes and I utter a
fragment of a French sentence. I say, "un demi, Munich, à
pression, s'il vous plaît." The waiter brings the beer and I drink
it. I leave some money on the table and leave. … [Notice] that
the scene as described has a huge, invisible ontology: the
waiter did not actually own the beer he gave me, but he is
employed by the restaurant which owned it. The restaurant is
required to post a list of the prices of all the boissons, and even
if I never see such a list, I am required to pay only the listed
price. The owner of the restaurant is licensed by the French
government to operate it. As such, he is subject to a thousand
rules and regulations I know nothing about. I am entitled to be
there in the first place only because I am a citizen of the United
States, the bearer of a valid passport, and I have entered
France legally.” p.3
John Searle
• According to Searle we live in and we
are surrounded by a different kind of
reality constructed by humans
• How is it constructed?
A Two-Levelled Ontology
In The Construction of Social Reality, John Searle
argues for a two-level ontology along the following
lines. Facts on the lower level - which he calls brute
facts - can exist independently of human beings
and their institutions. Facts on the upper level, which
he calls institutional facts, depend on institutions and
on an associated 'collective intentionality'. The
existence of Planet Earth is a brute fact, the
existence of Utah is an institutional fact.
As Searle confesses, there is a sort of magic involved
when 'we impose rights, responsibilities, obligations,
duties, privileges, entitlements, penalties,
authorizations, permissions ... in order to regulate
relations between people'
Searle’s construction of
social reality
• “Some rules regulate antecedently existing forms of
behaviour. For example, the rules of polite table
behaviour regulate eating, but eating exists
independently of these rules. Some rules, on the
other hand, do not merely regulate an antecedently
existing activity called playing chess; they, as it were,
create the possibility of or define that activity. The
activity of playing chess is constituted by action in
accordance with these rules. Chess has no
existence apart from these rules. The institutions of
marriage, money, and promising are like the
institutions of baseball and chess in that they are
systems of such constitutive rules or conventions.
Counts as
For Searle, institutional facts like marriage, money and
private property emerge from an independent
ontology of “brute” physical facts through constitutive
rules of the form
“such and such an X counts as Y in context C”
where X is any object satisfying certain conditions
and Y is a label that qualifies X as being something of
an entirely new sort. E.g., “X counts as a presiding
official in a wedding ceremony”, “this bit of paper
counts as a five euro bill” and “this piece of land
counts as somebody’s private property”.
Constitutive vs regulative
norms
• Two types of norms:
– regulative norms: obligations, prohibitions,
permissions
– constitutive norms: provide a legal
classification of reality
– institutional facts: legal categories
Institutionalized power
Jones & Sergot 1996
It is a standard feature of normgoverned
institutions that designated agents are
empowered to create particular kinds of
states of affairs by means of the performance
of specified types of actions. Frequently, the
states of affairs are of a normative kind, in the
sense that they pertain to rights and
obligations, as for instance when a Head of
Department signs a purchase agreement and
thereby creates an obligation on his employer
to pay for goods received.
Means of powers
The performances by means of which these
states are established will often be of a
clearly prescribed, perhaps ritualised nature,
involving the utterance of a particular form of
words (e.g., the utterance of a specific type of
performative sentence), or the production of a
formal document, or the issuing of a pass,
perhaps in a particular context (e.g., in the
presence of witnesses).
Definition of powers I
Within institutions, organisations, or other
normative systems, there operate constraints
to the effect that the performance by some
specified agent x of some designated action
is sufficient condition to guarantee that some
specified agent y creates some (usually
normative) state of affairs F. The agent y
might be identical with the agent x, but this
need not always be so.
Definition of powers II
Often it would be appropriate to say that the agent y
who creates the state of affairs F is the institution or
normative system itself; for instance, it may be the
registrar or priest who plays the role of x, performing
the marriage ceremony, but it is the legal system or
church which creates the normative relation of being
married. We are thus led to focus on statements of
the following kind: ``According to normative
system/institution s, if agent x sees to it that A then
agent y sees to it that F”
Ex A ==>s Ex F
where Ex A to stand for ``x sees to it that/brings it about
that A'‘
Real example
The United Nations Convention on Contracts
for the International Sale of Goods (1980)
Article 15
(1) An offer becomes effective when it reaches the
offeree.
(2) An offer, even if it is irrevocable, may be withdrawn if
the withdrawal reaches the offeree before or at the
same time as the offer.
Article 63
(1) The seller may fix an additional period of time of
reasonable length for performance of the buyer of his
obligation…
Counts as
Powx (F;A) = Ex A ==>s Ex F
a conditional connective
A ==>s B: A counts as B (in institution s)
properties
(A ==>s B /\ A ==>s C)-> A ==>s (B /\C)
(A ==> s B /\ C ==>s B)-> (A\/C) ==>s B
Monotony?
Makinson 1986
Consider the case of a priest of a certain religion who
does not have permission, according to instructions
issued by the ecclesiastical authorities, to marry two
people, only one of whom is of that religion, unless
they both promise to bring up the children in that
religion. He may nevertheless have the power to
marry the couple even in the absence of such a
promise, in the sense that if he goes ahead and
performs the ceremony, it still counts as a valid act of
marriage under the rules of the same church even
though the priest may be subject to reprimand or
more severe penalty for having performed it.
Makinson: meanings of power
institutionalised
power/competence
(‘authority’)
permission
to exercise
this power
ability
to exercise
this power
Normative systems as agents
• If the normative systems can be
described as an agent: it has goals and
beliefs
• In Boella and van der Torre’s model,
obligations are defined as goals of the
agents.
What corresponds to the normative
system’s beliefs?
Constitutive norms as beliefs
• If constitutive norms provide a legal
classification of reality, they can be
considered as the beliefs of the
normative agent.
• x Counts As y in C:
C,x → y is a belief of NS
– y is an institutional fact
– x is a “brute fact” or an institutional fact
– C is the context
Beyond Searle’s
constitutive norms
• Changing the normative system
• Hart: private citizens becomes
legislators
• Constitutive norms specify how the
system can be changed by itself or by
other agents