Essay

 on  Interaction  Design  for  

Ubiquitous  Computing  
Alexandre  Fleury,  July  30,  2010  

This  essay  is  part  of  the  PhD  course  on  Interaction  Design  for  ubiquitous  Computing  taught  at  Aalborg  
University  in  June  2010.  The  purpose  of  this  exercise  is  to  summarize  the  state-­‐of-­‐the-­‐art  literature  used  for  
the  course  and  discuss  some  of  the  current  research  questions  and  challenges  that  animate  the  field.  
Additionally,  specific  design  activities  carried  out  during  the  course  are  introduced  and  analyzed.  

State-­of-­the-­art  
According  to  (Sharp,  Rogers,  &  Preece,  2007),  interaction  design  is  concerned  with  designing  interactive  
products  to  support  the  way  people  communicate  and  interact  in  their  everyday  and  working  lives.  The  field  
of  ubiquitous  computing,  allegedly  started  in  the  early  1990s,  has  been  defined  as  a  parallel  shift  where  
technology  becomes  invisible  in  people’s  life.  

Combining  the  two  fields  has  been  the  topic  of  visionary  work  since  the  late  1990s  (Weiser, 1999),  where  
computer  based  systems  where  envisioned  to  be  omnipresent  in  our  environment  at  various  scales,  from  
stylus-­‐controlled  page-­‐size  “pads”  to  tiny  “tabs”,  assisting  people  at  home  or  at  the  office  in  their  everyday  
tasks.  More  recently  (Harper, Rodden, Rogers, & Sellen, 2008)  discussed  a  possible  future  for  Human-­‐
Computer  Interaction  (HCI).  The  authors  consider  even  more  dramatic  changes  in  the  way  people  interact  
with  technology  in  their  everyday  life,  looking  at  sociological  evolution  in  parallel  to  technological  
development.  One  of  the  novelties  in  their  approach  compared  to  earlier  studies  is  the  role  human  values  
play  in  shaping  future  HCI  advancements.  It  seems  that  user  experience  is  much  more  than  a  buzz  word  
replacing  usability  as  defined  for  instance  in  (Nielsen, 1994),  but  it  integrates  factors  that  match  more  
closely  what  humans  really  experience  in  their  daily  life.    

Key  challenges  and  research  questions  

One  of  the  key  challenges  for  integrating  those  values  into  the  design  process  of  future  interactive  
technologies  and  products  is  to  promote  sources  of  inspiration  from  fields  outside  traditional  HCI  practices.  
Such  sources,  taken  from  philosophy,  are  the  meaning,  purpose  and  desire  attached  to  human  beings  and  
their  perception  of  interaction  with  technology.  Future  ubiquitous  computing  development  should  
therefore  address  the  question  of  assimilating  these  new  values  into  the  design  process.  This  could  start  by  
extending  the  traditional  user-­‐centred  cycle  by  emphasising  a  stage  of  understanding  the  people  targeted  
by  the  system,  their  environment  and  their  expectations  toward  the  system,  as  proposed  in  (Harper,
Rodden, Rogers, & Sellen, 2008).  Another  approach  is  to  consider  the  shift  in  factors  that  motivate  change  
in  people  attitude  toward  information  technologies  and  integrate  their  visible  and  yet  invisible  impact  on  
that  change,  as  attempted  in  (Howard, Kjeldskov, & Skov, 2007).  

Another  challenge  comes  from  the  distribution  of  human  activities  among  dimensions  such  as  time,  
location  and  social  relations.  The  course’s  reading-­‐list  splits  the  work  studying  ubiquity  at  home  and  
ubiquity  in  public,  which  illustrates  the  difficulty  to  address  human  activities  as  a  whole.  On  the  contrary,  it  
is  argued  that  both  environments  have  their  own  cultural,  social  and  behavioural  rules  that  should  be  
considered  when  integrated  into  an  interactive  product.  The  key  questions  are  then  to  identify  these  rules  
and  apply  them  to  the  products  supporting  the  activities  they  regulate.  And  that  is  only  considering  the  
personal  part  of  people’s  life,  and  not  integrating  its  tendency  to  blur  with  its  professional  counterpart,  
which  opens  for  even  more  research  opportunities.  

Interactions  between  family  members  

One  of  the  key  purposes  of  ubiquitous  computing  when  it  comes  to  home  environment  is  concerned  with  
the  communication  between  family  members.  For  instance,  (Dalsgaard, Skov, & Thomassen, 2007)  
explored  situations  of  separation  between  children  and  their  parents.  The  authors  investigated  how  to  
cultivate  a  feeling  of  intimacy  and  support  experience  sharing  through  a  picture  weblog.  Based  on  an  
ethnographic  study  conducted  with  families,  it  can  be  concluded  that  the  weblog  helped  parents  
understanding  the  experiences  their  children  had  and  initiate  discussions  with  them  based  on  the  pictures  
taken.  For  the  children,  the  weblog  provided  an  easy  way  to  share  aspects  of  their  life  with  their  parents  
and  enhanced  their  feeling  of  presence.  However,  children  showed  little  motivation  to  take  and  share  
pictures  with  their  parents.  

In  another  study  of  computer  mediated  family  communication,  (O'Hara, Harper, Unger, Wilkes, Sharpe, &
Jansen, 2005)  investigated  the  use  of  a  person-­‐to-­‐home  SMS  system,  allowing  mobile  phone  owners  to  
send  public  messages  available  to  any  other  family  members.  An  ethnographic  study  of  the  system  showed  
that  micromanagement  messages  were  mostly  used  as  for  informing  about  one’s  current  activity  or  to  
remind  others  about  something  that  needs  to  be  done.  The  success  of  the  product  can  be  therefore  
attributed  to  its  support  of  simple,  already  known  interaction  paradigms,  although  for  a  different  purpose.  

Exploiting  public  space  

Designing  for  interactions  in  public  place  also  requires  careful  consideration  of  the  key  challenges  identified  
previously.  The  two  following  studies  show  that  in  such  context,  the  attitude  toward  interaction  is  
influenced  by  even  more  factors  than  when  interacting  within  a  close  circle  such  as  a  family.  For  instance,  
(O'Hara, Lipson, Jansen, Unger, Jeffries, & Macer, 2004)  developed  and  studied  a  voting  system  for  
choosing  music  to  be  played  in  a  café.  A  situated  field  trial  showed  the  importance  of  integrating  the  
mediating  device  (on  which  groups  of  users  casted  their  votes)  into  the  social  setting  of  the  area,  allowing  
focused  exchange  among  group  members,  supporting  communication  between  groups  and  requiring  low  
attention  in  order  to  best  fit  into  usual  social  activities  in  such  context.  

Finally,  (O'Hara, Glancy, & Robertshaw, 2008)  investigated  people’s  engagement  and  interaction  with  a  
collaborative  game  played  through  large  urban  displays.  Their  findings  concern  four  key  issues  to  be  
addressed  when  deploying  such  system:  1)  the  location  itself  and  how  people  occupy  it  in  an  everyday  
setting;  2)  the  public  nature  of  the  interaction  and  the  configuration  of  users  within  the  space;  3)  the  two-­‐
ways  impact  of  the  audience  on  the  users  and  of  the  users  on  the  audience  in  shaping  the  user  experience  
with  the  system;  and  4)  in  the  paper  specific  case,  the  unhosted  nature  of  the  game  proposed,  where  
explanation,  motivation  and  commentary  from  the  audience  was  limited.  

2   Alexandre  Fleury  |  Aalborg  University  

 
The  digital  kitchen  
The  process  of  interaction  design  involves  the  four  following  steps  (Sharp,  Rogers,  &  Preece,  2007):  

1. Identifying  needs  and  establishing  requirements  for  the  user  experience.  

2. Developing  alternative  designs  that  meet  those  requirements.  
3. Building  interactive  versions  of  the  designs  so  that  they  can  be  communicated  and  assessed.  
4. Evaluating  what  is  being  built  throughout  the  process  and  the  user  experience  it  offers.  

The  two  first  steps  have  been  the  focus  of  the  course’s  continuous  exercise,  during  which  six  groups  of  4-­‐5  
students  developed  creative  ideas  to  support  various  tasks  in  a  “digital  kitchen”.  The  target  concept  of  
group  #2  was  the  planning  of  cooking,  which  involves  all  the  activities  prior  to  the  actual  cooking.  The  
developed  concept  should  support  the  planning  of  cooking  experiences  through  tips  and  ideas  on  the  meal,  
the  produce,  and  cooking  methods.  The  following  sections  report  on  the  work  generated  by  group  #2,  from  
the  identification  of  key  issues  to  the  design  and  creation  of  early  mock-­‐ups  illustrating  an  innovative  
concept  developed.  

Issues  
Planning  the  cooking  depends  on  multiple  parameters  such  as  the  people  involved  in  the  cooking,  the  time  
allocated  to  the  planning,  the  place  where  it  happens  as  well  as  the  ingredients  and  tools  required  to  later  
perform  the  actual  cooking.  However,  the  following  wh-­‐parameters  that  determine  how  to  plan  the  
cooking  can  vary  quite  dramatically,  as  illustrated  by  the  related  questions  below:  

• Who:  Who  are  we  cooking  for?  Who  is  cooking?  Are  the  people  cooking  the  same  as  the  ones  
planning  (for  instance  doing  the  shopping)?  This  can  be  linked  to  the  where  question  (opportunistic  
planning)  
• When:  When  does  the  planning  take  place?  How  much  time  is  required  to  plan?  Is  this  time  
available?  Is  it  possible  to  allocate  this  time  to  several  actors?  How  does  this  influence  the  time  
later  dedicated  to  cooking?  
• Where:  Where  does  the  planning  take  place?  At  home?  At  the  shop?  While  commuting?  Does  this  
require  synchronisation  between  the  actors  involved?  
• What:  What  is  to  be  cooked?  What  ingredients/tool  are  already  available?  What  is  missing?    

We  can  see  that  the  activity  of  planning  the  cooking  spreads  along  multiple  dimensions  and  involves  
knowledge,  coordination/synchronisation  and  decision-­‐making  power.  Coming  up  with  a  concept  that  can  
fit  in  this  framework  should  therefore  consider  if  not  address  these  questions.  

Design  process  –  group  #2  

To  identify  concepts  that  address  the  overall  problem  of  planning  the  cooking,  each  member  of  group  #2  
started  by  individually  mapping  a  personal  representation  of  the  issues  that  seemed  of  importance,  which  
we  presented  and  discussed  as  a  group  afterward.  This  exercise  produced  a  general  perspective  of  the  
problem  and  helped  identify  key  issues  that  should/could  be  addressed.  As  an  example,  sharing  the  
inventory  of  available  ingredients  among  several  actors  located  at  various  places  (one  at  home  and  one  at  
the  store)  is  possible  today  using  mobile  phones,  however  combining  this  inventory  with  the  store’s  stock  
and  generating  ideas  for  recipes  are  not  so  convenient  in  this  way.  

3   Alexandre  Fleury  |  Aalborg  University  

 
The  group  then  though  about  a  system  that  would  support  this  distributed  environment  in  which  cooking  is  
generally  planned  in.  It  was  agreed  that  the  system  should  address  a  wide  population  (couples,  friends  and  
families)  and  not  focus  on  one  specific  target  group.  Furthermore,  the  system  should  focus  on  the  following  
values:    

• Sociality:  the  system  should  involve  more  that  one  person  

• Collaboration:  all  actors  should  act  together  in  the  planning  process  
• Interactivity:  the  system  should  allow  the  actors  to  communicate  easily  
• Creativity:  the  system  should  support  the  generation  of  creative  ideas  for  cooking,  through  for  
instance  new  recipes  or  optimization  of  ingredients  available  

Sketching  1  –  initial  design  

The  first  sketching  exercise  consisted  in  producing  an  early  design  of  the  envisioned  system,  illustrating  its  
function,  its  possible  look,  and  how  to  use  it.  

The  early  prototypes  individually  produced  by  all  group  members  have  been  quite  rapidly  combined  during  
the  discussions  that  followed  their  presentation,  into  a  theoretical  overview  of  the  system,  yet  still  blurry  at  
this  time  of  the  design  process.  Additionally,  these  discussions  put  light  on  an  issue  that  emerged  during  
such  discussion,  namely  the  diversity  of  opinions  and  the  difficulty  to  reach  agreements  within  short  delays.  
In  fact,  the  time  required  to  decide  which  direction  the  group  should  adopt  in  order  to  produce  a  creative  
and  innovative  concept  appeared  as  a  function  of  both  the  number  of  people  in  the  group  and  the  
individual  background  of  each  member.  [Maybe  develop  on  solutions  to  this  problem]  

The  idea  that  emerged  during  this  step  was  of  a  system  that  would  allow  multiple  people  involved  in  the  
planning  process  to  communicate  through  the  planning  process,  based  on  the  ingredients  already  available  
in  the  home’s  fridge,  some  ideas  for  recipes  from  the  actors,  and  the  food  available  at  nearby  stores.  The  
concept  of  the  “Knight  Freezer  2000”  (Figure  1)  that  we  presented  at  the  end  of  the  exercise  session  refers  
to  this  category  of  futuristic  almighty  automated  systems,  which  would  support  people  in  their  tasks,  as  
they  used  to  be  depicted  in  1980s  TV  series.  Such  system  seems  realistic  nowadays  in  the  case  of  a  digital  
kitchen.    

       
Figure  1  -­‐  Illustration  of  conceptual  presentation  

Sketching  2  –  redesign  
The  second  stage  of  the  design  process  originally  consisted  in  starting  mocking  up  the  early  sketches  
produced  in  stage  1.  However,  given  the  delays  the  group  took  during  the  initial  discussions,  the  pieces  of  
advice  and  comments  received  during  the  presentation  and  the  group’s  own  considerations,  we  decided  to  

4   Alexandre  Fleury  |  Aalborg  University  

 
keep  on  working  on  the  early  design  of  our  concept  by  slightly  shifting  its  focus  and  defining  it  in  further  
details.  

The  redesign  illustrates  various  key  stages  of  the  planning  process  that  encompasses  the  four  target  values  
introduced  earlier  in  this  document.  The  three  focus  stages  are  illustrated  in  the  following.  

1. Choose  

This  stage  takes  place  either  in  the  home  (Figure  2,  left)  or  at  a  shop  (Figure  2,  right).  It  involves  selecting  
the  products  required  for  the  cooking,  based  on  what  is  available,  what  is  possible  to  buy  and  the  desired  
meal.  

   
Figure  2  -­‐  Illustration  of  the  choosing  stage,  at  home  (left)  or  at  the  shop  (right)  

2. Discuss  

This  stage  allows  the  various  actors  of  the  system  to  communicate  while  planning  (Figure  3),  possibly  
remotely,  and  while  having  access  to  all  the  information  required  to  make  common  decisions.  

 
Figure  3  -­‐  Illustration  of  the  discussion  stage  

3. Guide  

Finally,  this  stage  involves  the  possibility  to  access  external  sources  of  inspiration  for  recipes,  information  
on  seasonal  products,  etc.  This  information  could  for  instance  come  from  shop  assistants  (Figure  4)  or  
community  based  web  services.  

5   Alexandre  Fleury  |  Aalborg  University  

 
  
Figure  4  -­‐  Illustration  of  guidance  from  a  shop  assistant  

Mocking  up  –  final  concept  

In  this  final  stage  of  the  exercise,  the  group  turned  the  previously  mentioned  design  sketches  into  mock-­‐
ups,  illustrating  the  concept  in  a  simple  yet  visually  explicit  fashion.  We  had  to  integrate  the  two  mocking  
stages  originally  planned,  showing  the  various  components  of  the  system,  their  integration  into  the  
environment,  the  related  human-­‐computer  interaction  as  well  as  some  specific  aspect  of  the  interactions,  
such  as  buttons,  interaction  paradigms  and  central  screens.  

To  do  so  we  based  our  presentation  on  a  storyline,  using  a  very  simple  mock  up  to  illustrate  the  stages  
introduced  previously.  In  the  following  illustrations,  one  of  the  actor  s  involved  in  the  planning  is  at  home,  
manipulating  a  digital  tabletop  that  displays  a  list  of  possible  recipes  based  on  the  products  available  in  the  
fridge  (Figure  5).  

 
Figure  5  -­‐  Planning  from  home  

At  the  same  time,  another  actor  is  shopping  and  connects  to  the  systems  via  a  similar  installation  located  in  
the  store  (Figure  6,  left).  The  two  actors  can  communicate  via  this  interface  and  the  one  at  home  can  access  
what  has  been  put  in  the  shopping  bag  of  the  one  in  the  shop  (Figure  6,  right).  

6   Alexandre  Fleury  |  Aalborg  University  

 
    
Figure  6  -­‐  Communicating  through  the  interface,  from  the  shop  (left)  and  from  home  (right)  

A  recipe  can  be  commonly  agreed  on  though  the  system,  and  a  shopping  list  (accessible  to  both  actors)  is  
automatically  created  based  on  what  products  are  missing  (Figure  7).  

 
Figure  7  -­‐  Creating  a  common  shopping  list  

In  the  meantime  a  shop  assistant  offers  a  good  deal  on  a  product  that  was  not  part  of  the  shopping  list.  The  
two  actors  discuss  the  necessity  to  buy  such  product  and  the  shopping  list  is  updated  accordingly  (Figure  8).  

7   Alexandre  Fleury  |  Aalborg  University  

 
  
Figure  8  -­‐  Updated  shopping  list  after  shop  assistant  recommendation  

This  concludes  the  planning  process  of  the  cooking  for  that  day,  and  at  the  same  time  the  course  exercise.  

Bibliography  
Dalsgaard,  T.,  Skov,  M.  B.,  &  Thomassen,  B.  R.  (2007).  eKISS:  Sharing  Experiences  in  Families  through  a  
Picture  Blog.  Proceedings  of  BCS-­‐HCI  2007  .  

Harper,  R.,  Rodden,  T.,  Rogers,  Y.,  &  Sellen,  A.  (2008).  Being  Human:  Human-­‐Computer  Interaction  in  the  
year  2020.  Cambridge:  Microsoft  Research.  

Howard,  S.,  Kjeldskov,  J.,  &  Skov,  M.  B.  (2007).  Pervasive  computing  in  the  domestic  space.  Personal  
Ubiquitous  Computing  ,  11  (5),  329-­‐333.  

Nielsen,  J.  (1994).  Usability  Engineering.  Morgan  Kaufmann.  

O'Hara,  K.,  Glancy,  M.,  &  Robertshaw,  S.  (2008).  Understanding  collective  play  in  an  urban  screen  game.  
Proceedings  of  CSCW  '08  (pp.  67-­‐76).  New  York:  ACM.  

O'Hara,  K.,  Harper,  R.,  Unger,  A.,  Wilkes,  J.,  Sharpe,  B.,  &  Jansen,  M.  (2005).  TxtBoard:  from  text-­‐to-­‐person  
to  text-­‐to-­‐home.  CHI  '05  Extended  Abstracts  on  Human  Factors  in  Computing  Systems  (pp.  1705-­‐1708).  New  
York:  ACM.  

O'Hara,  K.,  Lipson,  M.,  Jansen,  M.,  Unger,  A.,  Jeffries,  H.,  &  Macer,  P.  (2004).  Jukola:  democratic  music  
choice  in  a  public  space.  Proceedings  of  DIS  '04  (pp.  145-­‐154).  New  York:  ACM.  

Perry,  M.,  O'hara,  K.,  Sellen,  A.,  Brown,  B.,  &  Harper,  R.  (2001).  Dealing  with  mobility:  understanding  access  
anytime,  anywhere.  ACM  Transactions  on  Computer-­‐Human  Interaction  ,  8  (4),  323-­‐-­‐347.  

Sharp,  H.,  Rogers,  Y.,  &  Preece,  J.  (2007).  Interaction  Design:  Beyond  Human-­‐computer  Interaction  (2nd  
edition).  Wiley.  

Weiser,  M.  (1999).  The  computer  for  the  21st  century.  SIGMOBILE  Mobile  Computing  and  Communications  
Review  ,  3  (3),  3-­‐-­‐11.  

8   Alexandre  Fleury  |  Aalborg  University