Understanding The Connected Home

Table
of Contents
Introduction
1.1
Values
1.2
The home as the next augmentative technology
1.3
Past
1.3.1
Present
1.3.2
Future
1.3.3
Concepts
1.4
Interaction layers
1.4.1
Connected objects
1.4.2
Connected literacy
1.4.3
Negotiating with connected systems
1.4.4
Controlling privacy
1.4.5
Relationships
1.5
Hospitality
1.5.1
Sharing connected objects
1.5.2
Moving in and out of homes
1.5.3
Who controls your home?
1.5.4
Inviting connected objects into our homes
1.5.5
Surveillance & tracking
1.5.6
Research approaches
1.6
Sprints
1.6.1
Exhibitions
1.6.2
Research roadmap
1.7
Policy matters: The connected home and the smart city
1.7.1
Neighborhoods
1.7.2
Where from here
1.8
Acknowledgments
1.9
Contact us
1.10
Introduction
Understanding the connected home

While there has been great progress in terms of user experience, manufacturing and
technology, there has not been much reflection about the implication of the connected
homes for our lives.
How will ubiquitous connectedness and data change our domestic lives? What are
the powers currently shaping this landscape and what do they have to gain? And
more importantly, how can we as practitioners shape IoT to be a net positive for
society?
We wrote this book with practitioners in mind. We hope the people who read this will be the
people who make connected homes happen. That could include designers, developers,
strategists, entrepreneurs, researchers, activists, educators and many more.
About the authors
Introduction
Peter Bihr | @peterbihr

Peter Bihr explores the impact of emerging technologies. He founded The Waving Cat to
apply these insights through consulting, R&D, conferences and publications.
As a strategy advisor, he helps organizations large and small to excel in an environment
shaped by digitization, connectedness and rapid change.
He co-founded many emerging technology conferences including ThingsCon, UIKonf and
Cognitive Cities Conference and co-chaired Interaction16. He also co-founded the Good
Home Project.
Michelle Thorne | @thornet

Michelle leads Mozilla's exploration of the Internet of Things. She serves a professional
learning community seeking to shape IoT with openness and user empowerment.
Previously, as Mozilla's Director of Web Literacy Programs, she supported thousands of
professional educators and activists to teach and advocate for the web.
Michelle has a dedicated interest in open practices and design, curating the Mozilla Festival,
exhibiting with The Good Home, writing for Open Design Now and co-authoring the book An
Open Web.
Introduction
Intro
1
In May 2015, science fiction author Bruce Sterling gave a keynote at ThingsCon . In his
2
talk he introduced a new research project he and his partner Jasmina Tesanovic have been
working on: Casa Jasmina3, the open source connected home of the future.
It's time to live the life. Just go ahead and build the products and see if you can survive
being in a room with them. Casa Jasmina is our test bed. Bruce Sterling at
ThingsCon
A few weeks later in June 2015, we flew to Turin, Italy along with Alexandra DeschampsSonsino4 to visit Casa Jasmina and spend some time with the team.
We were curious to see how it would be to live in and contribute to this home of the future,
and we're very grateful we got to experience it so early on.
Ever since visiting5 Casa Jasmina, some questions wouldn't leave us alone. There hasn't
been a day where the topic of connected homes hadn't come up, where we haven't been
trying to get closer to figuring out answers, or at least asking better questions.
How do we interact with a connected home? How does a space communicate what's
expected of the people who live in it? What are the ground rules and who has
permission to change them? What objects require interaction and what don't? How do
we know how to build and live in a connected home?
We asked a great number of smart and experienced people for their thoughts. It slowly
emerged that "the connected home" is an area that's not understood yetit is so new, with
so many unanswered questions and with so few connected homes actually in use.
We know that connectivity increasingly makes its way into our living rooms, kitchens and
bedrooms. The internet is coming into our smoke detectors, lights, door locks, kitchen scales
and ovens. We bring in more connectivity through fitness tracking wristbands and our
phones and tablets, and take it along when we get into the driving computers that are our
cars.
Understanding the connected homehow it can be designed, how we can engage
with it and control it, how we can live with it and still have agency over our datawill
be essential to address.
Connected homes will impact the lives of hundreds of millions people around the globe.
There are many challenges ahead, including designing interactions, privacy, user
empowerment in this new field. We hope this modest exploration will be of use to designers,
developers, entrepreneurs and policy makers as we figure this out together over the
Introduction
decades to come.
Welcome to version 2
We published version 1 of Understanding the Connected Home a few months after our visit
to Turin, in the fall of 2015.6
It turns out this collection of essays could not have been more timely. Just about every major
tech company has now entered the field of connected homes with a smart home hub, a wifienabled light bulb or thermostat or some other gadget.
We believe that understanding connected homes is even more urgent and relevant
now.
While there has been great progress in terms of user experience, manufacturing and
technology, there has not been much reflection about the implication of the connected
homes for our lives. How will ubiquitous connectedness and data change our domestic
lives? What are the powers currently shaping this landscape and what do they have to gain?
And more importantly, how can we as practitioners shape IoT to be a net positive for
society?
We wrote this book with practitioners in mind. We hope the people who read this will be the
people who make connected homes happen. That could include designers, developers,
strategists, entrepreneurs, researchers, activists, educators and many more.
The lens we bring is that of experience working with fellow IoT practitioners. We are involved
in many IoT conversations, and we're learning as we go, just like everyone in this field.
In this way, we see ourselves as part of a professional peer-learning community. It's an
informal, global network of designers, technologists, activists and more who collaborate with
one another and exchange insights and best practices so that collectively we can evolve the
IoT conversation and influence the state of the art.
Why this update ?

The initial publication of this book coincided with the start of a number of projects &
conversations. Since then, we have developed our thinking and followed up on some of the
questions we addressed in version 1:
September 2015: In collaboration with Alexandra Deschamps-Sonsino7, Peter launched
The Good Home Project8 to explore the same set of questions this book addresses
through design proposals and exhibits.
Introduction
January 2016: Michelle took on a new role at Mozilla Foundation, leading the Open IoT
Studio with Jon Rogers, Professor of Creative Technology at the University of Dundee.9
March 2016: Peter co-chaired Interaction1610, a conference that gathered the global
interaction design community in Helsinki where many questions of designing connected
experiences, services and products were discussed indepth.
April 2016: The Good Home Project exhibited at Fuori Salone in Milan with Iohanna
Nicenboim11 and Michelle.
April 2016: Michelle curated a series of design sprints around privacy and the
connected home, notably the Mozilla Open IoT Design Sprint in Berlin.12
April 2016: The German government published a research report and set of policy
recommendations that Peter co-authored with Prof. Dr. Christoph Bieber on smart cities
and their implications for citizens.13
And throughout, ThingsCon hosted numerous local community events that explore the
techniques, design and ethics of connected products and services.
It has been a busy and productive year, with many touch points that are directly relevant to
Understanding the Connected Home. All of those activities evolved our thinking. That meant,
it was time to revisit the book.
For this new and updated version, we edited and revised the original essays, restructured
the whole bit, and added new content. We hope this makes for more accessible reading.
New chapters were added to bring the book up to date, offering fresh content for readers of
the first version.
As always, we look forward to hearing about your thinking and hope to talk to you online or
face-to-face.
How to use this book

We laid the ground for this book in a one-week content sprint in 2015. Since then, it has
become an ongoing research project spawning conversations and results both in this book
(as version 2) as well as in other areas of our work.
We have been writing this in public on Gitbook, and license the content under a Creative
Commons Attribution Non-Commercial license (BY-NC), so you can adapt and re-use the
texts, too. If in doubt or if you have feedback, please get in touch (details in the Contact
section).
We also published a Kindle-optimized version on the Amazon Kindle store (which is also the
way to support this and further books).
Introduction
Peter Bihr & Michelle Thorne

Berlin, June 2016
1. ThingsCon is a conference about the internet of things and maker culture that Peter
co-founded and has been running with Simon Hher, Max Krger and Emanuel
Schwarz since 2014 in Berlin. Learn more about ThingsCon at thingscon.com
2. Available at https://www.youtube.com/watch?v=6vtwoUugXrQ
3. Casa Jasmina is a joint research project by Bruce Sterling and Jasmina Tesanovic as
well as the Arduino team, represented by Lorenzo Romagnoli and Massimo Banzi. To
learn more about Casa Jasmina visit casajasmina.arduino.cc
4. Learn more about Alexandra on her website designswarm.com and follow her on
Twitter @iotwatch.
5. For detailed reports visit Michelle's blog post on michellethorne.cc and Peter's at
thewavingcat.com.
6. Version 1 is available in the archives as a PDF:
dropbox.com/s/4yp9delbrw9l0wu/20150911%20understanding-the-connectedhome.pdf?dl=0
7. designswarm.com
8. thegoodhome.org
9. Learn more about Jon on his university website dundee.ac.uk/djcad/staff/jonrogers/
and follow him on Twitter @ileddigital.
10. interaction16.ixda.org
11. iohanna.com
12. thewavingcat.com/2016/04/25/privacy-machines/
13
. www.thewavingcat.com/2016/04/25/smart-cities-in-the-21c-humanity-on-the-movethe-transformative-power-of-cities/
Values
Values
We believe that the connected home should respect several foundational values:
1. Privacy. We consider privacy a core value to be included throughout the design,
manufacture, usage and end of a connected product or service.
2. Read, write, participate. As a matter of empowerment, people should be able to read,
write and participate fully in their connected home.
3. Openness. Openness leads to more robust, resilient, and participatory products. For
creators of connected products, we advocate for open practices and technologies paired
with open reflection and documentation that can serve professional communities learning
how to make things with these values.
4. Diversity and inclusion. The connected home is a safe space and must be designed
with diversity and inclusivity in mind. This includes challenging assumptions about cultural
context as much as taking into account socio-economic factors, gender and other forms of
identity.
5. Security. Users and their data need to be protected from any party that might try to track,
spy, or hack. Aggregate data needs to be anonymized in a way that guarantees that data
cannot be traced back to identifiable users.
6. Sustainability. Sustainable sourcing and manufacturing as well as designing for end-oflifecycle are important ecological and social considerations as well as an opportunity to
position a connected home company better in the market place while saving their customers
money.
These values can guide makers of connected products and services. By building these
values directly into the products they make, designers and engineers can in aggregate
positively shape the Internet of Things landscape.1
1. Privacy
Our home needs to be a safe space. It must be respectful of its inhabitants, especially when
it comes to sensitive information and personal data. As the home becomes more connected,
data-driven services can put this safe space at risk.
Therefore, we must build safeguards such as:
strong privacy default settings
10
Values
requiring clear and unambiguous consent from users

contingency plans for data breaches
strict transparency rules
options to delete data
and the ability to port data to other services
These safeguards will have to be backed with legal muscle and regulatory accountability for
data service providers and technology companies.
Importantly, since many start-ups are creating connected home products, it is especially
important to ensure these volatile companies adhere to these safeguards as well.
Privacy by design
Connected home systems should be private by design2 by following measures such as:
Proactive not reactive; preventative not remedial
Privacy as the default setting
Privacy embedded into design
Full functionality positive-sum, not zero-sum
End-to-end security full lifecycle protection
Visibility and transparency keep it open
Respect for user privacy keep it user-centric
The best way of protecting data is to not collect it in the first place
Consent
Best practices around consent help build users' trust in connected products. So far, user
acceptance of smart home products has been limited, to say the least.
To foster trust, makers of connected products can ensure valid consent is explicit for
collecting as well as its usage. This consent must be verifiable. Data controllers must be
able to prove "consent" (opt-in), and consent may be withdrawn at any time.
Data breaches
Data breaches are a fact of digital life.
In Europe, there's already regulation in place to determine what happens in a breach.3 It
must be reported to a supervisory authority, and individuals have to be notified if adverse
impact is determined.
11
Values
The best companies will go way beyond the legal minimum and proactively seek out users
and propose actionable steps to remedy the situation.
Data deletion
Furthermore, users should always be able to delete their data. Again, the regulation in
Europe is already in place.
The data subject has the right to request erasure of personal data related to him on any
one of a number of grounds including non-compliance with article 6.1
(lawfulness)...where the legitimate interests of the controller is overridden by the
interests or fundamental rights and freedoms of the data subject which require
protection of personal data.
Of course, how deletion works with aggregated data and big data sets is tricky.
Nevertheless, these frameworks show how values around a user's control of their personal
data can be articulated and advocated.
Data portability
In addition to data deletion, users should be able to move their data across connected home
services. Portability should be guaranteed even as a user moves across different
companies' products.
As an innovation opportunity, connected products can offer easy data import and export
functions as well as adopt standards that build in portability.
2. Read, Write and Participate

Everyone should be able to understand how the connected systems in their homes work.
And from that understanding, everyone should have the ability to modify and adapt these
systems to meet their needs and fully participate in their connected home.
To achieve that, users need both the skills to interact with these systems (literacy)4 as well
as the technical features and functions that enable these interactions.
Read
Users can see connected systems and understand how its parts interrelate. This includes
aspects like being able to see actionable, contextual data and visibility in what happens in
the system after certain actions are taken.
12
Values
In practice, "reading" the connected home may be about recognizing common visual cues.
Perhaps these cues are standardized symbols displayed on connected objects, or status
lights that indicate what's happening to your data, or whether an object is actively listening to
audio inputs.
The ability to "read" also applies to data that the home generates. These data sets could
include maintenance reports, electricity usage, connectivity maps and more.
Users in the home should be able to read, evaluate and manipulate these datasets as well
as understand the algorithms that create and compute with them.
Write
Users should be able to modify their connected home, including the data and algorithms in it,
as well as create new ones.
We should be able to query APIs in our home and to build bespoke services on top of them.
It should also be possible to identify data that has been modified by other members of the
household and to navigate permissions to accept or edit those changes.
Participate
Lastly, we should be empowered to fully participate in connected home and the data
infrastructure that powers and augments it. This touches on aspects like sharing,
collaborating, and open practices.
How can "connected home data" be documented and distributed to other households or
contributed to the commons? For example, a home might generate interesting data about
the local weather, which it can share with the neighborhood or national weather service.
People in the home should be able to understand these kinds of contributions and
participate in them when they choose.
Furthermore, how can people cultivate healthy relationships with the people they live with
and next to through (and sometimes in spite of) connected systems? How can preferences
be expressed and conflicts resolved in a way that gives individuals agency and control?
As our homes become more connected, we must preserve the ability to read, write and
participate fully in them.
An open source motto says: If you can't open it, you don't own it. We should all be able to
openand changethe things in our homes.
3. Openness
13
Values
Open practices for better connected products

Best practices from the free & open source (FOSS) movementboth in software and
hardwarefully apply to the connected home.
Quality control
Especially the positive effects of increased transparency and decentralized quality control
can add essential additional security to connected homes.
Consider this as a rule of thumb: Bugs in proprietary software and apps are bad (or at least
annoying) but often users can work around them. Bugs and security holes in your home are
much worse, and due to the nature of networked complex computational systems, these
bugs tend to compound.
In other words, security, resilience, and strong code is key for the connected home.
Diversity & inclusion

The home is a place that is particularly strongly shaped by cultural context and the "users" in
it.
Open practices enable people to participate in the creation and use of technologies in their
home. This accessibility fosters inclusion and brings diverse perspectives to what a
connected home is for and what it can do.
Design & empowerment

Open practices apply not just to the making of hardware and software, but also to how we
design and learning about the connected home.
Open design practices, combined with democratized manufacturing thanks to the maker
movement, empower the residents of a home to maintain, fix, adapt, modify and improve
their connected homes.
Home improvement takes on a new dimension when paired with open source code bases
and open design resources, plus open networks of people learning and teaching each other
how to hack their connected homes.
Networking & APIs

Services and products in the connected home should be as open as possible. An API
enables others to bridge services and build new things by mashing up products.
14
Values
Where full open source is not an option, a strong and well documented API can offer
interconnectivity.
Sharing & learning

On the meta level, we encourage designers and developers to share as much of their
learning, process and solutions as possible.
In a field as young and unexplored at the connected home, there is tremendous value to be
gained by pooling insights, code, and designs. Openness and sharing encourages
innovation as well as professional development and portfolio-building.
In aggregate, these open practices will create a more robust and welcoming ecosystem for
the connected home.
4. Diversity and inclusion

The home must be a safe space. That means its residents must have the ability to shape it
regardless of, and in response to, their gender, religion, language, ethnicity, origin,
professional background, socio-economic factors or age.
This is no simple feat. Connected products and the code running them manifest values as
well as biases and assumptions, as do their underlying business models.
When a connected home product is designed, the team might ask themselves questions
like:
What do our users look like?
What are their needs, requirements, desires?
What are they likely to want to use our product for?
How do we expect them to interact with our system?
How will we listen to and adapt based on what users actually go?
All of these are good starting questions, but they might not suffice. We think it is just as
important to ask:
How can we make sure this product doesn't just work well for an average user, but all
users? What do our "edge cases" look like?
How can our product degrade gracefully if it's modified, adapted, hacked? How if the
business model changes or the company goes bankrupt?
How can we make sure to design for a wide range of cultural contexts, for example with
various numbers of users, across languages, and for different expectations of privacy?
How can we make sure to allow for socio-economic inclusion?
5
15
Values
5
How can we design for gender inclusivity and equality?
One way to designing for diversity and inclusion is to have a diverse team designing the
product as well as testing it.
Including ethnographers and anthropologists also can add a lot of value.
And finally, it's worth remembering that a product made with diversity and inclusivity in mind
also means it has a much larger potential user base, so it makes sense from a business
perspective as much as an ethics perspective.
5. Security
Computer systems are notoriously easy to attack once they are connected to the internet.
The connected home is no different.
Historically, because of the relatively low distribution of smart home devices, security was a
bit of a secondary concern.
Recently, with more and more reports of hacked internet-enabled baby phones, CCTV
cameras and fridges turned into email spamming machines6, security became more of a
priority both in the products' design process and communications.
We believe that tight security is absolutely essential. Users and their data need to be
protected from any party that might try to track, spy, or hackbe it criminals, commercial
entities or governments.7
Only then the home can be the safe space that is must be.
Aggregate data needs to be anonymized in a way that really guarantees that data cannot be
traced back to identifiable users. This includes when data sets are combined with other sets,
or in future big data scenarios.
Wherever possible, data should not be saved at all. The best way to prevent abuse of data is
not to have it in the first place.
6. Sustainability
Connected home products and infrastructure are computing infrastructure. As such, they
tend to be software-driven, and their life cycles are linked to the innovation cycles of the
processor industry.
To avoid connected homes becoming the next main contributor to hazardous e-waste piles,
we need to consider sustainability:
16
Values
Modularity in design can make it easier, cheaper, and more resource-friendly to replace
broken parts and those that need upgrades.
Open source, compliance to standards, and APIs can help connected products to do
their job longer, for example when a mobile operating system emerges or falls out of
fashion.
Better, more ecologically and socially conscious sourcing of material and manufacturing
can help reduce the footprint of a product.8
Recycling and end-of-lifecycle should be considerations through the design and
manufacturing process as well as ongoing user communications.9
Makers of connected home products need to make sustainability a priority.
We believe that companies strongly embracing sustainability ultimately put themselves in a
better position in the market10both from an ecological and social perspective (our planet
and society can only handle so much more abuse) and because it makes sense for their
customers (sustainability can lead to substantial savings in the medium and long term).
1. This set of values is likely to evolve over time as we explore more cases and
scenarios. As part of the Good Home project, we drafted a list of more concrete design
values, which we include here as a further inspiration: networked, communal,
interdependent, participatory, readable, open, humble, adaptable, hackable, diverse,
resilient, respectful, sustainable, perfect imperfection, one size does not fit all, local,
careful, provocative, post-capitalistic, constrictively critical. See
thegoodhome.org/values/
2. See en.wikipedia.org/wiki/Privacy_by_design
3. See the EU's General Data Protection
Regulationen.wikipedia.org/wiki/General_Data_Protection_Regulation
4. We explore this in more depth in the article on connected literacy
5. This starts from more straightforward solutions like dropping binary gender user
profiles (male/female should not be the only option) or not asking for gendered user
profiles to begin with (consider if they are really required for the core product), and goes
into much complex questions of roles and power dynamics surrounding gender in the
larger context of domesticity. Also, obviously, marketing.
6. For some up-to-date examples, just search online for the terms "iot" and "hacked".
You'll find plenty.
7. We touch upon these issues in more depth in the article on surveillance.
8
17
Values
8. For consumer electronics, this is notoriously hard, especially at the resource level.
Fairphone has been doing an exemplary job with their sourcing and the transparency
around making a smartphone ethical and sustainable: fairphone.com
9. For a great overview of the potential of closure experiences we recommend the
excellent work of Joe Macleod available at closureexperiences.com or as an
introductory talk recorded at Interaction16: vimeo.com/159666826
10. The B Corp structure is a promising way to encode values like sustainability in
corporate governance: bcorporation.net As a prominent example of a B Corporation
that does sustainable and fair production as well as recycling and end-of-lifecycle, look
no further than Patagonia's B Corp Annual report 2014):
patagonia.com/pdf/en_US/bcorp_annual_report_2014.pdf
18
The home as the next augmentative technology
The home as the next augmentative

technology
In the past, human-to-machine interaction was modeled primarily on a single user interacting
with a single machine. Inventions like the computer mouse signaled how computers would
begin to augment our lives. Meanwhile, while computing technology changed rapidly, the
lifespans of our built environment were still long. And most use cases for the "home of the
future" were about automation, not augmentation.
Today, multiple users interact with multiple machines to form complex computational
networks. Through simplified physical interfaces, the Internet of Things can offer users better
understanding and control of data. Now the foundation of digital augmentation is in the
hands of billions of people thanks to their first IoT device: a smartphone.
Smartphones are a sign of the speed of change that our homes may soon face. As our built
environment enters the rapid life cycles of software, it brings risks such as security
vulnerabilities and broken devices. There's an urgency to address these issues nowto
ensure that the home puts humans first, respecting our privacy and other needs, while
delivering on the opportunity to computationally augment our lives.
19
Past
Past
In the past, human-to-machine interaction was modeled primarily on a single user interacting
with a single machine. The lifespans of our built environment were long. And most use cases
for the "home of the future" were about automation.
Single user, single machine

The field of studying how humans interact with computers became established in the 1980s
with the publication of The Psychology of Human-Computer Interaction.1
The groundwork had been laid already in the 1960s by pioneering researchers such as
Doug Engelbart, inventor of the computer mouse.2
Engelbart and his team radically altered people's understanding of how humans could
interact with computers. Surprisingly, they found in user tests that even though people had
never seen or used a mouse before, they immediately caught on, making fewer mistakes
and moving faster than when using other computers interfaces like the keyboard and a
pointer pen.3
The team went on to invent a number of foundational interaction technologies, the first demo
of which has become known somewhat affectionately as "The Mother of All Demos."4
20
Past
Human-computer interface with mouse. Photo: Doug Engelbart Institute

If you watch Engelbart's demo (which you should, it's both brilliant and entertaining), you will
note that he has to start pretty much with first principles:
When introducing the mouse, Engelbart doesn't just introduce a new connected object. He
also introduces a new paradigm of human-machine interaction.
That means he doesn't just need to explain how the mouse works, but also needs to
elaborate on:
what the object is
what it is for
how it makes people's lives better
For example, there's a long passage in the demo in which Engelbart describes typing up a
shopping list. To do that, he also has to set up how and why you would sort and cluster the
list, how you could copy & paste text, and why the mouse makes it easier to do so.
That's a lot of explaining for just one small, novel device.
Engelbart's demo gives us a good understanding of the challenge for today's product
designers making objects for the connected home. Not only do they have to describe what
the object is and why its useful, they are also defining an entirely new field, with new layers
of interaction and more complex systems than we've ever seen in our homes before.
21
Past
Long lifespans
Human-to-machine interaction changes quite quickly. In recent years, we've seen major
improvements to interfaces such as touch, voice and VR. Behind these innovations is the
availability of ever-increasing computing power.
Since the 1970s, computing power has effectively doubled every two years.5
Moore's Law as illustrated by "A plot of CPU transistor counts against dates of introduction;
note the logarithmic vertical scale; the line corresponds to exponential growth with transistor
count doubling every two years.6
However, while our computing environment moves at such a rapid rate, our built
environmentthat of our homes, places of work and leisure, our streets and urban
infrastructurehave historically changed at a much slower rate.7
Buildings can take years to construct and then stand for decades, if not centuries.
Renovating and retrofitting are important ways we maintain and repurpose our built
environment. These are all relatively lengthy and costly processes, as compared to updating
a digital service or personal devices.
22
Past
With that in mind, we have to consider how these different timescales and lifespans
intersect. Buildings and their infrastructure move at slower rates than computing. That
means new digital services and connected objects must play with structures that may be
hundreds of years old.
Kitchen of an Italian home. Casa Jasmina in Torino, Italy.8
23
Past
Kitchen of an Indian home. Ahmedabad, India.9
Automation in the home

There is a rich history of "homes of the future." These visions say much about the time in
which they were created. What were the power dynamics? The sense of self and of
collectives? The relationship to domestic practices? To other humans? The role of
technology? Who created this vision, who profits from it, and who is excluded?10
The technique of imaging a future home has been used extensively in advertising. Two wellknown American examples are the Monsanto House of the Future (1957) in Disneyland11
and Frigidaire's "Kitchen of the Future" featured in General Motor's Design for Dreaming
(1956) industry short film.12
24
Past
These ads showcase the companies' products and capture what these industries believed
was useful and desirable at the time.
There are notable similarities to how connected products are pitched today.
Then, just as now, these product showcases promise time-saving, ease of mind, and more
leisure. The home is viewed as a place of domestic labor and where efficiency is needed.
Therefore, much of the focus is given to automation.
Here we return to Engelbart. He noticed this emphasis on automation in human-machine
interaction, to which he responded: Augmentation not automation.13
There is great opportunity for technology to augment human intellect. However, challenges
remain as we move into a new era of human-machine interaction, where computing is
ubiquitous, where it more rapidly affects our built environment, and where corporations vie to
shape our domestic lives.
Now is the moment to go beyond automation and determine what augmentation we desire.
1. Card, Stuart K.; Thomas P. Moran; Allen Newell (July 1980). "The keystroke-level
model for user performance time with interactive systems". Communications of the
ACM 23 (7): 396410. doi:10.1145/358886.358895
2. Thanks to Scott Jensen for the pointer to Engelbart's work for this context: jenson.org
3. dougengelbart.org/firsts/mouse.html
4
25
Past
4. Watch Doug Engelbart's demo at youtube.com/playlist?
list=PL76DBC8D6718B8FD3&feature=plcp.
5. Moore's Law: en.wikipedia.org/wiki/Moore's_law
6. By WgsimonOwn work licensed under Creative Commons CC BY-SA 3.0
commons.wikimedia.org/w/index.php?curid=15193542
7. Thanks to Ame Elliot for articulating the differences in lifespans for architecture,
connected objects and digital services: simplysecure.org/blog/lessons-fromarchitecture-school-1
8. By Peter Bihr licensed under CC BY 2.0: creativecommons.org/licenses/by/2.0/
9. By Laura de Reynal licensed under CC BY-NC 2.0:
creativecommons.org/licenses/by-nc/2.0
10. Thanks to Alexandra Deschamps-Sonsino for the critique of past "homes of the
future."
11. en.wikipedia.org/wiki/Monsanto_House_of_the_Future
12. en.wikipedia.org/wiki/Design_for_Dreaming
13. dougengelbart.org/pubs/augment-3906.html
26
Present
Present
Today, multiple users interact with multiple machines to form complex computational
networks. Through simplified physical interfaces, IoT can offer users better understanding
and control of data. The foundation piece of digital augmentation is already in the hands of
billions of people thanks to their first IoT device: a smartphone.
Multiple users, multiple machines

The era of the single user interacting with a single machine is, for all intents and purposes,
over.
Today, our machines are networked with one another and enable us to interact with millions
if not billions of people. Digital services today are served through our laptops, smartphones,
transport kiosks, banking terminals and countless other forms of physical peripherals and
digital interfaces.
We participate in real-time communication and collaboration across great distances
constantly. We can view and modify complex online systems by changing our user profiles
across platforms, chatting in interwoven communication threads, and make decisions and
purchases based on the analysis of huge data sets.
We live in an era of multimodality. Thanks to our pervasive and powerful computing
environments, we communicate with each other and with machines across text, sound,
image and languages. We are increasingly multi-literate. We know how to shift between
these modes naturally and understand the affordances and limitations of each.
Through these interactions, we contribute to the creation of enormous amounts of data.
There is a lot opportunity in the volume, variety and velocity of processing all of this data.
When we think about the connected home, we see it as an integral part of these complex
computational systems. Our interactions can call upon and contribute to these massive data
sets, if we make them in a way that is readable and writable to usand if we build interfaces
to help them augment our lives in meaningful ways.
Simple interfaces to complex systems
27
Present
When these complex systems are visible and modifiable for everyday users, we learn about
about how they work. This visibility can lead us to knowingly change our own behavior to get
the outcomes we want. Rather than building vast surveillance machines, let's employ this to
build a learning cycle as a tool of user empowerment.
The Toyota Prius' dashboard is a useful example of a simple, actionable display of complex
data. It shows how the car uses and stores energy. Most users would be unfamiliar with how
electric car engines worked, let alone how their driving behavior affects energy consumption.
However, by reading this simplified real-time information, people can observe how their
behavior changes the system and adjust their driving accordingly.1
Toyota Prius dashboard. By It's Our City, licensed CC BY 2.0.2

Now looking beyond digital displays, the Internet of Things has the potential to not only offer
simple visual interfaces, but to let us move away from screens and into physical interfaces.
Connected objects can enable user empowerment and feedback loops.
As we continue to explore this field, we will consider how users might observe complex
systems and make modifications. We'll look at the affordances of physical objects, such as
touch, sound, movement and light as more natural "human-readable" interactions. And we'll
28
Present
examine how IoT might extend or replace the digital displays we have around us, further
augmenting our computational abilities while maintaining the users' understanding and
control of the system.
The smartphone is an entry point to

augmentation
The foundation of this kind of augmentation is already in the hands of billions of people. We
tend to think of it as a communications device3 or portable computer, but it is also most
people's entry point to the world of IoT: their smartphone.
Smartphones sense the environment around them: location, acceleration, sound and more.
They also connect to the internet, and therefore tap into large, computational networks.
Because of the smartphone's inherent connectivity, its computational power and its
pervasiveness, we're increasingly seeing appliances and peripherals that connect to the
phone. Think wearables like fitness trackers or more advanced tools like virtual reality
equipment or even medical devices like blood pressure monitors. Many of the important
players in the connected home field are manufacturers of smartphones and/or provide
operating systems for them.
Therefore, as connected homes devices become more prevalent, they are built frequently
with the smartphone as primary interface in mind.
The smartphone extends our data selves. We carry it constantly on ourselves. It's the first
true, widespread digital augmentation device.
And yet, it's not the most elegant interface going forward: It can feel like a bit of a kludge. IoT
has the potential to move us beyond this black mirror and enable the next level of
augmentation.
1. Thanks to Ame Elliot for the example of the Toyota Prius teaching users about
energy consumption and how that changed their driving behavior:
simplysecure.org/blog/lessons-from-architecture-school-3
2. Toyota Prius dashboard. By It's Our City: flickr.com/photos/its_our_city/2838668732,
licensed CC BY 2.0: creativecommons.org/licenses/by/2.0.
3
29
Present
3. Depending on how you slice and dice the data, current estimates of global
smartphone usage range somewhere between 2bn estimated by statista
(statista.com/statistics/330695/number-of-smartphone-users-worldwide/) and 2.4bn by
GSMA (gsma.com/mobilefordevelopment/programmes/connected-society) smart phone
users globally, with projections of around 6bn by 2020 from Ericsson
(ericsson.com/ericsson-mobility-report).
30
Future
Future
Smartphones are a sign of the speed of change that our homes may soon face. These rapid
lifecycles bring risks, such as security vulnerabilities and broken devices. There's an
urgency to address these issues now, as well as innovation opportunities, to ensure
throughout that the home remains a place that puts humans first.
The speed of change

In 2007, Apple released the first iPhone. Shortly thereafter in 2008, Google released its first
Android device. At the time of writing, those milestones are just shy of a decade ago.
These touchscreen smartphones represent a threshold in the evolution and adoption of the
phone. In the years prior, there had been a range of exploratory devices, such as Nokia's
first clamshell phone and PDAs running PalmOS that one could interact with using a stylus.
Since the advent of the iPhone and Android, we've seen entire industries and social
practices boom. App stores, accessories, the rise of mobile photography, messaging,
payment and more.
31
Future
HTC Dream mobile phone with AZERTY keyboard for French market. By Akela NDE, CC
BY-SA 3.01
If anything, the last decade shows us how quickly new technologies can be picked up and
become mundane in their ubiquity. We anticipate a similar trajectory for the connected
home.
While today's connected objects can be clunky and exploratory, there will be some iconic
releases or "must have" devices that change how people see this space: The iPhone
moment of the connected home.
After this threshold, there will be electronics and connectivity in many of our everyday
objects. At this stage, the speed of change of our built environment will begin to resemble
that of our computing environment. The lifecycle of things in our homes will be shortened
dramatically.
Software will need to be patched. Devices will be rendered incompatible and obsolete. Entire
services will rise and fall in months.
Our homes will move at the speed of our smartphonesor faster. The era of the "disrupted
home" is coming.
32
Future
The risks
With this speed of innovation comes volatility and unpredictability. As a result, we will likely
see an increase in waste and exploitation. There will be vulnerabilities and security
breaches. If history teaches us anything, it is to expect these patterns, as we've seen them
with many technological advances before.
With that knowledge, we can approach the connected home with some understanding of
what is to come.
For example, let's look at security. As our built environment begins to change at the speed of
software, our homes may experience new kinds of security vulnerabilities.
A smart lock system is installed in your front door. The company who makes the locks goes
bankrupt. There is no longer a support service to maintain your locks. At some point, your
locks might get jammed, or hacked, or become incompatible with your smartphone's latest
update. You can no longer patch security bugs in your smart lock. And the lock on your front
door is no longer in your control.
These scenarios abound2 and serve as a reminder that how a product ends its lifecycle is
just as important as how it begins.3 Considerations such as security must be addressed not
only at the moment of purchase, but in anticipation for how the object will be used for years
to come.
From installment, to use and maintenance, to upgrading, to uninstalling and dismantling,
technology must be in service to humans and put our concerns and needs first.
Privacy as an opportunity
There aren't many connected homes today. Similar to smartphone proliferation, it might take
a decade for products to go from invention to widespread adoption.
Exceptions may be in some cities, such as in South Korea, where large-scale network
infrastructure is being installed directly into the new buildings. In other places, retro-fitting is
a more common path to connectedness, and that will likely move at a slower pace.
For a glimpse of what the connected home may be like, we can examine the trajectories of
other connected environments, such as retail spaces, office buildings, and hotels.
Already our behavior in these spaces is being tracked. As we move through the city, into
stores and offices, in transit and on our devices, many systems are recording what we look
at, what we click on and what we do.
33
Future
Due to the generally temporary and transactional nature of our interactions in those spaces,
we tend to use them with less emotional investment and critical thinking than our homes. For
example, in a high street store, we may be more accepting of privacy-diminishing
interactions4 if they come with financial gain or if they seem, frankly, unavoidable.
Mannequiner i en tjbutik i Canada. By Colin Rose, CC BY 2.05

Similarly, on the web, advertising companies are particularly aggressive in how they track
us.6 Through consumer education and protection as well as browser tools like ad blockers
and Do Not Track, we can mitigate these tracking measures. However, it is basically an
arms race with online advertising. Our privacy is increasingly eroded, and our web
experiences slowed down as ad companies try to ensure we are humans and serve us
tailored ads.
34
Future
We are seeing parallels in online tracking and behavior tracking in the physical world. As this
trend continues, we need to ensure we can make informed decisions whether to enter these
spaces and what information we consent to giving. Connected spaces, similar to websites,
should communicate what data they are collecting, how it's being used, and how to opt out.
Going forward, the tracking dilemmas we're facing in streets, stores and in our online lives
will confront us in the home.
For anyone making connected products today, there is a huge opportunity to build in better
privacy and control of personal data. Companies can innovate by offering higher default
settings, better controls over how personal data is collected and shared, as well as
assurances when data is deleted.
These privacy tools will be features that people increasingly seek out, as the prevailing
practices of tracking in our online lives and in public spaces encroaches into our home.
The home is for humans

There's an urgency to figure out these issues. A lot is at stake, from our personal data, to
safety and the health of our relationships and interactions at home.
As with the advent of smartphones, we will need to revisit vocabularies,7 literacies, and
ways to negotiate preferences and resolve digital conflicts in the connected home.
The future is one that promises augmentation. But the only kind of connection and
augmentation we want is one that we can fully read, control and participate in, and then,
whenever we want, turn off.
We will continue to explore concepts and scenarios that bring these risks and opportunities
to life. And throughout, we will be guided by the understanding that above all else, no matter
what technology can, the home is for the people living in it.
1. By Akela NDE: commons.wikimedia.org/wiki/User:Akela_NDE, Own work. Licensed
CC BY-SA 3.0: creativecommons.org/licenses/by-sa/3.0/, available at
commons.wikimedia.org/w/index.php?curid=6680413.
2. For a recent example, learn more about how Alphabet company Nest killswitched
one of its home automation products, Revolv, and explained to customers how the
warranty had expired: boingboing.net/2016/04/05/google-reaches-into-customers.html
3. For more thinking about how products and services should end, see Joe Macleod's
Closure Experiences: closureexperiences.com
4
35
Future
4. EyeSee mannequins, for example, recognize retail customers' faces and record their
movements in a store. theverge.com/2012/11/20/3670222/almax-eyesee-dummy-retailfacial-recognition-privacy
5. By Colin Rose: flickr.com/photos/73416633@N00 Licensed CC BY 2.0:
creativecommons.org/licenses/by/2.0, available at
commons.wikimedia.org/w/index.php?curid=3214087.
6. The documentary "Do Not Track" by Brett Gaylor explains the perils of tracking and
practical measures to reducing it: donottrack-doc.com
7. There is a growing number of apps and services to offer unified interfaces. San
Francisco-based startup Thington (thington.com) offers a simple concierge-like
interface that aims to bring all smart home devices into one app based on their APIs.
It's an easy-to-use approach with intelligent, well-designed interactions. Other
approaches that are even more explicitly conversational include the smart home hubs
Amazon Echo and Google Home, both of which bring voice control into the home (and
fold their respective mother companies' services right in).
36
Concepts
Concepts
Interactions in the connected home happen at two layers: human-readable and machinereadable. By "readable" we mean a human or machine can easily understand, modify and
execute actions.
To engage in the human-readable layer, we may need to learn certain skills and
competencies so that we can be empowered in connected environments and able to read,
write and participate fully in them.
For machine-readable interactions, connected objects are equipped with the technical ability
to sense, think, and act. We see different categories of connected objects appearing in the
home.
The human-readable and machine-readable layers require mediation. That means we'll
need tools and protocols that translate between humans and machines, as well as
mechanisms to negotiate and resolve conflicts in and between these systems.
Looking at one area of conflict more closely, we see how controlling personal data is
complex, contextual, cultural, and fuzzy. We need to develop both human-readable tools,
such as vocabulary and metaphors, as well as machine-readable ones, like metadata, so
that humans can control their privacy across these layers of interaction.
37
Interaction layers
Interaction layers
Interactions in the connected home happen at two layers: human-readable and machinereadable. By "readable" we mean a human or machine can easily understand, modify and
execute actions. With the rise of more complex computational systems, new layers of
interaction may emerge.
Introducing interaction layers

Borrowing from the three-tiered Creative Commons licensing structure,1 we'd like to explore
how interactions in the home can take place at different layers:
human-readable
machine-readable
new layers, i.e. environment-readable
Each of these layers require ways for humans, or machines respectively, to understand and
modify what's going on. There's also a need for interpreting across these layers, so that
humans and machines can negotiate preferences and commands as well as resolve
conflicts that arise.
Understanding "readability"
We use the term "readability" as shorthand for the broader ability to read, write and
participate fully. That encompasses the ability to view the source of an interaction, to
comprehend what's going on, to modify and execute it.
Human-readable layer
The human-readable layer contains interactions that humans can read and modify.
The purpose of this layer is for humans, regardless of their technical proficiency, to feel
confident in understanding what a connected object or service can do, and where desired,
modify or opt out of it.
As much as possible, this layer should convey things in simple iconography or text, or in
ways that are intuitive and build on behaviors that humans are already familiar and
comfortable with. Accessible control over machines is essential for people to live confidently
38
Interaction layers
and with dignity in their home.

The human-readable layer is how humans can readily tailor digital services and objects to
their own needs. Through this power, people can invent new and unanticipated things,
inspired by their local contexts and use cases. In this way, readability provides an important
path to both user empowerment and grassroots innovation.
Designing the human-readble layer

As the home becomes more connected, we have to ensure it remains human-readable.
Here are some considerations as we design and implement this layer:
1. How can connected objects or "invisible" digital services communicate what they
request users to do, without being intrusive or disempowering?
2. How do connected environments convey their level of connectivity as well as
mechanisms for opting out or for better controlling participation?
3. How can humans be guided and advised for their protection, for example against the
installation of malicious software or a device violating their tracking preferences?
4. How can a human's digital preferences, such as preferred temperature or Do Not Track
settings, travel with them and be controlled by them depending on the context?
5. How do these interactions change depending on the context, such as being someone's
house guest or having a roommate?
A blunt approach to human-readable interactions. A set of icons that indicate what is

collecting data, what is transmitting data, what is analyzing data. "Does this thing listen or
watch? Does it share data to the cloud?"2
Machine-readable layer
As connected devices enter the home, it is beneficial if the machines can talk to each other
to some degree. That means there will need to be a layer of interpretation and interaction
that is optimized for machines (such as APIs, etc.).
39
Interaction layers
Typically, this kind of layer is hard for a human to read. Nevertheless, by organizing and
structuring data following certain conventions,3 machines can read it effectively and that
information can then be translated into a more human-readable form.
There is a lot of technical work to be done designing this layer well. Through decades of
developing network technology and international standards, there are established tactics for
building interoperability among machines and their data.
If done correctly, the machine-readable layer can be incredibly powerful. However, since it is
hard for most humans to read it directly, we need to ensure that there are ways to trust and
verify what is happening at the machine-to-machine level even for non-experts. That way
devices can honor what the user wants and cooperates well with other devices, datasets
and systems.
New layers?
Possibly, given the complexity and ubiquity of connectivity ahead, there will be new
readability layers to consider.
For example, as embedded systems fill a space with sensors, computation and coordination,
does the resulting emergent system require the ability to read and modify the environment
as a completely different level?
Or do certain social relationships, such as neighborhoods or cities, gain the ability to read
and write collectively?
We're not sure what those new layers might be, but regardless there remains the need to
have human-readable and machine-readable layers so that humans and machines
respectively can understand the systems and engage in interaction.
1. Creative Commons licesening layers: creativecommons.org/licenses
2. Images from the Noun Project. Eye by Thomas Le Bas (thenounproject.com/search/?
q=eye&i=6186), Ear by Sren Michelsen (thenounproject.com/search/?q=ear&i=6200),
and Cloud by Aaron K. Kim: (thenounproject.com/search/?q=wifi&i=123908).
3. en.wikipedia.org/wiki/Data_model
40
Connected objects
Connected objects
A connected object is one that senses, thinks, and acts. The home has categories of
connected objects that tend not to be in other spaces, such as white goods and personal
memorabilia.
What is a connected object

Depending on the field, the terms smart and connected are used largely interchangeably.
For the purposes of this book, we primarily use the term connected.
A connected object is one that:
senses (i.e. takes input from the environment or a user)
thinks (i.e. computes and analyzes data, often on the cloud)
and then acts (i.e. makes something happen)
A connected object also has a physical component. It can interact with the physical world
through its input or output, or both. A connected object is likely networked to other connected
objects or digital services via the internet or local networks.
Sensing
An object that gathers information about its environment is sensing. Examples include
measuring temperature, sound, light, or movement. It can also mean receiving input from a
user, such as a button being pushed or a switch turned on.
Thinking
Thinking is the process of analyzing the input. Perhaps the incoming data is added to a
larger data set, or some computation is run using it. For connected objects, thinking
increasingly tends to happen "in the cloud", instead of locally on the device itself. This
increases the computational potential of the object, since cloud services are able to process
much more data than small, local hardware.
Acting
41
Connected objects
As a result of thinking, the object acts. It performs some kind of output, such as switching off
a light, turning down the temperature, unlocking a door, or sending a notification to a phone.
These actions can be physical or digital, visible or invisible.
Often, but not necessarily, these three behaviors (sensing, thinking, acting) happen in
chronological order. In larger systems, several of these processes might happen in parallel
or in ongoing loops. The interactions can be quite complex. Nevertheless, the basic pattern
holds.
Categories of connected objects

The home contains objects of various categories: furniture, appliances, devices, personal
memorabilia, food and more.
Here we outline several categories to see if they yield any interesting insights or questions.
Connected vs. disconnected

One simple distinction of objects is connected or disconnected.
It is binarysimply on/off. If we look at the last 10 years, this held up nicely. Lamp on, phone
off, refrigerator running.
Over the last few years, the lines have started to blur. Is the smart TV on or off, if its always
listening in even when the screen isn't on? As more things come online, and their
connectivity becomes more complex, the binary distinction might become more of a
spectrum.
Active vs. passive

Which objects in our home actively listen, sense, interpret, process, act on what's going on
in the room? Which ones sit there until we intentionally trigger them? How will we distinguish
these modes at all?
For example, an Amazon Echo listens actively for commands, as does your Android phone if
you set it up to react to the trigger word "OK Google".
However, the trigger word is primarily processed locally. Once uttered, the devices fires up a
connection to a server farm and gets ready. Is this active or passive listening?
What about a geofence that triggers an action once we approach it, such as when we enter
our home with our phone? That system doesn't listen for audio cues, but it does track our
behavior and switches from kind of passive to fully active.
42
Connected objects
What about a motion sensor that is connected to the internet? What about a learning
algorithm that doesn't act in a visible way at all, yet nevertheless learns about and adapts to
our behaviors?
The categories of active vs. passive data gathering objects are ripe with questions.
Consenting vs. refusing

User consent is complex, and it can depend on the kind of sensing happening in the home.
For example, carbon monoxide detectors might be perceived as non-invasive, whereas
someone may want to opt-out of a microphone or camera monitoring them.
How can a user understand and give informed consent for a microphone in a friend's home
that listens to their every word? Today, many devices already have actively listening
microphones in smartphones, laptops, game consoles, and TVs.
What's more, the number of actively listening objects looks like it will take a sharp upturn
now. Personal assistants like Amazon Echo and Google Home, or voice controlled
appliances like ovens, are on the rise. If it's voice-controllable, it's likely to include at least
some level of active listening. What does user consent look like here?
We see these questions around voice control as an important frontier for these questions of
consent vs. opting out.
Will every object sense?

Device makers are betting that consumers will flock to smart home hubs and other voicecontrolled appliances. Just about every major tech company or home appliance
manufacturer has some sort of smart play in the market.
How much so? At CES 2016, Samsung president BK Yoon announced that all Samsung
televisions will be IoT devices by 2017, and within five years all Samsung hardware will be
IoT-ready.1
Voice will likely be a major interface for these devices. It means that the home will soon have
a lot of microphones. There will also be many more sensors, meaning your home will be
listening to all sorts of inputs.
Adding computation and network capabilities to consumer and home devices is getting so
cheap that it hardly makes sense for a company not to put it in, even if the value isn't clear
yet or they don't know what to do with all the data that will be collected.
43
Connected objects
Here we outline a few further categories of objects in the home and how they might change
with connectivity:
Routers
First, routers might turn out to be the hub that controls all our smart home infrastructure. It's
not clear how it's going to play out, but it's a strong scenario. For example, Google's wifi
router On Hub2 comes with all the protocols equipped, plus microphones and speakers,
even though Google has a dedicated smart home hub in the market (Google Home).
Routers are already at the core of home connectivity, but also notoriously tricky to configure
and maintain. They might just be the least beloved of all tech objects in anyone's home. Do
we want these as hubs?
White goods
Second, white goods such as dishwashers, fridges, washing machines, and ovens have
been the connected fever dream of manufactures for years. (The internet-connected fridge
has become a running joke by itself.)3 There might be something there. Assume for a
second that a scenario where we have a home server in every house; a local cloud solution
of sorts. It's a long shot, but the fridge might not be the worst place to house this device, or
to double as a hub if integrated smartly.
We believe it's more likely that appliances will end up with minimum connectivity but with
voice-controls. A hands-free scenario is useful for cooking, whereas a fridge automatically
ordering groceries seems somewhat invasive.
This is purely hypothetical. We don't have much data on user acceptance. All it takes to
change the perception of a service or product is for one to get it just right. It would be an
"iPhone moment" for connected appliances that doesn't just improve an oven but redraws
the meaning and boundaries of the category entirely.
Home infrastructure
Third is infrastructure. What happens when existing infrastructure such as water pipes,
heating, electricity and the like get connected?
We see the first steps with smart meters that measure electricity consumption and make it
more transparent and actionable. There's still lots of previously dumb infrastructure to
explore.
44
Connected objects
But also, what about the infrastructure in a home we usually don't associate with
connectivity? Floor boards, wall paper, blinds? It seems to early to tell, but there might be
unexpected sensing, thinking and acting coming soon to our home infrastructure.
New kinds of objects

The fourth category is truly new stuff. The unknown unknowns we can only speculate
about. This is the category that's most sketchywith the most potential while also likely to
cause major friction.
We'll be watching with interest over the next five to twenty-five years as more connected
objects, from fridges to kitchen tables, come online.
1
. See AV Interaction (Jan 2016): avinteractive.com/news/samsung-announces-plansto-make-all-tvs-iot-devices-by-2017-06-01-2016/
2. on.google.com/hub/
3. fuckyeahinternetfridge.tumblr.com
45
Connected literacy
Connected literacy
To be empowered in the connected home, we will need to learn certain skills and
competencies so that we can fully read, write and participate in these technologies.
Building on web literacy

Over the last few years, there has been a lot of research and advocacy around the skills
required to be literate on the internet. Notably, this web literacy is not just about "learning to
code," but rather how to become a critical consumer, contributor and participant online.
As a champion of user empowerment on the web, the Mozilla Foundation developed the
Web Literacy Map to better define these skills and create curricula and programs to foster
them.
46
Connected literacy
1
Mozilla's Web Literacy Map
The Web Literacy Map contains three main elements:
Reading on the web is a critical skill for engaging content online. They can be viewed
as exploring, or navigating the web. Just as traditional reading requires knowledge of
the text and concepts of print, reading online requires a basic understanding of web
mechanics.
Writing on the web enables one to build and create content to make meaning. New
genres that blend texts and tools have emerged on the open web and are often referred
to as making. Learning through making involves constructing new content.
Participating on the web includes connecting with the communities that share, build,
and sustain meaningful content online. A healthy online community requires knowledge
of how to create, publish and link content, and an understanding of security in order to
keep content, identity, and systems safe.
47
Connected literacy
Today as connectivity enters everyday objects and our physical environment in new ways,
we'll examine how web literacy can offer a framework for understanding the skills and
competencies needed to be empowered in the connected home and generally with IoT.
Some of these will have a direct counterpart for the home, some might not apply at all,
others yet are likely to at least lead to better questions. Let's go through them one at a time.
Reading the connected home

The reading strand of web literacy is about navigation, web mechanics, search, credibility
and security.
Here we look at those elements in more detail to see if they yield interesting questions or
areas of research for the home:
In a connected environment, it will be important to recognize common visual cues in
digital services and connected objects. Maybe there will standardized symbols or
indicators on the objects in your home that explain what's happening to your data,
what's expected of you, and how to turn the objects on and off.
A user will want to read, evaluate and manipulate the data in their home. This entails an
understanding of the role algorithms play in creating and managing content, and even
creating and modifying those algorithms to suit local needs.
Search will take on different nuances in the connected home. How will you find real-time
or time-sensitive information about and in your home? How will you discover information
and resources to help you navigate your home, to fix things and solve problems
effectively? Perhaps search will even encompass datasets from the neighborhood or
city.
No doubt users will have to compare and contrast information from a number of
sources. Maybe a smart thermostat gives one reading while the connected windows tell
you another. How will you read that information, compare sources and make informed
judgments? How will you know which manufacturers or service providers to trust? How
will you investigate those sources further given the affordances of connected physical
objects?
What are the ways to protect yourself from scams and phishing in your connected
home? Where will you manage your accounts and logins securely, not to mention
knowing how to encrypt your objects properly?
Lastly, how will you be able to access your objects preferences and settings to ensure
you have the level of security you desire?
Writing the connected home
48
Connected literacy
The "writing" strand of web literacy is about composing, remixing, designing,

coding/scripting, and accessibility.
Again, we'll look at each element for some potential insights:
How will you identify content and data that you can modify? What will be the techniques
and conditions needed to combine this data to create something new? How will you
properly cite and reference original content or other data sets and services?2
What options will you have to change the style and layout of your connected
environment? What mechanisms can you use to improve your user experience through
feedback and iteration? What will you be able to make and generate that can be used in
an environment-agnostic way, say for example when you move between homes or into
other settings?
In the connected home, APIs will be quite important. How will you query APIs and how
will you build on top of them? How will you learn about and apply scripting frameworks
effectively?
There will also be repercussions in design. How will you evaluate different interfaces
and understand what will best suit you or others you are caring for? How do you
improve accessibility for your loved ones through the usage and maintenance of your
home?
Participating in the connected home

The "participating" strand of web literacy is about sharing, collaborating, participation,
privacy and open practices.
Taking a closer look:
In your home, how will you create and use a system to distribute content to others? How
can you contribute to and find content for the benefit of others? What techniques will we
use to elicit peer feedback, as well as understand our desired audiences to make
relevant contributions ourselves? Importantly, how will you identify when it is safe to
contribute content?
Collaboration in your home is also interesting. This could be doing a chore together, or
doing something together for entertainment, etc. How will you choose what platforms
and tools to use for the particular collaboration at hand? How will you co-create?
What will be the notifications that you'll configure to keep up-to-date with your
collaborators, or with the communities you care about? How will you develop and
articulate shared expectations and outcomes? Or work towards a shared goal using
synchronous and asynchronous tools?
These skills tie directly in with privacy as well, including knowing how to advocate for
privacy as a value and a right in the networked world.
49
Connected literacy
To be literate in connected spaces, you'll want to explain the ways in which unsolicited
third parties can track you across devices and environments. How will you control
metadata shared online, or identify the rights retained and removed in user
agreements? Also, there will definitely be a need to manage online identities.
Perhaps essential in all of these skills in an understanding of how and why to advocate
for healthy ecosystems and things that work for you, on your terms.
As connected homes come online, these might be some of the skills and competencies
required to be empowered in these new environments. These questions might also provide
interesting avenues of design and technical research, as well as considerations for future
lesson plans and courses.
1. See Mozilla Web Literacy Map: learning.mozilla.org/web-literacy
2. As an example, what would the reference look like for a digital artwork in a
connected picture frame like the Electric Object?
50

As we invite more algorithmic agents into our lives, we'll see more conflicts both between
machines and humans, as well as among machines. How can we understand and mediate
these conflicts?
How can interactions and understanding be mediated across the human-readable and
machine-readable layers?
Here we discuss the idea of a "user agent" that translates and negotiates across the layers.
Historically, this has been metaphor-based software (GUIs) like the browser or mobile
operating system. It might look different going forward.
As we invite more algorithmic agents into our lives, we'll see more clashes between
them and, to a lesser degree, us.
For now, humans will be the ones who have to mediate these conflicts. In the future,
machines may take on more negotiation.
Figuring this out is a clear challenge for interaction designers.
Where conflicts arise

Whenever we enter or leave a connected environment, some type of data engagement will
take place. This could be an opt-in or opt-out or something more granular. It could be an allor-nothing, binary choice like a EULA, or something negotiated between humans or software
agents.
We need to find ways to make this type of interaction and exchange understandable,
human-readable, and negotiable. It seems better to discuss today how to get this right, or at
least how to ask the right questions that will arise tomorrow.
The other aspect is the negotiating of agreements on a (semi-)automatic level. Even more
than on the web, in the physical space we need to be in control of our privacy.
For example, CCTV cameras in the public space are somewhat controversial because they
do not reliably prevent crimes while increasing the level of surveillance; Wifi sniffing trash
cans crossed the line.1 This is a case-by-case negotiation of what's acceptable in the public
environment.
In homesboth permanent and temporarystricter rules should apply. Privacy is key, and
strong privacy protection must be the default. We need opt-in, not opt-out, for the sharing of
personal information.
51
So what could that look like? What's the interface for negotiating these questions? What are
the social norms going to be? What kind of recourse might there be for cases of abuse?
We cannot yet answer these questions but can make some educated guesses:
Social norms might solve many of these questions before they even become
problematic. Most home owners might turn down "sniffing" levels to the lowest default,
while some early adopters turn it up. In both cases, their friends and peer group are
likely to self-select, exactly like it happens today. Some people use CCTV cameras on
their premises, others don't. Some have guns in their homes, others don't. Some play
loud music, others don't. No technological negotiations are at work here. Social
interactions, legal regulation and cultural norms are used to resolve these things.
Design principles that honor privacy will win the market. It makes sense to honor
privacy in the home and to make it an overwhelmingly strong default. The market might
simply solve this as users vote with their money. Different regions/markets might
produce different outcomes, as these preferences differ across countries.
Smart contracts could provide a technological backend. Unlike in software and web
services where users are routinely forced to accept End User License Agreements
(EULAs), there is a window of opportunity to create a better system for IoT in general
and the connected home in particular. Each person could store their personal
agreements on a digital ID that negotiates a deal with the connected environment. Say
Alice the visitor is fine to be sniffed for wifi devices, refuses to be captured by CCTV,
and doesn't have a strong preference around mood lighting. Bob the home owner has
similarly expressed his preferences. A relatively simple algorithm could match them up
so that the camera stops recording, while Bob's mood lighting preferences override
Alice's (since she doesn't care anyway) and the system does connect to Alice's wifi
devices because she agreed to it.2 For heightened security needs, the blockchain could
be used to verify these negotiations. This might seem like overkill now, but might turn
out to be relatively seamless and offer smooth sailing.
We need a Do Not Track for the physical space. As a lowest-common denominator,
we might need a Do Not Track3 for the physical world. Not just for connected homes,
but especially for smart cities and connected retails spaces. This could be a device or
service that allows us to reliably opt-out of marketing & advertising tracking as it enters
physical spaces.
These rules and types of interactions need to be human and machine readable. For any of
this to work, we need to come up with a way to communicate these rules in a format that is
legible by humans and machines alike, like Creative Commons licenses.4
Managing Conflict
52
When we introduce connectedness into infrastructure like buildingsinto our homeswe

stitch a technological network into our lives. And with it we introduce smart agents of sorts.
Software that has more or less its own goals and agendas.
For example, a Nest thermostat's primary goal might be to achieve and maintain a certain
temperature in the living room; a secondary goal might be to save energy.
The owner of the Nest has given that objective to the thermostat. And while it will undergo
some interpretation at the hand of the algorithm. Say you express you prefer a desire for the
temperature to be 19 Celsius and the algorithm knows to translate this statement into "you
want 19 Celsius in your living room when you are at home but while you're gone
temperature can vary to lower energy consumption". The algorithm's goals are set directly or
indirectly by the user.
The user this example is a single human being. It's important to stress this as these kinds of
interaction models tend to break down, or at least be challenged, along three axes once we
do not talk about single-user scenarios:
user-to-user conflicts
user-to-agent conflicts
agent-to-agent conflicts
User-to-user conflicts
In a multi-person household, the technology could easily be faced by conflicting goals. Think
a couple with different temperature preferences. These are things that technology won't be
able to solve elegantly. They are social challenges that require social solutions, not
technological ones.
User-to-agent conflicts
If the multi-user scenario challenge wasn't hard enough, think of example of friends
mentioned recently, in which only one partner uses the thermostat's app and the other
doesn't want to engage with it. In the latter case, thermostat doesn't recognize that person
as a user.
If the app user isn't at home, but the non-app user is, the thermostat reads the home as
empty. It doesn't turn on by itself but requires human intervention.
This is clearly a technological problem, and one that should have never appeared to begin
with. Technology, especially in the home, needs to be sensible about its demands on the
residents.
53
Agent-to-agent conflicts
When we introduce smart appliances to our home, we deal with software agents.
Individually, in isolation, these might do a good, or good enough, or even stellar job. But
combined into a complex and often unpredictable network, it can all get a little messy. Here
we need a systemic view. This is seriously challenging terrain for designers.
Good friend and excellent systems thinker/designer Louisa Heinrich articulated this perfectly
with an example of different appliances fighting over the blinds being open or closed based
on their respective goals: The coffee machine wants them closed so the milk keeps longer,
dishwasher wants them open to grab some solar power, the Nest wants them closed to keep
the temperature low, the plants want them open...
Who moderates these conflicts? What happens when youthe masterare not at home?5
In Louisa's words, "There is a narrow but very deep gap between assistance and
6
replacement."
When things clash

We still need a bit of a framework to interrogate and explore these conflicts between the
smart agents in our environment.
Enter Scott Smith, who at ThingsCon 2015 shared his excellent research project,
Thingclash. It's a framework for considering cross-impacts and implications of colliding
technologies, systems, cultures and values around the IoT:7
54
As Scott points out starting around the 27 min mark, it's crucial to think about the legacy of
UX decisions, business models, etc., to explore the different cross-impacts, the implications.
"Can we find a way to surface and make legible the tensions and the frictions and the
conflicts that arise when new connected, data-collecting objects are introduced into our
world?"
For now, just assume that putting several smart agents/appliances to work side by side
might yield unintended consequences. These might not be devastatingmost likely they're
simply annoying, like lots of notifications on your phone or shutters that open and close
seemingly at random. But as more things are connected, the clashes could become more
dire.
Who is the user agent?

The question now is that if you fill an environment with humans and connected objects, is
there something that helps the humans navigate everything and to facilitate between
humans and machines?
We'd argue there is a need for a tool of interpretation that can advocate on the user's behalf.
This might be software or some device that that articulates the users' interests, ensures they
are respected by the machines in the environment, and also communicates back what the
machines are saying and doing.
This tool could function like the web browser does today when surfing online. As a user
moves through connected environments, the brower helps interpret or render that
environment for the user. It is also a protective program, sometimes taking action on the
user's behalf, such as blocking malware or spam.
This interpretation tool can also communicate preferences to devices and connected
environment. It might also be where conflicts among devices and preferences are brought to
the attention of the user, who can parse and override certain decisions. This could also be
the interface for the user to express themselves, either to machines in the environment or to
other users.
Given the above needs, the smartphone seems to be a natural place to start exploring what
a user agent in the connected home could look like. In due time, other interfaces will surely
develop. Until that is the case, the challenge for designers is not to increase even more the
smartphone's ongoing attention grabbing, this time on behalf of our apartments.
1. For now. Wifi sniffing smart rubbish bins were installed, and then abandoned, in
London a few years ago. cnet.com/news/london-tosses-out-wi-fi-sniffing-smart-bins
2
55

2. Read more about what smart contracts could look like in practice in our Controlling
Privacy chapter.
3. See
[en.wikipedia.org/wiki/Do_Not_Track(https://en.wikipedia.org/wiki/Do_Not_Track)]
4. Read more about human-readable and machine-readable layers in our Interaction
layers chapter.
5. Watch Louisa Heinrich's talk at NEXT14, especially around the 7 minute mark:
video.nextconf.eu/v.ihtml?photo%5fid=9802229.
6. See louisaheinrich.com/2015/03/23/good-help-is-hard-to-find-post-26100
7. Check out Scott Smith's research project thingclash.com and his personal website
changeist.com. The video of his talk is available at youtube.com/embed/CjcJfGrWKNA.
56
Controlling privacy
Controlling privacy
Privacy is complex, contextual, cultural, fuzzy, and we need to develop the vocabulary and
tools to put people in control of their privacy.
Privacy is complex, contextual, cultural, fuzzy. It is hard to get right, especially in the context
of connected things. Really hard.
Our vocabulary has not evolved as quickly as technological and societal change has. We
are essentially stuckfor nowwith crutches, metaphorically speaking.
We try to describe and solve problems of connected privacy with terminology and metaphors
from non-connected privacy. This is as tricky and often misleading as trying to depict a fourdimensional figure in three-dimensional space. It's possible to a degree, but we simply are
not, as a society, fluent in doing it.
Yet controlling your privacy and making informed decisions about the factors that impact it
are absolutely essential going forward. How can anyone decide which
product/service/network/company to invite into that unique safe space of their home?
It might be helpful to think about privacy-related choices as consent rather than just
preferences.
We need better privacy user experiences & a new

vocabulary
We see two things we need to address the issue of navigating privacy better:
1. In the medium term, we need to evolve a new vocabularyterminology and metaphors
to help us think about the complex arising issues that make up privacy in the 21st
century.
2. In the short term, we need to design privacy-related interactions to be much, much
easier. Controlling your privacy should be as intuitive (or at least simple) as possible.
Even if simplifying means losing granularity, that would be better than the alternative of
making ill-informed decisions.
Dimmers and other metaphors

One exploration of simplified privacy interfaces was a direct result of our work on the first
version of this book: The Privacy Dimmer1.
57
Controlling privacy
The Privacy Dimmer at The Good Home. Milan, April 2016.
The Privacy Dimmer at The Good Home. Milan, April 2016.

For the Milan installment of The Good Home project 2, we created a series of physical
explorations. The exhibition consisted of a number of speculative designs and conceptual
prototypes. Two of them are directly relevant to how we could think about privacy and UX.
58
Controlling privacy
Privacy is a spectrum
A light dimmer allows you to control the lighting of a room gradually. In the same way, a
Privacy Dimmer would give more gradual control over the amount of sensing/smartness/data
processing going on in the home at any given time. Privacy is a spectrum, and it is highly
contextual.
59
Controlling privacy
60
Controlling privacy
Privacy is a spectrum. We need to explore how to best express and refer to this spectrum.
The Privacy Dimmer

The Privacy Dimmer would accommodate this gradual dimming by mapping the various
systems and subsystems (sensors, analytics, etc.) that make up a connected home to
settings on the privacy spectrum.
If the privacy was turned up to the max, no object would sense or process activity in the
housethe house would be "dumb" again. It would be in a state like all homes until ten
years ago. Turned all the way the other direction, all tracking, sensing, processing would be
active, whatever this might mean for any given smart home.
The Privacy Dimmer allows for a gradual dimming of privacy vs sensing/tracking.

For example, if your home had pressure sensors in the floor, movement tracking for the
automatic light switches, a facial recognition video system for home security, a smart
thermostat that tracks your presence through a smartphone app and an Amazon Echo that
listens for your voice commands, you might want all these systems on in the morning when
you get ready for work and the kids ready for school. That way the temperature would be
just perfect, the lights on, the weather and traffic predictions at the ready. The smart home
would do what it's built to domake your life easier through automation.
61
Controlling privacy
However, in the evening you might want to enjoy a quiet dinner with your partner. Over
dinner and a glass of wine you might not want any alerts and notifications, any lights
switching on and off, any device listening to you, waiting for orders.
Turning the privacy dimmer would switch off all of these systems one by one. Starting with
the Amazon Echo, the security camera, the movement sensors, even the smart thermostat.
You'd have full privacy once more.
The privacy keyfob: Controlling your privacy while out and

about
The dimmer is installed in the house or room. It's immobile. But what happens if you move
about? You still should be able to make informed decision about your privacy and control the
level of sensing you participate in.
As an on-the-go extension of the Privacy Dimmer, imagine a keyfob. It's a little device you
carry on yourself at all times. Rather than switching on or off any systems in its environment
(which would be hugely invasive and aggressive), it reads a signal from the connected home
(or retail space, or public space...) and compares the amount and types of sensing and data
processing against your personal preferences as stated in that keyfob.
62
Controlling privacy
How the dimmer and keyfob would work together.

If the environment's sensing and tracking is compatible with your privacy preferences, the
keyfob might give a subtle visual cue that everything's ok. If more sensing takes place than
the wearer is comfortable with, it gives an alarm signal.
Why just a notification and not an active sensing blocker?
We're convinced that a purely technical solution to a social problem rarely works. In this
case, the notification would put the wearer into a position to make an informed decision. In
their current context, they might be comfortable sharing more than they would otherwise.
They might ask their host to turn up the privacy/turn down the sensing, or they might choose
to leave. It's a social solution to a social problem.
63
Controlling privacy
Like the Privacy Dimmer, the keyfob would also map preferences to types of services on a
spectrum (even if it would likely be a rather crude system). No sensing/full privacy would be
a self-sufficient description, but it might be hard to find any place that matched that
description, at least in urban centers.
Turning up sensing agreement (by turning down privacy requirements) on the keyfob's
spectrum, we might agree to types of sensing and data analysis like registering our
presence via our phone's wifi signal. Then (one more notch down on the privacy scale) we
would allow for CCTV, then (another notch down) agree to allow for an always-listening
device like certain types of smart home hub or controller.
For example, we might choose to set our default to agree to a minimum of sensing. But in a
close friend's home, we might not mind their smart home hub listening in on the
conversation, so we might adjust our privacy settings after a little alarm signal there rather
than leave.
On the other hand, if we are in a retail space or in, say, a colleague's or client's apartment
we might not want to share too much and adjust the keyfob setting accordingly to know
when sensing takes place.
Privacy Machines: Exploring privacy through the metaphor

of time
At the Mozilla Open IoT design sprint in Berlin, my group worked on Privacy Machines Inc, a
fictional company's privacy products.3
64
Controlling privacy
Privacy Machines at the Mozilla Open IoT design sprint, Berlin April 2016.
65
Controlling privacy
One of these machines was the Wayback Machine. It's a little box that explored how to
control privacy in the home through the metaphor of time.
Concretely, it would switch off various media-related technologies one by onemuch like
the privacy dimmerin reverse chronological order in which they were introduced. Go back
to 2013 and most smart home products would have stopped working. Turn the dial back to
2004, and you would have lost access to Youtube, Facebook, Twitter. Go back to 1996 and
your internet access might be turned off.
66
Controlling privacy
The Wayback Machine plays with two notions:
First, the time angle makes it immensely relatable. While there are many issues with thisit
fosters nostalgia, it's technologically and historically tricky, it doesn't necessarily make a lot
of senseit does help start great discussions. Because it removes the technological barriers
and works with simple metaphors and examples we found that most people would much
more happily engage in this kind of debate than if you approached from a perspective of
privacy, policy, or surveillance.
Second, it underlines that media and communications technologies have evolved from oneway (broadcasting) to two-way (phone) to systems that are tracking the users' behaviors
through cookies, traffic analysis, meta data, etc.
Media and communications infrastructure since the advent of the modern web has turned
from something watched or consumed into a system that stares right back at the user.
Connected homes are extending this right into our living rooms.
What's next?
All of these examples are speculative, non-functional prototypes. Nevertheless, we do
believe they might offer valid starting points for real products and services.
As connected homes become a mainstream reality, we need to design and build products
that make it easy to make informed decision. Users should be empowered and in control of
their privacy, rather than relying companies to determine the settings for them.
67
Controlling privacy
As these products are built, our policies need to adjust as well. Rather than playing catch-up
(and failing to do their job well as they shoot at a moving target) or killing off innovation
through over-regulation (which would likely just drive the development of connected home
products outside our jurisdictions into less-strictly regulated regions), these policies need to
be sensible and forward-looking. This is no easy task, and law makers will need all of our
support.
In the mean time, the brunt of the burden is on the shoulders of UX designers. As a group,
they might be the best positioned professionals with both the skill sets and the mandate to
ensure users are empowered to control their own privacy.
1. thegoodhome.org/projects/privacy-dimmer/
2. thegoodhome.org/milan
3. Our group consisted of Rachel Uwa, Martin Skelly, Vladan Joler and Peter Bihr. You
can find photos and more descriptions of the various fictional privacy machines at
thewavingcat.com/2016/04/25/privacy-machines/
68
Relationships
Relationships
We believe that a home, whether connected or not, should primarily serve the humans in it.
Products and services we make should augment and respect human relationships. And, as
more computation enters the home, we must be aware of how some human interactions and
scenarios may take on new norms and considerations.
Hospitality. How does hospitality change in a connected environment? Certainly there are
timeless elements of etiquette. Nevertheless, there might be new ways we interact when
we're a guest in someone's connected home.
Sharing connected objects. In a household, it's common to share lots of objects. From
appliances to furniture to toys and media and more, what are the design opportunities that
await when we think about sharing connected objects?
Home & data. The connected home is a computer. What happens to its data when we
move? What does it look like for residents to use and maintain data about their home? What
role does connectivity play when buying and selling a home?
Who has control? As homes become more connected and software-based, questions
around access and rights control become acute. Who controls your home's infrastructure?
What rights does the city have, or the neighbors, or different members of a household? How
are those permission levels negotiated?
The connected things we surround ourselves with. What criteria do we use to decide
what connected objects to invite into our homes? How do we evaluate when we want to own
something or not?
The home as a target for commercial tracking and government surveillance. If a device
has the ability to track and record you, it's likely that it will, even when you don't want it to.
Advertisers, governments and criminals alike will use the Internet of Things to track our
behavior to an even greater degree than today. We need to build safeguards against
commercial and government surveillance and protections against abuse.
69
Hospitality
Hospitality
How does hospitality change in a connected environment? Certainly there are timeless
elements of etiquette. Nevertheless, there might be new considerations when we're a guest
in someone's connected home.
Being a house guest and host in a connected home will in many ways be similar to how
humans have socialized for centuries. But we expect there will also be some crucial aspects
that are new, or that need to be negotiated. What might that look like?
A good guest
If someone invites you to their connected home, there might be some things for you, as a
guest, to consider.1
Bring a gift. It's a common practice to bring a gift for your host, such as bottle of wine,
flowers or chocolates for your host. Of course, if you knew your host didn't like one of
those things, you would want to take that into account. In the connected home, would
there be other kinds of acceptable gifts? With these gifts, how can you respect your
hosts preferred level of connectivity, so that they are delighted and not annoyed or
upset by your gift.
Communicate your connectivity preferences. If someone invites you to dinner, you
might tell them in advance what your dietary preferences and needs are. Are you
vegetarian? Are you allergic to fish or have a gluten intolerance? In the connected
home, we may find that guests will also have to communicate their connectivity
preferences. Are you ok with devices knowing your location or transmitting your
movements to a third party service? Do you have an issue with your device connecting
to the home system, and what data will it be allowed to share? Knowing these
preferences in advance could help avoid awkward situations on arrival ("Sorry, I didn't
know you don't like active vital sign tracking!"). Save your host, or your guests, the
stress and address these things in advance.
Ask permission before modifying. A good house guest is considerate of the home
they are in. They leave the space tidy and they don't mess with any objects or
equipment without permission from the host. The same will likely apply in the connected
home. Ask permission before you change the smart thermostat, either actively by
pushing a button on the object, or passively by having your preferred settings override
the hosts.
Leave data dirt outside. In many homes, it's common to take off your shoes when you
70
Hospitality
arrive. This ensures that the dirt from the outside isn't tracked throughout the house.
There might be equivalents to this practice upon entering the connected home. Turn off
your connected services that might bring unwanted data or sensors into the home. If
you recently had a virus or security breach, take precautions and let your host know so
that they can protect themselves or accept the risk.
Understand the house rules. Every home has its way of doing things. Objects that
belong in certain places, rooms that aren't intended for guests, times of day when things
should be quiet. As a guest, ask your host about the house rules and respect them. As
a host, think about subtle ways of communicating these rules, especially until more
commonplace social norms have evolved.
All that said, as a guest, you certainly have some rights or courtesies afforded to you. For
example, you should be able to ask the host what monitoring is going on in the home or to
ask to have data logs deleted.
A good host
What does etiquette look like from the host's perspective?
Communicate your connectivity preferences. Just like your guests, it's good for you
to communicate your preferences in advance and then again upon arrival as a reminder.
This could mean asking for certain kinds of objects or sensors to be turned off or not
brought into the home. (A few years ago we saw this interaction play out with Google
Glass.) It could be the opposite, and a request for them to bring a certain kind of
connectivity because you need it to prepare the meal or heat the home to their preferred
temperature or some other function.
Prepare a guest information pack for guests who stay over. We've all experienced
what it's like to ask someone for the wifi password in their home. You can prepare an
info pack in advance or leave one in a prominent location in your home so that your
guests have all the access info they need. This could be guest passwords to your home
network or server, instructions for changing the settings on certain objects, apps you
need to download to interact with basic infrastructure, or other advice and logins for
your connected home.
Allow guests to opt-out. That said, it's always good form to allow your guests to have
the option of opting out. Your role is to make them feel comfortable and welcome. If you
have different preferences for going about connected spaces, give them a graceful way
to not take part. Don't presume consent: No means no, and consent should be given
explicitly.
Anticipate needs. It's an amazing feeling to have a host that provides something even
before you realized you needed it. Carry that forward and think of what your guest might
71
Hospitality
want or need during their visit and plan accordingly. This could be food, toiletries, and
linens. But it might just as well mean providing enough power outlets for them to
recharge their devices, lending them the adapters they might need,2 or getting those
permission levels for your fridge and blinds just right.
Brief your guests about the neighborhood. If your guests are visiting from another
area, it's a nice welcome gesture to let them know about your neighborhood. This
includes things like transportation options and local sights, as well as what the
connected environment is like. What are the rules and norms of the neighborhood
regarding data collection, sensors, tracking and other issues? Should your guests be
advised that most shops only accept payments with mobile NFC, or that the data
retention laws have changed and that they may want to turn off certain services when
they go for a walk? Do locals tend to be ok with their photos being taken? Being
knowledgeable about your area and sharing that information will help your guests avoid
unnecessary friction and have a more enjoyable stay.
Hospitality is the same, but different

Much of what made for a good guest/host relationship will likely remain unchanged: being
respectful, politely communicating needs and preferences, being gracious and considerate.
Nevertheless, in the connected home that will likely manifest in slightly different ways. This
will especially be the case for interactions that are not overt, but rather happening passively,
for example among our devices and services.
1. For a first exploration of these questions based on our experience at Casa Jasmina,
see Michelle's blog post, "How to be a guest in an open source connected home":
michellethorne.cc/2015/07/casa-jasmina-how-to-be-a-guest-in-an-open-sourceconnected-home/
2. For many great examples see "How to Be the Perfect Host in the 21st Century" by
Jason Fitzpatrick: lifehacker.com/5606282/how-to-be-the-perfect-host-in-the-21stcentury
72

In a household, it's common to share lots of objects. From appliances to furniture to toys and
media and more, what are the design opportunities that await when we share connected
objects?
On the whole, you find wealth much more in use than in ownership. Aristotle1
Living together means sharing objects. The circumstances where we feel comfortable
sharing depend on the relationship and personal preferences. Different rules apply among
families, partnerships, roommates, etc.
Be it the TV remote, a book, the dining room table, or the dishes, the home is filled with
objects that might be used by multiple peoplesometimes simultaneously, sometimes
asynchronously, sometimes even without the owner's permission.2
Through personalization and modularity, we can prepare for more shared usage. By
switching accounts, the software layer can adapt the object to our preferences. Think of a
connected object as an equivalent of a game console where you log in, and the systems has
your customized digital spaces ready.
Trust management can help to better navigate who can use what object. Is there a way to
provide transparency to who used the objects and how?
Learning algorithms can start to mold objects to better fit each of their users. The collected
data can be used to improve the service for both for the individual user and for the overall
group of users. In some cases, it might make sense to contribute data to a data pool or
commons, where usage across many users can be compared and service improved.
Connected object collect a data "patina" from use. If designed right, they get better through
use, gaining value and delivering better services the more they are used. This trend in turn
could lead to more sustainable physical goods.
Rival vs. non-rival goods

Connectedness could make physical goods "less rival."
Historically, physical objects are rival goods. That means if one person has it, no one else
can have it. For example, I'm reading a paperback book, and unless you're comfortable
reading over someone's shoulder, it's not possible for you to read the book at the same time.
However, digital things are often "non-rival goods." That means if I'm reading a digital copy
of this book, you can also have a copy on your device that you're reading at the same time.
73
When we add connectivity to an object, could we start to use physical objects in a non-rival
way?
When we imagine objects being shared, what if we consider usage beyond the household
neighbors, friends, the broader world? What if an object could even be used by several of
these groups at the same time?
How can we build and use objects in the connected home so that others can use them as
well? Is there a way to build shared resources in the connected home, for example through
pooling and sharing computational capacity when we don't need all of our processors' full
capacity?
There are many questions to further explore here. The early days of collaborative
consumption and the sharing economy can provide some useful pointers in the right
direction.3
Some of the key concerns we see:
Trust management. If we have pseudo-non-rival goods being shared thanks to their
connectivity, how do we facilitate trust among users?
Value add through usage. How do we design connected objects to improve, rather
than degrade, through use? Algorithmic learning seems to offer promising avenues to
explore.
Modularity. Because shared connected objects will get a lot of usage (hopefully), they
should be built to be easily maintained and repaired.4
Privacy. In connected objects, multiple users means that we need to firewall against
data leaks between users. If user or aggregate data is to be shared back into larger
services or the commons, that data needs to be squeaky clean5 to make sure it cannot
be linked back to individual users if they have opted out.
Decision-making and resource coordination. Do we need technological or design
solutions to make decision-making and allocation of shared resources easier, or is that
a social challenge in need of a social solution? Can a connected object be designed to
facilitate this resource coordination?6
Connected objects offer a great opportunity for designers and technologists to explore how
sharing could add value and increase usage and lifecycle of physical objects.
1. Supposedly. Citation needed. Sounds great, though, doesn't it? Kidding. Property
and Ownership by Stanford Encyclopedia of Philosophy is a fantastic resource around
the philosophy of ownership: plato.stanford.edu/entries/property
2
74

2. If it is even clear who the owner is. As anyone living in a long-term relationship or
family will confirm, the lines of ownership tend to blur over time. If anything, we expect
this effect to grow stronger as learning objects will start to mold to their users over time,
potentially making even the question of ownership in this kind of setting pointless.
3. See Michelle's 2010 post Designing for collaborative consumption for an overview:
michellethorne.cc/2010/12/designing-for-collaborative-consumption/
4. The Fairphone is a great example of a device that uses modular design to enable
repair and upgrades without having to toss out the whole thing. We should foster this
kind of modular design, as it will help make more sustainable objects as well as objects
that are more shareable: fairphone.com
5. Totally a technical term.
6. With his Addicted Toasters project, creative technologist Simone Rebaudengo
explored this in a playful way: When his connected toasters felt under-utilized, they
tweeted publicly to find a new owner who would use them more frequently:
simonerebaudengo.com
75

If we live inside a computer, how do we handle and maintain our data? How does data
change the way we live in a house or move in and out?
A connected home is a home kitted out with sensors and network infrastructureit is
essentially a computer we live in.
That means we need to consider implications of security and control.1 It also means we
need to think about technical failure and personal data.
Most failure situations in a connected home are likely to be variations of:
Users not understanding their home's infrastructure2
Things clashing with things3
Designing interactions to mitigate these issues and building them in a way that is reliable
and user-readable (read: fixable) will be key. You don't want your flat's operating system to
crash on you.
Computers crash
Not understanding your home (or not being understood by it) is bad enough. But what
happens in case of a more fundamental crash? What does the blue screen of death look like
for a home?
76
The blue screen of death, universal symbol of crashed computer systems.4

Can you reboot your apartment? How? Have you tried turning it off and on again?
The IT Crowd: Have you tried turning it off and on again?

Often, a simple reboot of the wifi router or oven will do the trick. But what about more
systemic cascading crashes? The more complex a networked system the more likely bugs
and crashes are to domino. Currently we don't have the infrastructureor help hotlineto
help us fix these issues.
Moving in & out

77
We don't know yet what kind of data will be common to be created in the apartment and
moved with us from one place to another. But we do know that moving with both data and
preference settings will be "a thing" we all do.
As a way of exploring mechanism that might help us with this process (independently from
the types of data), we created a bit of speculative design for The Good Home5, the Home
Totem6:
The Home Totem represents the owners privacy and sharing preferences. It also
serves as a link between the home and the owners secure data or profile on the
internet. Think dietary requirements, energy consumption profiles, records of former
home ownership and the like.
The Home Totem is a physical avatar that the owner moves from home to home. When
they move in, they set down the Home Totem and the home adjusts its privacy and
sharing preferences accordingly. By moving it from one connected home to another
they transfer their preferences to the new home and establish residency. In a sense, it
is the heart of the connected home as far as the residents are concerned.
The Home Totem is a piece of speculative design created by Peter Bihr for The Good Home
Project.
The core idea is to make it easy to establish residency by simply putting down this symbolic
object. It would hide the backend work of setting preferences, linking a digital ID to the place,
etc.
In the sketch you'll notice that the Home Totem consists of several pieces stacked on top of
one another. This might be a blunt, but intuitive way to allow for moving in and out of shared
living situations.
78
What happens when you move out of your home?

When stacked, shared residency and preference/data settings are in effect. Upon moving in
or out, you would add or remove your piece from the stack, triggering a change of the
settings.
As we fill our networked homes in personal data, as our kitchen clouds and algorithmic
assistants learn about our behaviors and preferences, we imprint ourselves on our living
spaces not just physically as we have done traditionally, but also digitally, through data.
And much like we tend to renovate and paint the walls either upon moving in or moving out
(depending on which country you're in) we'll want to do the same on the data & algorithmic
level. In other words, what does it look like to reset the apartment?
Just like we don't want to leave any smudges or holes on a wall after living at a place, we
don't want to hand over a house with our data still inside it. Fresh coat of paint, please.
Factory Reset
So how does a factory reset work? Does it wipe all data completely? Are there parts of our
home's neural network that cannot be fully wiped, just like really old houses have grooves in
their wooden floors from generations of people walking the same beeline from bedroom to
bathroom?
When we arrive in a new place, do we start from scratch or do we transfer everything over to
the new place? (Think about the hassle it still is to move your data and settings to a new
computer!)
What if the location change is only part of what's new? For example, what happens if a
couple splits up, and their home's data sets were generated by two people, and now the new
place is for one? Will data from a bachelor pad seep into a new shared couple's space, like
some old Bob Marley poster? Will it be greeted by the other data set with a similar kind of
resentment as its physical counterpart? When children grow up and leave the home, do they
take their childhood data with them? Or does it get boxed up and stored as mementos, to be
recalled and possibly discarded at a later date, while meanwhile clogging up their parents'
basement?
Maintenance and logging

As another contribution to The Good Home project, Alexandra Deschamps-Sonsino created
the Home Sweet7:
79
When we think of the home in the UK, were talking about a living and breathing
infrastructure which moves and reacts to its environment. Ive lived in homes that have
leaked, had to replace boilers, flooring, tiles, grout. Its always felt like an endless battle,
and many moves in London have added to the feeling of unsteadiness. Every time I
move, I ask the same questions: the landlords details, where the meters are, how to
turn on the taps without burning myself. I get insurance policies moved over, making a
set of assumptions on information I dont have yet: when the building was built, whether
there has been subsidence recently. I bring in my own data but I also need a lot of
information that belongs to the home itself.
Home Sweet is a sketch of a digital service that captures and hosts a homes data
forever (presumably set up by the local council) enabling owners and tenants to come
and go with their own data but also leave something behind for the next person.
There is potentially great value to be gained by logging and unlocking this type of data and
building services around is.
Think about just how much easier it makes the life of residents to be able to tap into this kind
of maintenance log, find out who has in the past provided reliable maintenance and repair
services (or botched them up), to see if there are bits and pieces of infrastructure that have
been creating problems over and over.
When buying a home, this type of data is notoriously hard to find or verify, yet priceless. The
documentation on infrastructure in homes (especially older ones) is usually thin and bad.
This holds especially true in cities where there is a higher turnover of residents.
80
There is tremendous potential in figuring out how we handle data residue when moving in
and out, and how to unlock services built on top of this type of data. This goes for the
individual unit as much as for the neighborhood or city-wide level.
How do we merge our individual sets of "home data" when moving together, and divide it up
upon moving out?
What do failure modes in connected homes look like? Is there an off switch? How can we
build and design connected homes to be resilient and if they fail to do so gracefully?
1. As Cory Doctorow does here: wired.co.uk/magazine/archive/2015/07/features/stopspies
2. See the chapter Future or for earlier thoughts on this topic:
thewavingcat.com/2015/08/03/understanding-the-connected-home-ground-rules/
3. See the chapter on Negotiating, or for earlier thoughts
http://www.thewavingcat.com/2015/08/17/understanding-the-connected-homemanaging-conflict/
4
. Source: Wikipedia
commons.wikimedia.org/wiki/File:Windows_9X_BSOD.png#/media/File:Windows_9X_B
SOD.png, Licensed under Public Domain via Wikimedia Commons.
5. More about the Good Home project and the other exhibits we made for the Fuori
Salone at Milan see thegoodhome.org/projects
6. See thegoodhome.org/projects/home-totem/
7. For a more detailed presentation see thegoodhome.org/projects/home-sweet/
81

As homes become more connected and software-based, access and rights control become
more salient: Who controls your home?
When our homes become connected and more software-based, it will be software that
grants or denies the rights to use and change everything that goes on inside the room. (At
least everything that has a digital or data aspect to it.) This suggests that going forward,
access and control over the home might resemble access and control over digital media
content.
The problem with Digital Rights Management

(DRM)
Enter Digital Rights Management: DRM for short. Time for a little refresher.
The world of commercially distributed digital contentmovies, music, bookshas been
shaped by discussions about Digital Rights Management, or Digital Restrictions
Management as some critics call it. DRM is a set of technological systems that aim to control
who can and who cannot access digital media and under what conditions.
The concept sounds simple enough: Is this paid content? If so, has this user paid and is
therefore eligible to consume it?
But it isn't quite as simple. In fact, DRM is highly controversial. Because it works on the
premise of restricting access, often it punishes legitimate uses. The files or streams a
customer pays for are those with built-in restrictive technologies. In other words, the
customer pays for damaged files, whereas a (potentially illegally) downloaded version would
usually be free of such restrictions.
If this sounds silly, it's because it is. A label can never predict how a buyer of music or film
might want to consume the music they boughtyet often DRM measures restrict playback
of a song or movie to certain devices, operating systems, or contexts.
This is so confusing and limiting. Even when someone pays for content legally, their freedom
to use it is not guaranteed.1
In What We Buy When We "Buy Now", a paper forthcoming in The University of
Pennsylvania Law Review, respected copyright scholars Aaron Perzanowski and Chris Jay
Hoofnagle report on an experiment testing what people think they get for their money when
82
they click "buy now" in stores selling digital things (ebooks, games, music, videos, etc). In
short, it's not what they think.
So while there is certainly a need to raise consumer's media literacy here, it's also not
primarily their fault. The study shows that retailers use misleading language all along:2
"Retailers such as Apple and Amazon market digital media to consumers using the
familiar language of product ownership, including phrases like 'buy now,' 'own,' and
'purchase.'
Consumers may understandably associate such language with strong personal
property rights.
But the license agreements and terms of use associated with these transactions tell a
different story. They explain that ebooks, mp3 albums, digital movies, games, and
software are not sold, but merely licensed. The terms limit consumers' ability to resell,
lend, transfer, and even retain possession of the digital media they acquire. Moreover,
unlike physical media products, access to digital media is contingent-it depends on
shifting business models, the success and failure of platforms, and often on the
maintenance and availability of DRM authentication systems years after the consumer
clicked 'buy now.'"
DRM for the home: Residential Rights

Management
Now we have to assume that similar challenges will arise around the connected home. Only
there, the effect would be compounded as it would affect on our essential domestic
infrastructure.
Let us be blunt: DRM for domestic infrastructureResidential Rights Management if you will
should under no circumstance become part of our homes.
Rights managements is hard to understand. If you've ever encountered a situation where
you needed to assign read/write/execute rights for a personal computer, server or network
without proper training, you'll know how confusing this iseven professionals regularly fail at
selecting the appropriate permissions settings.
In a connected home, we'd have to work with complex, interdependent access rights to
buildings, apartments, rooms, devices, usersall of which are entirely non-intuitive to
handle.
With connectivity come power struggles and access & rights management.
83
Who has root to your home?

Root access3 is what we call full, unfettered accessincluding the rights to shut out other
users, delete all data and install any program.
Who are the parties who have this kind of access to your home? Even at the best of times,
scenarios where another entity, person, or organization has control over your home seem
problematic. Screw-ups seem both unavoidable and potentially horrendous.
For a little taster of power struggles to come, consider the following scenarios and questions
of remotely controlled rights or restrictions management:
A renter is late on their payment. Can the landlord shut down services or access until
the rent is paid? Potentially non-essential services like air conditioning? Essential
services like plumbing? Access to the apartment? Under which conditions?
A teenager comes home later than agreed with their parents and finds the door
does not unlock. The parents, feeling wronged and slightly vindictive, set up the
home's smart lock not to let their kid in after 10pm. And the teenager doesn't have the
permissions to override it.
An elderly person in an assistive living facility wants to cook a meal. Yet, the oven
won't turn on this morning. An algorithm set up by their son has decided to reduce the
risk of fire and burns, so the resident's usage rights to the oven were remotely revoked.
A malicious landlord wants to get rid off a renter and starts messing with lights
and heating. What kind of recourse do renters have if they lose control of the
infrastructure they live in?
A resident is a diabetic. Can their health insurance track if a diabetic takes any sweet
beverage or food out of the fridge? Could the insurance company block access to the
fridge? Or could they modify orders for grocery delivery services?
A young couple is two weeks late on their mortgage payments. Can the bank shut
them out of their homes by taking them off list of authorized users in the smart lock's
facial recognition system?4
We chose those pointedly dystopian scenarios for a reason. Many of them begin with a
legitimate reason, concern, or gripeand then they go overboard, leveraging connected
home systems in ways that go entirely beyond what's acceptable.
We believe that residents should be in control of their domestic lives. As a rule of thumb, no
company or other entity should unilaterally be in a position to shut down services in the
home. Nor should companies be able to sweepingly change terms and services, like they
currently do on our personal computing environments. The iTunes Terms of Services just
changed.
84
We need to ask ourselves: Who controlsor should be able to controlaccess to and within
a home? Is it going to be the resident, landlord, owner, renter? An insurance, bank, or
Airbnb? What are the kind of legal frameworks that can ensure that residents are in charge
of their home?
The home is a space that needs special protections.
1. See Cory Doctorow's summary of the study: boingboing.net/2016/05/13/clicking-buynow-doesnt.html
2. Perzanowski, Aaron and Hoofnagle, Chris Jay, What We Buy When We 'Buy Now'
(May 13, 2016). 165 University of Pennsylvania Law Review (2017), Forthcoming.
Available at SSRN: ssrn.com/abstract=2778072
3. See Super User on Wikipedia: wikipedia.org/wiki/Superuser
4. We have already seen cases where leased cars won't start if the lease payments are
overduea truly malicious, out of proportion invasion of privacy and basic rights:
boingboing.net/2014/09/25/class-war-meets-the-war-on-gen.html
85

How do we choose which connected objects to invite into our homes?
Which connected objects do we invite into our homes? We believe that fundamentally, when
answering this question, we should not consider connected objects differently than any other
object.
However, as a matter of personal philosophy we believe that it pays to be mindful of what we
surround ourselves with. As connected things in our homes might change through software
updates or learning algorithms, a careful look at what we invite in is certainly warranted.
Mindful ownership
"You are what you eat," the saying goes. In the same spirit, you are what you own. Just like
one can be mindful about their diet in an effort to have a healthy, balanced body, one can
also be mindful about the objects one invites into their lives to ensure a healthy, balanced
environment.
The home is where we house most of our possessions. It matters what we choose to
surround ourselves with. A comfortable chair makes us feel at ease and supports us. A
cherished family memento makes us feel loved. A malfunctioning appliance causes
frustration.
Objects affect us. In the connected home, we should be mindful about what we invite in.
"Solving problems" isn't the only criterion

In technology circles, a lot of attention is given to building useful things and then to
optimizing them. Does this fix something? Does it solve a problem? If so, how can we solve
it more efficiently?
Yet, the ability of an object to "solve a problem" isn't the only criterion humans have when
choosing the things they surround themselves with. The urge to replace human judgments
with algorithms and optimized efficiency has its limitations.
"Constructing a world preoccupied only with the most efficient outcomesrather than
with the processes through which those outcomes are achievedis not likely to make
them aware of the depth of human passion, dignity, and respect." Evgeny Morozov1
86
As we imagine the connected home, one that helps us be more human and that brings us
joy, we must explore a fuller set of selection criteria when deciding what objects to invite into
it.
Four categories of objects

A possible set of criteria for personal possessions is laid out by the science fiction writer and
design critic Bruce Sterling:2
1. Beautiful things.
2. Emotionally important things.
3. Tools, devices, and appliances that efficiently perform a useful function.
4. Everything else.
Sterling explains how to assess the objects you own using these categories. He encourages
the mindful selection of the objects you surround yourself with everyday.
You are not "losing things" by acts of material hygiene. You are gaining time, health,
light and space. Also, the basic quality of your daily life will certainly soar. Because the
benefits of good design will accrue to you where they matterin the everyday.
Does it spark joy?

An even more compressed approach is suggested by Japanese "tidying up" specialist and
best-selling author Marie Kondo.3 She advocates for using just one question when deciding
whether to have an object in your home:
Does it spark joy?
Her argument is that if an object is useful, it will spark joy because you acknowledge its
ability. If it's beautiful, it will also bring joy. If it's emotionally important, you will also recognize
that through joy.
Make joyful objects

Humans deploy a range of criteria when selecting what objects to invite and keep in their
homes. The ability for an object to solve a problem is certainly a factor, but not the only one.
As we build and consider things for the connected home, let us bear in mind the other ways
in which objects help us be human: Are they beautiful? Are they emotionally important? Do
they spark joy?
1
87

1. For his critique of so-called solutionism see Evgeny Morozov's To Save Everything,
Click Here. clickherethebook.com
2. See viridiandesign.org
3. See The Life-Changing Magic of Tidying Up. tidyingup.com
88

Surveillance and tracking are serious issues for the internet of things and the connected
home, and we need to build safeguards against them.
As we have learned over the last few years through the Snowden revelations, we live in a
system of ubiquitous, nearly limitless tracking and surveillance. It's to a degree which easily
matches the dystopian scenarios of even the most paranoid security activists. Our every
step, at least in the digital world, is watched by commercial entities and governments.
Directly or through meta-data, complete privacy is impossible.
Here are four kinds of tracking:
Personalization and context-aware services are a kind of tracking and surveillance we
voluntarily agree to. These assistants could not work without being aware of where we are
and what we have been doing. Therefore, we allow them access to this information in
exchange for personalized recommendations and search results.
Other types of tracking are more obscure and more invasive, and they often come from
advertisers. We might unknowingly agreed to data mining by installing an app and clicking
"yes" on a lengthy, impenetrable end user license agreement. A so-called super cookie
tracks our behavior across the web, even long after we have logged out of the service that
initially installed it. Adtech can be nasty business indeed.
Another type of trackingusually referred to as surveillancecomes from governments
who tap into our devices and communications channels. Historically, at least in democratic
regimes, surveillance was prohibited against the country's own citizens, with limited
exceptions for criminal investigations. Every non-citizen was considered fair game for
intelligence services. Since Snowden, we have evidence that most democratic countries run
extensive surveillance on their own citizens, directly or indirectly exchanging data about their
citizens with other countries: I'll watch your citizens for you, you'll watch mine.
Lastly, there is criminal cracking of our systems. Malware, trojan horses, fraud, taking over
devices and using them for nefarious purposes are common tactics. A recent story got lots of
media attention when a criminal network took control of internet-enabled fridges and used
them to send spam and run coordinated denial-of-service attacks. It would be funny if it
didn't stand for a serious, larger problem.
Tracking and surveillance are a serious

problem
89
Tracking and surveillance are a serious problem for digital communications, for the internet
of things, and especially for the connected home.
They hinder innovation as they undermine trust in these new technologies, and rightfully so.
More importantly, they undermine the social contract that our society runs on.
Increasingly, user rights equal citizens rights.
A recent study by the US Census Bureau1 found that nearly one in two internet users say
privacy and security concerns have now stopped them from doing basic things online:
"Americans are increasingly concerned about online security and privacy at a time
when data breaches, cybersecurity incidents, and controversies over the privacy of
online services have become more prominent."
The report suggest that this has reached a critical pointpeople stop using the internet
altogether. "The research suggests some consumers are reaching a tipping point where they
feel they can no longer trust using the internet for everyday activities."
The report provided a breakdown of internet users' biggest concerns:
Nearly two thirds listed identify theft (63%)
About a quarter was worried about data collection by online services (23%)
Just about one in five was concerned about loss of control over personal data (22%)
and data collection by government (18%).
It is this last point that is particularly concerning from a political and societal point of view, but
really it is the full package that means we are in trouble.
Now this study focused on internet use, not the internet of things. But it is safe to assume
that for IoT, and especially the connected home, these concerns will be as strong or
stronger.
Governments are surveilling IoT, including in

the home
The US intelligence services have publicly stated their intentions to use connected devices
for mass surveillance. Other governments won't be far behind.
In a 2012 interview, CIA director David Patraeus called the surveillance implication of the
internet of things "transformational... particularly to their effect on clandestine tradecraft."2
90
More recently, James Clapper, the US director of national intelligence, stated that
"intelligence services might use the [internet of things] for identification, surveillance,
monitoring, location tracking, and targeting for recruitment, or to gain access to networks or
user credentials."3
The consequences to these developments were highlighted in a study by the Berkman
Center called Don't Panic: Making Progress on the "Going Dark" Debate.4 The internet of
thingsand again, this applies particularly in the connected homemight be a back door
that helps criminals and governments alike to work their way around all other defenses
(technological and legal alike) and towards a ubiquitous surveillance.
"Networked sensors and the Internet of Things are projected to grow substantially, and
this has the potential to drastically change surveillance. The still images, video, and
audio captured by these devices may enable real-time intercept and recording with
after-the fact access. Thus an inability to monitor and encrypt channels could be
mitigated by the ability to monitor from afar a person through a different channel."
It does not help at all that the majority of users find it difficult to figure out the right tools and
strategies to meaningfully enhance their privacy, as a 2015 Pew Research study found.5
Again, this is for online communicationsand it will apply to an even larger degree to IoT in
general and to connected homes in particular.
Yet, taking privacy in their own hands might be the only way for users to protect themselves
given the sad state of security in IoT these days. This is verified with a quick look at search
engine Shodan, which indexes thousands of unsecured web-connected devices.6
We have to build safeguards against tracking

and surveillance in the connected home
It looks that as of today, we have to assume we are being tracked and surveilled, with
benevolent and nefarious purposes, by good and bad actors. We have few tools to defend
ourselves, and they are hard to use. Civil society policy makers and scientists are
scrambling to keep up rather than steering the course. It's a bleak outlook indeed.
To get in front of a major catastrophe, we will need to work on all aspects of securing IoT
and our connected homes:
1. Educating policy makers and scientists to understand the implications of complex
connected, data-driven systems, like IoT, connected homes, and smart cities, so we can
get to a robust, resilient legal framework that governs tracking and surveillance both
from commercial and government actors and protects our privacy.
2. Educating users and citizens about the importance of security and privacy when living in
91
connected environments so they demand stronger privacy and data protection, and we
get to stronger regulation and market incentives for companies in the IoT market to
make security and privacy a priority for connected products and services.
These are significant, hard, and complex challenges. Yet they are essential for us to tackle.
We cannot, under any circumstances, afford to allow widespread surveillance into our
homes.
Yet at the same time, disconnecting is not an option either. The innovation happening
around IoT and connected homes has tremendous potential both commercially and
societally, and can improve the quality of life of hundreds of millions.
While the journey might not be easy, this is an endeavor worth fighting for.
1. The study is available online in full at: ntia.doc.gov/blog/2016/lack-trust-internetprivacy-and-security-may-deter-economic-and-other-online-activities. The Washington
Post has a summary of the results: washingtonpost.com/news/theswitch/wp/2016/05/13/new-government-data-shows-a-staggering-number-ofamericans-have-stopped-basic-online-activities/
2. See WIRED 2012 at wired.com/2012/03/petraeus-tv-remote/
3. See The Guardian (2016) at theguardian.com/technology/2016/feb/09/internet-ofthings-smart-home-devices-government-surveillance-james-clapper
4. The full study "Don't Panic: Making Progress on the 'Going Dark' Debate'" by the
Berkman Center (2016) is available online (PDF):
cyber.law.harvard.edu/pubrelease/dontpanic/Dont_Panic_Making_Progress_on_Going_Dark_Debate.pdf
5. For details, refer to this 2015 study by Pew Research: pewresearch.org/facttank/2015/04/14/why-some-americans-have-not-changed-their-privacy-and-securitybehaviors/
6. The search engine Shodan indexes thousands of unsecured web-connected devices,
incl. webcams and more: shodan.io
92
Research approaches
Approaches
Over the last year, we've experimented with various ways of exploring the questions
surrounding the connected home.
We're trying out a wide range of formats and research methods and are happy to share our
experiences with them so far, as well as learn about new approaches.
Content sprints are a great way to do a first quick collaborative deep-dive into a topic as
complex as connected homes.
Designing speculative prototypes is a worthwhile way to make complex, abstract issues
more graspable, and an exhibition can provide helpful deadlines and a platform for sharing
these ideas with the public.
We hope that this can be useful to others in their own investigations, and we'll certainly keep
experimenting with these and other formats.
93
Sprints
Sprints
Content sprints are a great way to do a first quick collaborative deep-dive into a topic as
complex as connected homes.
Content sprintssometimes also referred to as topic or book sprintsare heavily
condensed writing exercises with a focus on collaboration and peer review. A group comes
together and within a few days or weeks writes a chunk of content to be published by the
end of the sprint.
This very book was started as a content sprint back in 2015, and updated in another in 2016.
We believe that this is a great way to explore a new topic. While you will never get as deep
into the subject matter as with longer term academic research or the deep immersion that
comes with writing a book for a traditional publisher, great insights can be gained with this
faster format.
The trick is to get the framing and team right. Gather experts from different backgrounds.
Identify practitioners with a range of experiences who can draw on learnings and
publications without much reading or research as they write.
Frame the topic so they can be chunked into discreet pieces of writing. We found it's helpful
to start by writing the table of contents. Each chapter should have one sentence that
describes its main argument. And then each participant understands how the whole book
sits together and is able to pick up a piece.
We think of content sprints as a learning technique thats part workshop, part collaborative
writing exercise, part shared reflection. Combined with a strong bias towards shippinga
term from the software development word meaning output, or just getting stuff out the
doortopic sprints favor publishing over planning, producing copy over drafting outlines,
peer review over control through external established publishers.1
We have been involved in a number of similar sprints, from super quick one-day mini-reports
to large groups writing a full book over the course of a week. (With two authors and several
weeks of writing time in total, this book sits somewhere in the middle.)
If you plan to explore the connected home, we recommend you consider a content sprint.
Sprints have strengths and weaknesses. They can be great tools, but arent fit for every
context as they are built on principles that favor certain types of outcomes over others.
Within this framework, its relatively easy to fine-tune the process to match various needs.
2
These unshakable pillars of the sprint are:
94
Sprints
An emphasis on shipping. Done is better than perfect, as the saying goes. By the
end of the sprint, the result is going to be published, no matter what. (Or handed over to
a copy editor for final polishing, if thats more your thingbut no more work should
happen on the core content.)
Strong focus on collaboration. This is a team exercise that lives on collaboration,
sharing, and open exchange. Where there is too much ego at play, a sprint will not fulfill
its full potential. Credit where credit is due. In a highly collaborative environment it is
hard to give credit for any given word, sentence or paragraph. However, everyone
working on the sprint or any of its chapters should be visibly credited.
Peer review. No matter whos the leading expert of the group for any given topic,
chances are that others can add value and refine, both in substance and in style. Peer
review and co-editing are at the very heart of the sprint process.
To write this book, we used Github and Gitbook. These tools enabled us to collaborate
readily on texts and to have version control. It also meant our texts were published
immediately to the web, which somehow made us less precious about when something was
"done" or not.
We would love to see more writing on these issues. If this sounds like it might work for you,
we urge you do try out a topic sprint.
In recent months, we've also experimented with design sprints. In future writing, we'll share
our reflections from that format as well.
1. For an overview of strengths and weaknesses, as well as some hands-on pointers,
see Peter Bihr's article in E-180 magazine, Topic Sprintsthe fastest way to dive into a
topic: mag.e-180.com/en/2015/09/topic-sprints-the-fastest-way-to-deep-dive-into-atopic
2. Again, consider Peter's E-180 magazine article from which this is re-quoted.
95
Exhibitions
Exhibitions
Designing speculative prototypes is a worthwhile way to make complex, abstract issues
more graspable, and an exhibition can provide helpful deadlines and a platform to share
your work with the public.
In addition to the intellectual exploration of connected homes that has become this book, we
wanted to also proposing actual design solutions to the complex and often abstract
questions posed by increasingly connected homes.
Thus exhibiting became a regular tool in our projects, such as the Mozilla Open IoT Studio,
and new ones were born, namely the Good Home Project in collaboration with Alexandra
Deschamps-Sonsino.1
Our mission statement for the Good Home Project is as simple as it is ambitious:
The Good Home project explores ideas for 21st century urban living. In an on-going
series of installments it will explore what life for a technologically-savvy household
could look like while they are (and sometimes arent) at home in the near future. We will
showcase how flexible living interacts with the limitations of diminishing household
budgets, limited global resources, evolving concepts of privacy, the sharing economy,
and global migration.
We build prototypesoften purely conceptual or speculative, but increasingly also working
prototypesthat fill these questions of domesticity and the connected home with life. This
always considers the larger context, the mega trends shaping the world over the next few
decades.
This is an approach that inherently produces outcomes that are easy to criticize (and
criticized they should be!), that are likely to look dated quickly, and that in some cases will
turn out to seem pointless in the near future. However, in the short and medium turn lots and
lots of valuable insights can be gained this way both from the design process and the
conversations that follow.
The fact that there's an exhibition date also means the team works against a hard deadline.
This helps get things done, especially in contexts where the team is decentralized and has
other, more commercially viable obligations. It also is fun, and the importance of that cannot
be overstated in volunteer-driven projects.
To give a few examples of the role of physical prototyping, the Good Home's Privacy
Dimmer2 is both the basis for and the physical manifestation of the chapter on controlling
privacy in this book and The Wayback Machine that emerged out of Mozilla's Open IoT
3
96
Exhibitions
3
Design Sprint . In both cases, questions of privacy in the connected home and the
interactions surrounding privacy control were explored by different collaborators, with their
different background, and manifested in a speculative design object.
These conceptual prototypes essentially serve as narrative devices. Narratives can help fill
concepts as abstract and complex as privacy, empowerment, participation, or data
protection with life. This in turn lowers the barrier of joining the conversation. If a debate is
too abstract, technical, or just jargon-filled, many people will not feel like their contributions
are welcome. This means not all voices are heard, and inclusivity and diversity are at the
very core of what makes a home a home.
We also see prototypes serving a community-building function. This approach is deployed
regularly together with Jon Rogers at the Mozilla Open IoT sprints. By making a
collaborative prototype, new relationships are formed, peers learn with one another, and
skills are shared. This form of learning in action grows a professional community that learns
and makes together.
We strongly urge anyone exploring these topics to prototype heavily and frequently and
exhibit the results so others can engage with, and be inspired by, the results.
1. The Good Home Project is an initiative co-founded by Alexandra DeschampsSonsino and Peter Bihr that explores 21st century living to life through design solutions
(both speculative and real) exhibited around the world. For all exhibitions and
installments see thegoodhome.org.
2. See thegoodhome.org/projects/privacy-dimmer
3. See thewavingcat.com/2016/04/25/privacy-machines
97
Research roadmap
Research roadmap
Within this book we touched on a wide range of questions and some approaches. But by no
means can it be considered a comprehensive guide to the connected home.
In the following section, we list a few areas that deserve more research, as we expect them
to yield lots of insights. They are also topics we are likely to explore further, both within
future revisions of this book and in our other work.
98
Policy matters: The connected home and

the smart city
Connected homes exist within the larger context of a smart city, and we need to design both
services and policies to make this work for citizens.
In this book, we focus primarily on the home as a connected environment. However, as is
the nature of networked systems, homes do not exist independently but rather in context.
We need to consider a smart home in the context of the smart city. This holds especially true
here where data will be exchanged between systems..
A smart home in a smart city

How does a connected home interact with an increasingly smart city? How do we want the
two systems to interact?
These question touch upon privacy, data ownership, and participation. Each of these are
huge fields that require us to drill deeper, with implications for different groups and
professions.
Designers of products and services might want to extend their services in both directions,
making sure their ideas reach from the city into the home and vice versa. How to build
human-centric systems and interactions that create value at the intersection between inside
and outside the home? How to design the data-aspects of the home's threshold?
Data scientists and software developers might want create data models that are
applicable and appropriate to use in this context, and determine which standards to adopt.
Startups and entrepreneurs might want to explore how to build useful, desirable products
that create and capture value when transactions take place between the two systems. Some
scenarios to consider:
Urban mobility & data
Energy consumption and production data
Real estate, lease, and holiday rental scenarios
Policy makers around the globe are working hard to understand the implications of what it
means to make a city "smart" as they start pilot projects and draft smart city policies.
99
Policies for smart cities & connected homes

Earlier this year, the German federal government released a major report about the future of
1
urbanization. As part of this report, Prof. Dr. Christoph Bieber and Peter were
commissioned to contribute an expert study on the implications of the smart city for citizens
(as opposed to vendors and other commercial entities).2
Many of the questions explored in our study apply directly to the connected home, or offer
avenues of exploring this context further, as they tap directly into larger questions of
participation, ownership, and resilience.
We took those very questions, and as a little playful exploration, replaced "city/urban" with
"home", "public" with "private".
Here is the result:
What does digitalization mean for the urban home context today? What happens to
cities homes when infrastructure, public space private spaces and citizens are
becoming increasingly technologically networked, tracked by sensor networks, and part
of a rich data ecosystem?
The various elements of a data-smart city home and its integration into urban culture
are to be recognized as a part of urban governance structures. Aspects such as
sustainable development, education, inclusion, transparency and openness deserve
attention accordingly.
The path to increased security and resilience of the smart city home must include
transparency and the principles of open source. Strong data sovereignty of citizens is
the basis for participation and problem solving competence especially when facing
possible technological problems of digital urban home infrastructure.
A city home that is measured and sensed through sensors, cameras and other survey
systems is always a city home under surveillance that could discipline its citizens
residents. This means a conflict between problematic surveillance and control on one
side positive knowledge and data based opportunities on the other. The ambivalence of
the potentials that threaten democracy and those that foster it need to be reevaluated
constantly.
Not all of these simple replacements fit 100 percent. And yet you get the idea: The larger
issues, threats and opportunities all apply to the context of the connected home to some
degree. And in building connections and connectedness across the boundary between the
twoin strengthening the data flow between both worldsthe lines increasingly blur.
100
What happens inside the home is part of the larger cityscape, and now it is measurable in
real time, too.
We need policies for the connected home

Lawmakers have long been playing catch up with technology. To some degree, this is in the
nature of things. However, we do need sensible policies to govern connected systems that
are as sensitive as the connected home.
As part of our smart city report we drafted some guidelines that could serve as basis for
smart city policies.
These guidelines, in turn, are inspired by the de facto principles that shaped the open web in
its early days: open source, openness, decentralization, bottom-up innovation.
For connected homes, these same principles might apply, with one difference: There should
be a stronger focus on privacy and data protection.
1. The English version of the WBGU report is available at wbgu.de/en/flagshipreports/fr-2016-urbanization
2. An executive summary, some background, as well as our policy recommendations
and further links can all be found at thewavingcat.com/2016/04/25/smart-cities-in-the21c-humanity-on-the-move-the-transformative-power-of-cities
101
Neighborhoods
Neighborhoods
How can connected homes contribute to a healthier neighborhood, and how can it offer
residents a better ambient understanding of what's happening in the neighborhood?
When connected homes are networked together, we may see tremendous potential to
improve the neighborhood they're in. Here we outline some ideas.
Increasing ambient awareness

What kind of information and data about a neighborhood would be useful for residents?
Traffic, crime statistics, peaks and valleys of energy consumption? Increase and decrease in
the flow of people in the streets? Sharp changes in the number of tweets sent or photos
shared online? Local weather data like temperature changes and rainfall?
How can we communicate these types of data in an ambient way without the blunt
instrument of screens? How about ambient sounds like birdsong, glowing lights, water
sloshing in a tank, a Tempescope1, a Good Night Lamp2, a fan spinning faster, a cuckoo
clock, or a vibration3 through jewelry or the park bench?
What are the most useful, least intrusive models of notification? Should these alertsbe
they ever so subtlebe sent when something changes (state change), when something is
wrong (alert) or continues to be ok (affirmation), when certain milestones are reached
(positive notification), or continuously as an ongoing background ambient awareness system
(monitoring)?
Connected homes could contribute to larger systems in the neighborhood and offer ambient
ways of monitoring the neighborhood's health.
Contributing data to the commons

What types of data are useful and safe to share with the neighborhood and the city beyond?
What services can be built on top of data like that to save energy, reduce waste, increase
production of energy, enable more rich social interactions?
What levels of community data are there (building, street, neighborhood, postal code...) and
what frameworks would be most appropriate to apply?
102
Neighborhoods
We need to figure out models to use community and neighborhood data in a way where
neighbors are empowered to collect and use this data and better services can be built with
them, keeping privacy and security, as well as inclusivity and diversity in mind?
Applying to retail and commercial use

Traditionally, private business has been data savvy in terms of value extraction and creation.
There is huge potential value for retail and commercial uses of data from homes, buildings,
and neighborhoods.
What kinds of services improve lives of citizens, visitors, and other stakeholders in the
neighborhood? What kind of obligations should be tied to using these types of data
commercially? What does that mean for transparency, APIs, licensing, policy and political
work?
Fostering stronger social relationships

How can connected homes and the data systems surrounding them help residents have a
richer social life? Can residents be empowered to more easily meet or help each other, or to
resolve problems or social tensions? Can it be a tool for networked social action and
participation?
1. The Tempescope is a connected object that sits on your shelf and creates a
contained weather environment. It can read weather conditions and forecasts from the
internet and generate that environment within the object, complete with clouds, lightning
and more: tempescope.com
2. Alexandra Deschamps-Sonsino's Good Night Lamp is a family of lamps set up in
different locations: If the larger lamp is turned on, it will signal to the smaller lamps to
turn on as well. They are beautiful objects that send an ambient social signal:
goodnightlamp.com
3. What information these various vibrations could best convey is still somewhat
uncharted territory, but Iskander Smit has done some thoughtful research and design
dismystifying vibration notifications, including the dimensions and range of these
signals as well as what parts of the body or environment would convey what kind of
information: medium.com/the-startup-magazine-collection/design-for-timelyinteractions-b7d7b3ef5d50
103
Where from here
Where from here

This book became an on-going research project about the connected home.
It opened a lot of lines of inquiry. Some we may see answers to in a short while. Others
might be laughable in a decade from now when many of these technologies will have settled
in and become quotidian.
The book has also helped spawn a number of very concrete projects and activities in our
other lines of work. As part of the head of Mozilla's Open IoT Studio program, Michelle has
brought a lot of the thinking in this book into life through an ongoing series of Mozilla Open
IoT design sprints (the insights of which in turn feed back into this book).1
2
Peter has joined forces with Alexandra Deschamps-Sonsino to co-found The Good Home ,
which aims to take the theoretical work of this book as basis for concrete design proposals
for better living in the 21st century. Peter will continue to organize ThingsCon3, an event for
practitioners in IoT to share their skills and lessons learned. Many of the themes we touch
upon in this book are topics of conversation at ThingsCon events around the globe.
Over the coming years, we anticipate many ways to continue these reflections. Be it by
organizing events like ThingsCon and Mozfest,4 by hosting workshops or designing
curricula, by further writing and prototypingwe're excited to make and learn with a growing
community of people interested in the connected home.
There is a lot of work to do as we collectively figure out how to shape the connected home.
It's going to be exciting.
1
. See wiki.mozilla.org/Open_IoT
2. See thegoodhome.org
3. See thingscon.com
4. See mozillafestival.org
104
Acknowledgments
Acknowledgments
Alexandra Deschamps-Sonsino has been blogging about and building products for the
internet of things for over ten years. She has also instigated the Good Home project where
we put the words from this book into action. There's no one to whom we're more indebted for
helping us understand IoT and the connected home.
A huge thank you to Jasmina Tesanovic and Bruce Sterling for their kind hospitality at
Casa Jasmina and truly inspiring project. We are so grateful to play a part in it, however
small and remotely.
Grazie mille to the whole Arduino team especially Lorenzo Romagnoli and Davide Gomba
for hosting us and making us feel so welcome in Torino.
It's also such a pleasure to think about and tinker with connected homes together with Jon
Rogers from the University of Dundee. From facilitating workshops, to inviting us to Fife,
and to always bringing a collaborative, open spirit to everything he does, we can't wait to
keep exploring the future together.
A big thank you for lovely, inspiring and informative chats to Matt Webb, Matt Biddulph and
Tom Coates.
Louisa Heinrich has been doing tremendous work in this field. Conversations with her, as
well as her frequent talks, have been priceless.
Scott Smith and his team at Changeist have been an ongoing source of inspiration.
Especially their research project Thingclash and their outstanding critical IoT reading list are
priceless.
A big shoutout to the Max Krger, Simon Hher, and Emanuel Schwarz as partners in
crime for ThingsCon. Theay all are always thinking about how to make events around this
topic great. An amazing bunch to work with.
Conversations with Alper uun are always insightful and entertaining, but his thinking
around conversational interfaces has been especially helpful. We highly recommend you
seek out his upcoming book on the topic.
Thanks to Prof. Dr. Christoph Bieber. Writing a study and policy recommendations around
smart cities for the German federal government was just the perfect excuse to work together
once more, and the process was both full of learning opportunities and a lot of fun.
105
Acknowledgments
The importance of the work Marcel Schouwenaar and his network of collaborators across
the Netherlands and Belgium have been doing around ethics in IoT cannot be overstated,
from the IoT Design Manifesto to the new Just Things foundation.
106
Contact us
Contact us
This book is a work in progress. We plan to keep adding and revising articles. We always
welcome feedback.
If you'd like to work with the content of this book, it's licensed under Creative Commons (for
details see the introduction). If you'd like to explore a collaboration with us, please get in
touch:
Say hi to us on Twitter: @peterbihr and @thornet.
Contact us via the form provided by Gitbook: theconnectedhome.org.
107

Understanding The Connected Home

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Understanding The Connected Home

Uploaded by

Copyright:

Available Formats

Table

The home as the next augmentative technology

Negotiating with connected systems

Sharing connected objects

Moving in and out of homes

Who controls your home?

Inviting connected objects into our homes

Surveillance & tracking

Policy matters: The connected home and the smart city

Where from here