how (8 years of) eclipse changed my views on software

development

Erich Gamma
IBM distinguished engineer
IBM rational zurich research lab

© 2008/2009 IBM Corporation

Outline

Open
Source

2000 2001 2002 2003 2004 2005 2006 2007 2008

Fall June November June March June June June June June
Project Tech 1.0 2.0 2.1 3.0 3.1 3.2 3.3 3.4
Starts Preview

Closed Development Open Development

Built to Last Community Development Process Team

Extensibility Transparency Agility Tools
API

Eclipse Lessons
how buildings last

• Stewart Brand: how buildings learn

– what happens after they're built
• stuff: furniture
• services: electrical, plumbing (7-15y)
• structure: foundation, load bearing walls (30-300y)
• site: geographical setting (forever)

• layers:
• evolve at different rates during the life of a building Site
• shear against each other as they change at different rates
• an adaptive building must allow slippage
¾ a building that lasts is adaptive and can change over time
¾ lasts for generations without total rebuilding

© 2005 IBM Corporation

structure foundation

ƒ the eclipse plug-in architecture

ƒ everything is a plug-in
ƒ simple and consistent

© 2005 IBM Corporation

eclipse plug-in architecture
ƒ plug-in == component
plug-in
ƒ set of contributions
ƒ smallest unit of Eclipse function
ƒ details spelled out in plug-in manifest
ƒ extension point – named entity for collecting plug-in
contributions
ƒ extension – a contribution
ƒ Example: a specific spam filter tool
ƒ runtime – controls and manages contributions
platform

runtime
Extension
Extension point

© 2005 IBM Corporation

services plumbing: APIs

ƒ Plug-in dependencies through APIs

ƒ define APIs for stability
ƒ binary compatibility is highest priority

© 2005 IBM Corporation

APIs first

ƒ APIs don’t just happen; we need to design them

ƒ specifications with precisely defined behavior

ƒ what you can assume (and what you cannot)
ƒ it works ≠ API compliant
ƒ documented classes ≠ API

ƒ must have at least one client involved, preferably more

© 2005 IBM Corporation

practical extensibility

ƒ extensible in ways that are known useful

ƒ needed by us, requested by others
ƒ we don't provide hypothetical generality - we want to be extensible in ways that
matter
ƒ don't over generalize

© 2005 IBM Corporation

stuff, furniture - UI

ƒ eclipse extension architecture is contribution based

ƒ extensions contribute to the workbench
ƒ the workbench manages and presents the contributions
ƒ enables UI evolution
ƒ 3.0 new look

© 2005 IBM Corporation

 Key Lessons
ƒ Modularity matters
ƒ Everything is a plug-in
ƒ “no exceptions”
APIs are a huge commitment
ƒ we would rather provide less API than desired (and augment)
than provide the wrong (or unnecessary) API and need to
support it indefinitely
ƒ the tyranny of stable APIs
ƒ API layers…
ƒ the challenge of product developers
ƒ which API level does our product require and support
ƒ n–1, n-2

10 © 2008 IBM Corporation

 Eclipse 3.4 API Tools

ƒ Support to define an API baseline

ƒ e.g. Eclipse 3.3 when working on
3.4
ƒ Check access restrictions
ƒ API javadoc tags:
@noimplement, @noinstantiate,
@noextend
ƒ Detect binary compatibility
violations
ƒ Detect version problems
ƒ @since
ƒ Problems are reported during
builds

11 © 2008 IBM Corporation

 stable API evolution patterns

ƒ compatibility layer
ƒ I*2 extensions interfaces
ƒ extension interface: IAdaptable
ƒ restart in a new package

12 © 2008 IBM Corporation

 extension interfaces: IAdaptable
IPersistable p= (IPersistable ) o.getAdapter(IPersistable.class);
p.saveState(…);

ƒ supports adding interfaces to existing types

ƒ Interface negotiation

<extension point="org.eclipse.core.runtime.adapters">
<factory
class=“someAdapterFactory"
adaptableType=“someType">
<adapter type="IPersistableElement"/>

13 © 2008 IBM Corporation

 Outline
Open
Source

2000 2001 2002 2003 2004 2005 2006 2007 2008

Fall June November June March June June June June June
Project Tech 1.0 2.0 2.1 3.0 3.1 3.2 3.3 3.4
Starts Preview

Closed Development Open Development

Built to Last Community Development Process Team

Extensibility Transparency Agility Tools
API

14 © 2008 IBM Corporation

How we Started: Closed
development

ƒ The Swiss Bank approach

to software development
 If it hasn't shipped
it doesn’t exist

ƒ Strong firewall between

developers and customers
 Eclipse Nov

Shipping Matters… 1.0

Eclipse
2001
June
2.0 2002
Eclipse Sept
2.0.1 2002
our objective is to ship software
Eclipse Nov
ƒ anything that contributes to this goal is good 2.0.2 2002
ƒ anything that does not is bad Eclipse Mar
2.0.3 2003
Eclipse Mar
2.1 2003
team member recognition is based on the ability to
ship Eclipse June
2.1.1 2003
ƒ quality software Eclipse Oct
ƒ on time 2.1.2 2003
Eclipse Feb
2.1.3 2004
the culture: "If you ship, then you may speak." Eclipse June
3.0 2004
the question: "Did they ever ship anything?" Eclipse Sept
3.0.1 2004
the insult: "They never ship anything!"
Eclipse Mar
3.0.2 2005
Eclipse Jul
3.0.3 2005
Eclipse June
November 2001: “Open Source”
Reaction from the development team
You want us
to do what?

Why are we doing this

again?

Code in public?

Answer all those

dumb questions?

Have technical
discussions in public?
Key Lessons
ƒ Transparency helps existing
development
 Better understanding of current
status
 Responding to feedback takes time,
but pays off Transparency
ƒ Use same communication channels
inside as outside Developers

¾Not limited to Open Source projects

Community
¾ “Open Commercial Development”
Feedback
and Support
Open Commercial Development
ƒ Open Commercial Development is more than publishing the
source code
 Open, transparent process, from feature requests and
planning through delivery
ƒ What can the community members do:
 Download milestones, try them, and provide feedback on
betas and incubators, including source code
 Access, Create, and update defects
 Access milestone and component iteration plans
 Access the development wiki
 Participate in discussions on the development community
newsgroups
ƒ Example: www.jazz.net
Outline
Open
Source

2000 2001 2002 2003 2004 2005 2006 2007 2008

Fall June November June March June June June June June
Project Tech 1.0 2.0 2.1 3.0 3.1 3.2 3.3 3.4
Starts Preview

Closed Development Open Development

Built to Last Community Development Process Team

Extensibility Transparency Agility Tools
API
The Eclipse Way Practices
continuous consume your
continuous
integration validate own output
testing
enable
sign drive with
off open eyes
reduce stress community
end transparency involvement
game
milestones attract
live show progress to latest
first
always have betas
feedback new &
a client update learn noteworthy
validate

API adaptive
component
first planning
centric enable retrospectives

explore

dynamic common agile practices

validate
teams common Open Source practices
scaling-up practices

Eclipse Lessons
Our Expanded Practices Today
continuous consume your
continuous
integration validate own output
testing
enable
sign drive with
off open eyes
reduce stress community
end transparency involvement
game
milestones attract
live show progress to latest
first
always have betas
feedback new &
a client update learn noteworthy
validate

API adaptive
component End of iteration
first planning
centric enable retrospectives demos/reviews
Product Backlog
explore
Burndown Stories
dynamic
Daily Standup validate
teams Adoptions
Expectations PMC
Buddy Review
Eclipse Lessons
What is behind the Eclipse Way
ƒ Practices underpinned with values
 ship quality on time
ƒ Used, developed and improved over time
 A mix of practices that worked for us
 Another mix of practices works for others
ƒ Practices are from all kinds of sources
ƒ XP, Scrum, Crystal Clear, RUP, …
ƒ Patterns - Organizational Patterns of Agile Software Development – Coplien
ƒ It is not low ceremony
 Approvals, verifications, reviews
ƒ It is agile: incremental, iterative, collaborative, transparent, customizable
 And it scales up

Eclipse Lessons
In the Past…

Say goodbye to your loved ones

Look at calendar.

Effort/
Pain
Level

Heads down

Exhaustion

All the time in the world

Time

Eclipse Lessons
Iterative – No hanging rope

fitness

t
In the 3.1 3.2
no “hanging rope”
past
⇒ stress reduction

Eclipse Lessons
 3.1
decompression

retrospective
warm-up
initial release plan
plan
release 3.2
M1

develop
Iterative – Time-boxed

6 weeks
stabilize
plan
sign-off

M2

fitness
develop
6 weeks
stabilize
plan
sign-off

develop
6 weeks

stabilize
fix - spit & polish
test
fix
endgame

test
Eclipse Lessons
sign-off
Continuous…
ƒ It is all about being continuous
 Continuous iterative and adaptive planning
 Continuous design/refactoring
 Continuous integration/testing
 Continuous delivering/demos
 Continuous feedback
 Continuous learning

¾ Many effective teams work like this

Eclipse Lessons
Key Lessons
ƒ Agility can scale up to globally distributed teams
ƒ … but it would be nice to have better tool support

Eclipse Lessons
Outline

Open
Source

2000 2001 2002 2003 2004 2005 2006 2007 2008

Fall June November June March June June June June June
Project Tech 1.0 2.0 2.1 3.0 3.1 3.2 3.3 3.4
Starts Preview

Closed Development Open Development

Built to Last Community Development Process Team

Extensibility Transparency Agility Tools
API

Eclipse Lessons
Reflections from an Eclipse PMC Lead

ƒ Many Eclipse observers do not realize the

amount of work/stress/human factor involved in
shipping on time every time.

It reminds me of the duck quietly advancing on

the pond, but no one realizes that he is paddling
like crazy underwater.

Philippe Mulet – former Eclipse PMC Lead

Eclipse Lessons
But… there are Pain Points…
ƒ joining a team
ƒ get my environment configured to be productive
ƒ what is happening in my team
ƒ collecting progress status Collaboration
ƒ following the team’s process
ƒ ad hoc collaboration/sharing of changes
ƒ starting an ad hoc team

ƒ is the fix in the build?

ƒ what will be in the next build? Development
ƒ tracking a broken build
ƒ Avoid breaking a build/personal build
ƒ why is this change in the build?
ƒ reconstructing a context for a bug/build failure

ƒ creating, tracking iteration plans

ƒ interrupting development due to a high priority bug fix
Boring and painful
ƒ working on multiple releases concurrently
ƒ tracking the code review of a fix Project
ƒ referencing team artifacts in discussions Management
ƒ how healthy is a component?
ƒ collecting project data/metrics?

Eclipse Lessons
Why are we doing Jazz?

Late
Late 90’s:
90’s: Focus
Focus on
on Point
Point Tools
Tools
Who can build the best Java IDE, the best C IDE, the best Web Tool,…

When
When we
we built
built Eclipse:
Eclipse: Focus
Focus on
on One
One Developer
Developer
Seamless integration across a set of tools to improve the productivity of one developer

Today,
Today, we
we must
must focus
focus on
on the
the Team
Team and
and its
its Collaboration
Collaboration
ƒ Geographically Distributed
ƒ Accelerated Delivery Demands Seamless
Seamlessintegration
integrationacross
across
all
all the Phases of the SoftwareLifecycle
the Phases of the Software Lifecycle
ƒ Agility with Predictability totoimprove
improve
ƒ Innovation and Repeatability the
theProductivity
Productivityof
ofthe
theEntire
EntireTeam.
Team.
ƒ Increased Need for Transparency

Eclipse Lessons
Jazz and Team Concert

Eclipse Lessons
Team Concert
ƒCollaboration in Context
ƒTeam First
ƒProcess
ƒTransparency

Eclipse Lessons
Collaboration in Context

Eclipse Lessons
Collaboration with Context

Eclipse Lessons
Collaboration: Builds
ƒ Scoped builds
 Personal builds
 Continuous team builds
 Scheduled integration builds
ƒ Collaborate in context on build failures

Eclipse Lessons
Rules/Process
ƒ All collaborations have underlying
context specific rules
ƒ Each project follows a
process
ƒ Team’s own their process

Eclipse Lessons
Process
ƒ Process manifested through:
artifact types and their states
Scrum
role-specific preconditions
and actions on operations
manipulating artifacts Eclipse Way
role-specific and permissions

OpenUP

Eclipse Lessons
Transparency/Visibility: Dashboards
ƒ At a glance view on team artifacts Viewlets
ƒ Three dashboard types:
 project
 team
 individual user

Eclipse Lessons
A Personal Dashboard

Eclipse Lessons
A Team Dashboard

Eclipse Lessons
Team of Teams Dashboard

Eclipse Lessons
Team of Teams Dashboard

Eclipse Lessons
Jazz Project Self-Hosting since Oct 2006 Ottawa
Toronto

•Source Control
•Reporting
Jazz
•Community Site
Development
Server
Bangalore

Beaverton
•VS Client

Lexington
Zurich
•Build
•Process
Raleigh
•Source Control
•Requirements •Work Items
•Interop •Agile Planning
•Code Coverage
•Repository
•Web UI
Eclipse Lessons
 Summary: Our Journey
Eclipse Way

Try it yourself on www.jazz.net

Eclipse Lessons