Professional Documents
Culture Documents
Given: Dinesh
Family: Mohata
Job Title: Senor Consultant
Email: dinesh.mohata@in.ibm.com
Dinesh Mohata is a Senior Consultant in the Oracle Retail Practice within Distribution Sector.
The practice is responsible for providing project management, end to end services in retail
industry, and business solutions. Dinesh has over 9 years of IT & consulting experience in
software design, development, deployment and testing. He has retail industry experience of
over 5 years. Dinesh has played a role of Technical Architect.
His areas of interest include Agile Development Methodology, Design Patterns &
Implementation Data Cutover. He has earlier published a paper “Application Agility: A Design
Perspective” with Oracle. Dinesh can be reached at dinesh.mohata@in.ibm.com
Abstract:
This paper details out various Agile Software development methodologies like XP, SCRUM,
RUP etc. The second half of paper details out agile approach taken in one of the real life
projects. It covers the background of the project, approach taken, benefits gained, challenges
and conclusions. This paper will benefit the audience in providing varied perspective for agile
adoption. The tailoring done to the agile approach for data intrinsic development will definitely
provide users with some tips to take and may implement in their projects as well.
Page 1 of 14
Agile Approach in Data Intrinsic Development
Page 2 of 14
Agile Approach in Data Intrinsic Development
Table of Contents
1. Introduction............................................................................................................................ 3
2. Agile Software Development Methodologies..........................................................................3
2.1. Extreme Programming (XP)........................................................................................... 3
2.2. SCRUM......................................................................................................................... 4
2.3. Agile Modeling (AM)...................................................................................................... 5
2.4. Agile Data (AD).............................................................................................................. 5
2.5. Rational Unified Process (RUP)....................................................................................5
2.6. Open Unified Process (OUP)......................................................................................... 5
3. Agile Approach....................................................................................................................... 5
4. Case Study: Traditional to Agile Approach in Data Intrinsic Development..............................6
5. Benefits Gained.................................................................................................................... 11
6. Challenges........................................................................................................................... 12
7. Conclusion........................................................................................................................... 12
8. References........................................................................................................................... 13
Page 3 of 14
Agile Approach in Data Intrinsic Development
1. Introduction
Extreme Programming (XP) is the most well-known agile methodology. XP is based on a series of
concepts that include: having the business customer on-site, pair programming, collective code
ownership, continuous code integration, small releases, designing tests before writing code,
standup meetings, refactoring, and 40-hour work weeks [2].
Disciplined agile software delivery processes are Dynamic System Development Method
(DSDM), Open Unified Process (OpenUP), and agile instantiations of the IBM Rational Unified
Process (RUP) process framework. All of these methodologies function differently from traditional
SDMs and help organizations meet the challenges of the digital economy [1].
This paper covers the background of the project, approach taken, benefits gained, challenges
and conclusions.
XP is a collection of practices for software construction, include refactoring, test-first design, pair
programming, on-site-customer, continuous integration, whole team, and collective ownership.
XP Practices
Page 4 of 14
Agile Approach in Data Intrinsic Development
2.2. SCRUM
The focus of Scrum is project leadership and requirements management. Scrum proponents
claim that it is a process framework, but if so then it is a sparse and incomplete one at best. A
more accurate description is that it is a high-level lifecycle for construction iterations (what Scrum
calls “sprints”), see Figure 1, and several practices such as daily stand-up “Scrum” meeting,
product owner, product backlog, iteration/sprint planning, and potentially shippable software.
Scrum Approach
Accept:
o This is a complicated, unpredictable world and development environment.
Page 5 of 14
Agile Approach in Data Intrinsic Development
o You can't predict or definitively plan what you will deliver, when you will deliver it,
and what the quality and cost will be.
Assume:
o You can estimate, and then negotiate according to various risks as you proceed.
Understand:
o Following any cookbook approach won't improve the definition of "best", and will
only hinder appropriate responsiveness to the complexity and unpredictability.
o the delivery will be the best possible given the circumstances
The above Scrum Approach content has been referenced from http://www.controlchaos.com/old-
site/philo.htm
RUP is a comprehensive process framework for iterative software delivery which can be
instantiated anywhere from a very agile form to a very traditional form as your situation warrants
[6]. RUP practices include risk-value lifecycle, whole team, test-driven development (TDD), and
business process. See http://www-01.ibm.com/software/awdtools/rup for details.
3. Agile Approach
Agile Approach has to be tailored to the needs of the project one is working upon. Agile process
adopted in this project was culmination of various precepts pronounced in various agile SDMs.
Page 6 of 14
Agile Approach in Data Intrinsic Development
Various precepts applied in the current work with respected to Agile SDM were as follows:
Requirements Envisioning
Redefining Stake Holder Participation
Incorporation of Late Requirements
Work Prioritization
Continuous Integration
Pair Programming
Refactoring
Cross Training of whole team
Applying TDD
Technical Excellence
The above approach is explained in detail with all due examples in the subsequent case study
section of this paper.
Retail is all about business agility. Requirements change on the fly. To beat the competition
retailer has to change the dynamics of its organization at rapid pace. This requires systems and
processes to be agile. IT has to be an enabler for this happen. This in turn requires IT systems to
follow sound agile process to deliver value to the customer in the minimum time possible. Retail
being highly margin sensitive area retailer has to capture lot of data to optimize its pricing engine
to get the best value to the customer and in turn protect its margins. This requires huge data
churning within the retail organization. This required defining and proving the agile processes
work in packaged implementation. Moreover, the sound agile processes adopted during the
implementation will become the sound backbone for the retailer for future.
This section details about the various facets of agile software development applied for this
retailer. The tenets of agile development have been picked and experience around this has been
penned down in the paragraphs below.
Requirements Envisioning
Page 7 of 14
Agile Approach in Data Intrinsic Development
Agile approach in traditional environment is really tough. People are not willing to understand the
agile approach as such. Customer is not fully aware of the approach and has no willingness to
here it as well. Approach taken was internally we will satisfy our customer i.e. functional
consultants using agile approach. Agile approach was to internal customers of the project team.
Stake holder for the development team is the internal teams which certify the product before
releasing to the external stakeholder.
Initial requirements were prototyped. The design was envisaged keeping in mind the changes
coming late in the project to be incorporated. Each report was driven by an Oracle Database
View. Report were predefined by customer in ms excel templates. These templates were
uploaded in the database. Any additional column required a change in template and subsequent
upload a CLOB database object type. Since each report was based of database view it was just
matter of adding the additional column in the view from the table already in use in the view
definition.
Work Prioritization
Changing the stakeholder definition to internal customer helped team to prioritize the work. Entire
team was co-located and any suggestions on work priority were discussed right at the moment to
decide team’s future course of action. Real stakeholders were intimated on the details of the work
scheduled for the team. Communication to real stakeholders was done only after having internal
stakeholder approval.
Continuous Integration
Continuous integration was key approaches adopted in the data intrinsic development. In all the
instances the code was deployed based on few iterations of develop, build and test cycle. The
development was done based on requirements provided by the customer. The requirements were
then prioritized. Based in these prioritizations the development would work with iterations of
development. The end product was continuously integrated with the bigger system to ensue this
integration does not break the larger system. This resulted in smooth transition during the
movement of products in production environment.
There was an instance wherein application was developed in the corporate environment using
intranet network. This application was supposed to be deployed at a remote location for the
customer. This application was working perfectly fine when the server was accessed using the
local area network (LAN). But application just was not working when deployed in the remote
location. The connectivity to the server was not appropriate for certain protocols required in the
application. This was a serious issue since without opening up this protocol in all the routers to
server this application wouldn’t work. Design was re-organized to attain this in two step process.
This design was also thought through with customer in mind. The process using the protocol was
replaced with a two step process. The first step was making a direct database entry of the data to
be transported from local client to the server. The second step was creation of the file at the
server side using database server file utilities. These two steps isolated the protocol error
occurring in the application. Cost of change was quite less since this kept the other steps in the
application intact and re-designed a portion of the application process.
During the one of the demonstrations of the code developed during the project the agile approach
was quite handy. Two team members from the team were executing the demonstration to the
customer. One of programmers was displaying the features of the code and performing the initial
Page 8 of 14
Agile Approach in Data Intrinsic Development
tests for the customer on the developed application. This demonstrator was connected to the
projector for the entire audience. Another programmer was sitting just beside and noting the
comments of the audience. There was couple of enhancements requested by the customer
during this session. While the first demonstrator was demonstrating the application, the second
developed already coded these new features in the code and deployed the same code during the
recess break of the session. Post the recess break once more the application walkthrough was
conducted. This walkthrough resulted a huge success maximum of changes requested by the
customer was successfully implemented and demonstrated within a very short duration.
Pair Programming
There was instance in the project were in we had to deploy an unknown application. This
definitely required some initial training to get the job done. Customer arranged for online CBT
training but this did not help. We did form a two member team to get the job done. There was one
programmer doing the typing effort and another person religiously followed the application
deployment guideline to guide the typist. This combined development effort paid off and we were
successfully able to implement the application in very short duration. As a side note I would
emphasize that this required good resources with a good attitude to get the job done.
All the members in the team were provided an opportunity to work within other modules of the
project. The whole team was collocated in a single venue. Team sitting in a single location
actually helped in various ways. Discussions between team members of various modules by and
large happened in the same room. This resulted in each and every individual having a fair amount
of idea on each module being implemented. As in IBM servers, it applies to humans as well. Not
all the time each individual is fully engaged. IBM virtualization concept utilizes spare processing
power from servers in the stack which are not in use. Applying the same concept in the co-
located, it helped in filling any immediate requirement from a module which would require some
extra effort for any dynamic requirement. This in turn also gave some amount of knowledge to the
cross team member who approached to solve the issue.
Page 9 of 14
Agile Approach in Data Intrinsic Development
ADD A TEST
Pass
Fail
Pass
WRITE/REFACTOR CODE
Fail
Add a test
In test-driven development, each new feature begins with writing a test. This test must inevitably
fail because it is written before the feature has been implemented. In order to write a test, the
developer must understand the specification and the requirements of the feature clearly. In all the
developments developer was provided with use cases and stories to cover requirements and
exception condition. This made developers aware of the requirement before writing the code.
An instance of test in current scenario wherein a development of Retail application for financial
accounting was developed. User story of this application was beforehand given to the developer.
The requirement was to develop application that sends data on daily basis of certain attributes
Page 10 of 14
Agile Approach in Data Intrinsic Development
and thereafter updation of flag at the subsriber end. The test condition was written by the
developer stating the attributes of the database required to pull the data from. If any of the
columns required fo certain table does not exist then this test condition will fail. The code had to
be rewritten to take the appropriate column from the table for the test to pass.
This validates that the test harness is working correctly and that the new test does not mistakenly
pass without requiring any new code.
The new test should also fail for the expected reason. This step tests the test itself, in the
negative: it rules out the possibility that the new test will always pass, and therefore be worthless.
The next step is to write some code that will cause the test to pass. The new code written at this
stage will not be perfect and may, for example, pass the test in an inelegant way. That is
acceptable because later steps will improve and hone it.
It is important that the code written is only designed to pass the test; no further (and therefore
untested) functionality should be predicted and 'allowed for' at any stage.
If all test cases now pass, the developer can be confident that the code meets all the tested
requirements. This is a good point from which to begin the final step of the cycle.
Refactor code
Now the code can be cleaned up as necessary. By re-running the test cases, the developer can
be confident that refactoring is not damaging any existing functionality. The concept of removing
duplication is an important aspect of any software design. In this case, however, it also applies to
removing any duplication between the test code and the production code.
Repeat
Starting with another new test, the cycle is then repeated to push forward the functionality. The
size of the steps can be as small as the developer likes, or get larger if s/he feels more confident.
If the code written to satisfy a test does not fairly quickly do so, then the step-size may have been
too big, and maybe the smaller testable steps should be used instead.
The first step is to quickly add a test, basically just enough code to fail. Next you run your tests,
often the complete test suite although for sake of speed you may decide to run only a subset, to
ensure that the new test does in fact fail. You then update your functional code to make it pass
the new tests. The fourth step is to run your tests again. If they fail you need to update your
functional code and retest. Once the tests pass the next step is to start over (you may first need
to refactor any duplication out of your design as needed, turning TFD into TDD).
Page 11 of 14
Agile Approach in Data Intrinsic Development
Technical Excellence
Based on the requirements the report development was started developing the skeleton or
framework for one of the reports. Brainstorming was conducted on technical challenges of the
report and work for the same started. The working report was ready in a day. There were issues
of performance in the report as far as the data display in the screen was considered. The data
was displayed in the spreadsheet on a cell by cell basis. Oracle’s Webutil technology was
leveraged to write data in the excel sheet.
Following steps were taken to foresee any bottlenecks and combat well in advance
Since each report was based on a database view, SQL tuning the underlying view
database query was indispensable.
Server Statistics report collected in daily basis check the server health and apply
appropriate check if necessary
Each SQL session and application session was traced. Oracle database utilities were
used to analyze the performance impact of session parameters
This approach of continuous integration and collective ownership as a team was adopted to
benchmark the technical approach in an agile way.
5. Benefits Gained
Page 12 of 14
Agile Approach in Data Intrinsic Development
By following the agile-tional approach to software development for data intrinsic development
following was gained
Design and Build of reports were prototyped resulting in creation of working software
before discussing requirements further with customer
Effort spent on creating this working software was restricted to two days only
New alternatives/suggestions were advised to customer on first meeting itself
Realistic expectations were set for the customer from day one based on the technical
limitations of the development
Functional fitment done well in advance because of requirements envisioning
Ability to take any additional touch-ups on the code as along as the framework of the
development does not change
Customer was very satisfied on the approach of creating prototype in initial phase
Requirements late in the project were easily incorporated
The working reports were used by the customer for extensive data analysis from their
end.
They used the reports to further verify the data in their system
On the fly change during presentation using Pair Demo as part of continuous integration
was just excellent and was highly appreciated by the customer
Pair Programming resulted in huge customer satisfaction
Quick Results
High Quality standards were set through refactoring codes
Good Clean code that works
Agility in the Software development process
Huge confidence in the development team
6. Challenges
There were many challenges faced in journey of applying agile-tional approach to development
It was difficult to make team agree on the agile approach since the school of thought
was more towards waterfall model
Getting the buy in from the Project Management was tough and hence workaround
stakeholder idea was devised
Technical Challenge of publishing an excel report from Oracle 10g server
Performance of the reports in general and this was dealt with continuous monitoring
of database and application
Regression Testing of deliverable upon incorporating late feedback
7. Conclusion
Software development can be done in agile way bringing delight to the customer as an end result.
It provided quick turnaround time to deliver quality product to the customer. Willingness from the
all stakeholders is indispensable for successful agile methodology implementation. Requirements
late in the game can also be incorporated with agile SDM. Customer is fully aware of the end
product getting delivered to him.
Agility is more of perspective. It’s really just a mindset. It’s about adaptability to changing
business requirements. Based on the points highlighted in this paper, and supporting points
given, it could be very well stated that agile SDM can be adopted in an organization whilst
Page 13 of 14
Agile Approach in Data Intrinsic Development
providing huge benefits and requiring few cultural changes that will greatly impact the
implementation.
8. References
1. Boehm, B. & Turner, R. Management challenges to implement agile processes in traditional
development organizations. IEEE Software. 22(5), 30-40. 2005.
2. Theunissen, W., Boake, A., & Kourie, D. In search of the sweet spot: Agile open collaborative
corporate software development. Proceedings of the 2005 Annual Research Conference of
the South African Institute of Computer Scientists and information Technologists on IT
Research in Developing Countries. White River, South Africa. 268-277.
4. Holmstrom, H., Fitzgerald, B., Agerfalk, P., & Conchuir, E. Agile practices reduce distance in
global software development. Information Systems Development. 23(3), 7-18. 2006.
6. Kroll, P. and MacIsaac, B. (2006). Agility and Discipline Made Easy: Practices from OpenUP
and RUP. Boston: Addison-Wesley
Page 14 of 14