10 Jul

Why Agile methodologies are good fit for ETL development?

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

Benefits of Agile methodologies have been realized in software product development as this approach attempts to break the complexity of software development by frequently(regularly) releasing working software.

As this model aligns itself with the ever changing software requirements, these methodologies are increasingly adopted in software engineering. There are various methodologies which fall under Agile umbrella, such as, DSDM, ASD, SCRUM, Extreme Programming. This article will refer to  Scrum while discussing ETL development in conjuntion with Agile.

The challenge for Scrum master and product owners is to define the scope of a sprint, which produces a working model at the end of it. ETL development comprises of logical pieces of components (ETL jobs), which makes it convenient to plan a number of ETL jobs in a sprint.

A sprint is normally of two or three weeks of duration. On the basis of the number of resources available at hand, the scrum master can plan a sprint which is realistic and achievable.  This is not being suggested here that Scrum or Agile can totally replace all phases, e.g. requirements for ETL normally flows from the “upstream” systems, but build phase is an area which can definitely be exploited.

Many would see this as a challenge that how one can follow the various approaches(SDLC and Agile) at the same time. What is being advocated here is that these methodologies are not intended to replace another, rather they should be used judiciously to leverage the pluses available with them.

A scenario: A software vendor wins a contract (may be Fixed price) to deliver an ETL project. They follow the traditional approach for requirements and high level design phases. During the build phase, they can internally use the Agile principles to drive the ETL development. The advantages are:

1. better planning as planning for less.

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

2. better estimation.

3. better use of lessons learnt, as lessons learnt from one sprint can be applied to another.

4. minimizing the risks as the working software is rolled out from early stages, providing an opportunity to early identification of errors.

5. A chance to get customer feedback early.

The planning for scheduling software can also be left to later sprints as there would be useful information available in form of jobs and dependencies which would again help in planning more effectively.

in a T&M model, using Agile for ETL development is even better as the customer can find it easy to relate with number of jobs available in a sprint. This in turn will help in early feedback, confidence building of customer as he sees the product much earlier and can provide invaluable feebback.

The Agile methodologies bring the team together and provide a platform to enagage the customer. Combine with the ease of use in ETL development and you can develop a project management model which is effective.

09 Mar

Agile Manifesto

Learning never ends...

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

 Agile Manifesto

Twelve principles underlie the Agile Manifesto, which includes:

Customer satisfaction by rapid delivery of useful software

Welcome changing requirements, even late in development

Working software is delivered frequently (weeks rather than months)

Working software is the principal measure of progress

Sustainable development, able to maintain a constant pace

Close, daily co-operation between business people and developers

Face-to-face conversation is the best form of communication (co-location)

Projects are built around motivated individuals, who should be trusted

Continuous attention to technical excellence and good design


Self-organizing teams

Regular adaptation to changing circumstances

03 Mar


Agile methodologies have been primarily used to help in product development, but slowly this trend is catching for software project management as well. These methodologies are driven on the principles of Agile Manifesto.

The Agile Manifesto highlights over:

Indiviudals rather than processes.

Working software

Customer collaboration

Response to change

These methodologies address the one basic problem which causes the many projects to fail, that is involvement of the users/stakeholders. Waterfall is predictive, Agile is adaptive and the Risk is minimized by frequent release of working software.

Agile methodologies comprise of Extreme Programming, DSDM etc.

Plugin from the creators ofBrindes Personalizados :: More at PlulzWordpress Plugins