Barry's Page

Our clients have different requirements with regard to design.  Some want signed-off design documents before starting development others are satisfied with as-built documents and some think even that is overkill.  Who is right?  Or does it depend on circumstances?

Why design?

Everything needs to be designed even if it is just made up on the fly by the developer(s).  Do we want to document the design?

Design is a conversation between the various stakeholders in the project – present and future.  It may take the form of a formal design document, or it may be 3 people standing around a whiteboard.  The important point is that all finish with a shared understanding of what is to be built and why.  The needs of current participants are easily satisfied, but what about future needs?  Does the design collateral help the people that need to maintain/extend the software?  What happens if someone leaves and takes their design understanding with them? 

Advantages of Formal Design Documents

In formal engagements a signed-off design document forms the basis of a contract between the project sponsor and the developers.  The delivered software is tested against the requirements, but code-reviews are matched to the design.  The design can be reviewed in advance to see that the planned approach meets the needs of the project and fits the infrastructure before the expense of building is incurred, and this can avoid costly rework.

Where a large number of different parties are involved (business, infrastructure, networks, application developers, testers etc) a formal design document gives everyone a clear picture of what their tasks are and how they are matched to the needs of the other parties.  it allows a project manager to allocate work and check progress.

For enterprise requirements (e.g. security, performance, non-interference with other projects etc) a design document gives peripheral actors a chance to have their input.  This is important where strong governance processes are in play.  Everyone can have confidence that their needs have been considered before signing off.

The design document provides a place to document the design decisions that were made.  It is important for people who come later to understand the reasons for choosing various options.  This avoids relitigating decisions but also allows a re-examination of the options when conditions change.

Advantages of Informal Design

in smaller projects a design document may not be needed.  A daily standup can provide a forum to discuss any design decisions required.  The outcome should be recorded in the project backlog.  Comments in the code and/or scripts should be used liberally to allow future participants (or current ones with non-exceptional memories) to understand why things are done.  This can provide the needs of documented design without the overheads.  

A decision can be made in the morning and incorporated in code, checked in, built and tested all the same day.  This way the documentation is always up to date.  The documentation is always available as it is saved in the CMDB with the source.  Depending on the tooling used the documentation can often be extracted (e.g. javadoc) and made available at the end of a project without the expense of developing as-built documents.

Where should design be stored?

Often organisations have formal document management or record keeping systems for design documents.  It is often difficult to find the most up-to-date version and often even the most up-to-date version is not updated during the project so doesn’t reflect what was actually built.  This can be a problem for later projects as they have to undertake an expensive discovery phase to find out what the real state is.  Mature organisations can handle this, and it may be correct for them.

In general the documentation should be stored in the CMDB along with the code.  The same versioning/branching/tagging processes that make source code management easy, ensure that the documentation matches the deployed artefacts.  However, they don’t ensure that it is up-to-date.  A design document should always be supplemented by access to code and deployed artefacts.  

A good project-close-out process will have a supplementary document describing what was actually built.  This can be an appendix to the design document describing variations and reasons for them.  This can also be a separate as-built document.  At the least it should be release notes with a manifest of deployed artefacts along with where to find the (commented & versioned) code/configuration items.

Design for Integration

Integration projects have particular needs.  There are always several different products involved.  A clearly defined interface is almost always required up-front.

If new integration applications are required then the project looks like any software project with infrastructure, software, networking requirements.  The organisation’s normal design processes are used.

For service development, the design needs to articulate the different parties involved in the service and how they interact.  Parts of the design (e.g orchestration, mediation) are often dependent on the product set used as they will normally have graphical design tools that match design to implementation.  The interface and data design will usually use some sort of entity design tool like Sparx EA.

It is useful to have a catalogue of services in use and how they interact.  This can be a spreadsheet, but often the linkages are more involved and a 2 dimensional table cannot easily represent them.  A graph database may be an option to capture all the dependencies.

Conclusion

There is no right answer for how to document design.  That is no excuse for assuming that no design is necessary.  Sometimes it is only after the project is finished (and the next one underway) that you know whether your design strategy has been successful.

One thing is always true.  Design is never “finished”.  Even in a waterfall process with signed-off designs before coding starts there will always be further design decisions to be made as the project proceeds.  the design process should recognise this and make sure that whatever is produced is still of value at the end of the project.

The external consultant is in an unenviable position.  We come from outside an organisation that is not performing well and tell them how to fix their problems.  In so doing we come up against politics and personalities.  We give busy people more work.  We threaten fiefdoms and islands of incompetence.  Sometimes we find people that are desperate for solutions to problems that are not in the scope of the consultancy.

Why do we do it?

• We want to make a difference.
• We have skills to offer that most companies don’t need to have on their staff.
• We learn a lot from our clients.
• We get a variety of work that we couldn’t get otherwise.
• We meet lots of people.
• We travel and experience different environments.
• We get to work within deadlines.

A consultant can help a company with a particular aspect of their business that is struggling or not making progress.  We start by getting to know the clients and how they operate.  We talk to as many people as we can and devour the contents of their content management systems.  Right through the engagement we are still learning new things about the business, the management, the roles and the landscape.  We document the weaknesses (and strengths), agree a goal state and formulate a plan to get there.

For us as integration specialists it is usually about data, and how it is handled, managed, owned, maintained, transformed, transferred, described, synchronised, cleaned, and maybe even disposed of.  We come into organisations where integration is an expensive afterthought and leave them with an understanding that it needs to be baked into projects from the start.  We help to imbue them with an “API first” mentality that means that no software is purchased or developed unless it comes with integration points and a clear data model.

Our first responsibility is to satisfy the goals of the client for the engagement.  Often, however, we are able to suggest additional goals that are of great value to them.  Most clients are only too aware of their shortcomings when they come to us.  We are often able to point out strengths that they didn’t know about.  We focus on the outcome rather than worry too much about what people are doing wrong.  If we get the process right, people will find the right way easier than any other.

“Getting buy-in” and “bringing people along for the journey” are clichés but they describe accurately our way of operating.  We need to be aware of what people want from their work and help them achieve it.  Even the most disengaged employees can be enthused when they see that they are listened to and can help to make a difference.  The engagement is not only about the CxOs (although they pay the bills).  A successful engagement leaves the organisation healthier and the staff happier.

So if we are invited into your workplace come and say hello.  We will listen to you and see what we can do to make your work more effective and more valued.

The next certification I plan to do (before the end of August) is the IBM cloud architect exam.  I have a free voucher for the exam.

http://www-03.ibm.com/certify/tests/eduC5050-280.shtml

This is partly general cloud knowledge but to make it harder it needs quite detailed knowledge of the IBM cloud management products and the IBM CCMP-RA (Cloud Computing Management Platform –  Reference Architecture).

So I shall be looking at TSAM, TUAM, ISDM and Cloudburst even though I don’t think I will ever use them in anger.