Evolving Processes – Part 1

Since I started working at collaboration Factory AG in the beginning of 2015, our software development process has changed quite a bit. In this post I want to outline where we started, what challenges we faced, and which solutions we tried to employ to solve those. Very likely, lots of companies have come across these issues, too, and we are still far away from perfect as you will see in the end. I would love to hear your feedback and experiences as well as the measures you have taken to improve and streamline your workflow.

Starting Up

By early 2015, the company in total comprised less than ten people. As such, the development team was small with only 4 people, too. Everyone of us basically knew our product very thoroughly – there were some but comparably few core components, projects, and especially only a few customers to deal with. Therefore, we were in a sense omniscient – we knew everything about our code, features, and requirements.

It starts simple - small, omniscient team

It looks – and really was – simple and allowed us to very rapidly develop the product, implement new features, or react to changing customer requirements and feedback. Since the team fit in a single room and of course talked with each other during the day, every piece of information like what had just been implemented immediately spread and was shared. At the core the process solely consisted of a very flexible, self-organizing team.

Then came Growth

However – or better: gladly – growth set in, as a small, startup-like company eagerly hopes:

  • new developers and consultants came on board of the company
  • the product grew with a rapidly increasing feature set
  • the number of core components, projects, and especially customers largely increased

Apart from all the positive factors this growth caused some problems regarding our so-far lightweight development process. Team members worked on various numbers of projects and core components in parallel to increase the number of out-of-the-box features as well as adapt our product to specific customer requirements. Would you have drawn a diagram – it probably would have looked like below, not simple anymore.

It gets complex - more projects, customers, and team members

Quite quickly, one of the key advantages of the small, coherent team started to fade: it became a lot harder to share knowledge among the members of the team. A main driver of this problem were the customer projects since they typically require a lot of domain specific knowledge. It would not have been efficient – if at all feasible – for every team member to know every detail from a customer point-of-view.

Real Agile to the Rescue (?)

To tackle these issues we decided to take a more or less proper and Scrum-like approach. Why just Scrum-like? Despite introducing the Sprint concept, tracking a Backlog, and doing a Planning every three weeks, we did not fully enforce it because it was not really possible.

What is not enforcing a Sprint? In our case, it was hardly possible to properly plan the upcoming Sprint in the Planning meeting. The development cycle and deployment dates for customer projects were not in sync with our internal Sprint track. Thus, requirements changed frequently – especially in prioritization – bugs had to be fixed or deployments had to be built and delivered. The core concept of having an un-interrupted, committed to Sprint with focused work was constantly violated. Furthermore, developers were still separated project-wise and such information did not spread as expected.

While providing a good framework with rough rail guards, Scrum itself just did not fit for us at that time. Really enforcing it would have cost flexibility that was necessary business-wise to fulfill customer requirements to get and keep them onboard.

Battling the Odds

Our decision was to keep the idea of a Sprint – the fixed time frame and a regular Planning meeting every three weeks. The focus of Planning shifted to a couple of things:

  • a quick recap of the last Sprint and how every project had evolved – i.e. some kind of Retrospection
  • prioritizing core issues of our standard platform
  • doing a rough resource planning to decide who works on which project to what extent
It is dealt with - plan resources, lower interruptions

Project specific details were discussed face-to-face with our consultant working together with the customer. If new issues came in with a higher priority they were implemented first in a Kanban-like fashion. As such we could drastically reduce the amount of interruptions. This approach also allowed us to handle the oncoming wave of challenges.

In the sake of these changes we also introduced a new development workflow regarding how we deal with code changes of our core platform in general – our new approach to Pull Requests on GitHub is an article on its own.

Remaining Challenges

The little overhead in terms of unnecessary communications – “need-to-know” – meant that knowledge sharing did not improve. Without a proper Backlog and Planning, forecasting which features and functions would be included in the next release was nearly impossible. Moreover, releases were still driven by customer projects, i.e. whenever a customer milestone was up ahead, a release branch was created and deployed. This also meant that features and breaking API changes were built on top of a published release branch (insert crying kittens).

We are still finding our way and clearly see the need of improving our development process to fit our current state – in terms of how we work as well as how large the company has grown now. It will be a long one to go but an interesting one with lots of things to learn. I will continue to post about our evolvement and look forward to reading your feedback.

Leave a Reply

Your email address will not be published. Required fields are marked *