A microservice journey - part 3: Agile and Microservices

Agile has been around for a while now, and has been proven to be tricky to implement and returns questionable value.

I have seen lots of different forms of agile, and sometimes the ceremonies get all the focus, and the essence of agile is lost. Especially for big corporates, they need to scale out this approach, so it becomes a playbook of steps you need to do.

As we move into microservices, adopting agile was a critical aspect. Being able to change direction or approach means microservices can evolve as we go, rather than being predefined right at the beginning. This is an extrealmy important aspect of microservices. Generally in a monolith, the patterns are established and followed without question. Changing a pattern is hard, and needs commitment, and understanding from everyone. Microservices on the other hand, mean every domain can use a different pattern, and we can grow and develop the right pattern as we go. Whereas a change like this is massive in a monolith.

Test and learn

A big benefit of microservices is the test and learn approach it enables. We should be able to bring up a microservice into prod without affecting anything. Bring up a microservice in a sprint, test with a subset of customer and either iterate and keep developing, or give up and throw it all away. 
A major problem with large scale application with lots of customers is building the right thing. We can have really good ideas, but how do we test what we think is right?? Deploy it! Test it in prod. Get customers to validate it. Big companies can not make these changes Willi nilli, as a change to a monolith can have deviating ripples effects. But to release a separate service with zero impact on the existing production stack is a much more palatable situation. It also gives us the opportunity to test ideas with customer and allow the customer to shape the final solution. This is game changing.

Evolution

Architecture is never finished. If you think it's finished, then you have either lost interest of lost the drive to innovate. To embrace this philosophy, microservices and Agile work together to make things better. A key ceremony in Agile is the retrospective. This gives the team to tool to reflect on what happened and decide how they can change something to make it better. Because microservices are small and isolated, this same logic can be applied to microservices and evolving the architecture with every initiative. Tech debt is a problem in every organisation, but it is also an opportunity to evolve the architecture in meaningful ways. Tech debt is a massive issue in a monolith, as it can severely restrict innovation and progress. As stuff is built decisions are made all the time, which pattern to use, how to structure the code etc. Sometimes the wrong decision is made but not realised until a long time after and it is released and it officially becomes tech Dept. At this stage, the problem has been build on top of and reused. To change anything like this is extremely hard as it means refactoring massive amounts of code (depending on how long it took to realise it was tech debt). With a microservice all the code is separated into much smaller domains. So the blast radius is contained to one microservice. You can't reuse something that isn't there. You can build new tech debt but again it is limited to the one microservice. With each new microservice you can choose to start again and evolve the architecture without having to refactor massive amounts of code.

Ownership

I have already spoken about the importance of keeping good people, and the importance of ownership. Agile is all about the team feeling empowered to define what they are delivering and have the power to affect the final product. Microservices enables this because each team decides how the microservice is structured, what it does, how it works. This autonomy means the team doesn't just own the work in the sprint, but also extends that same mantra of autonomy to the final product. This is such a critical aspect of keeping good people. Good people also have really good ideas, and these ideas can be used in other teams, which can make things even more interesting and enable other to look at thing differently, which generates more ideas. Kind of like a self fulfilling prophesy.

Servant leadership

This is an important concept in agile, and just as important for building microservices. I have been an Architect for some time now, and everywhere I have been the Architect leads the design and the developers build what the Architect says (sometimes... Actually wait?!), and the testers test it. In an Agile world, the team own the process and deliver as a team, so why not the design too. If we think about the value of servant leadership and apply it to Architecture, why should one person or small group of people be making all the decisions? Being a servant leader in this space means stepping back and allowing designs to be owner by the team themselves. Architecture is still important, but more of a leadership role helping the teams make the right decisions, rather than designing everything and carrying a big stick around. Architecture becomes a member of the extended team making sure the decisions being made are the right ones for any upstream or downstream systems. Architecture makes sure we don't duplicate work or build something which is not compatible with something else. This lends itself to ownership, accountability, commitment. As talked about before, good people build good software. to keep good people they need to feel like they are making a difference and own these services they are building.

A good video on servant leadership can be found on youtube (its only 12 minutes long)
https://www.youtube.com/watch?v=lmyZMtPVodo&feature=youtu.be

One of the promises of microservices is the ability to scale out, and yes this is most definitely true. As we endevour to build more things, we can quickly add more people to build those new things. But how can we scale out if one group of people is/are the bottleneck? We would need to build Architecture at the same rate to keep things moving. But Architecture is more about knowing the lay of the land, and making sure things fit, rather than designing and building stuff. So getting new Architects and training them up on the ecosystem is hard, and can take a long time.


Fail fast

I think this term is wildly miss-understood. I believe fail fast refers to testing an idea quickly to see if customers like it or not. NOT cutting corners to delivery things quickly. Unfortunate facebook has made this catch cry popular and now people are using this as an excuse to build sub-optimal solutions with the intent of coming back and fixing them when they "fail fast". 
Microservices enable this "fail fast" (not the corner cutting version) by building a framework which allows us to plug in new components in a consistent and safe way. We can easily add new features without disrupting production, and just as quickly remove them if they do not pan out. By having a distributed system in nature itself promotes this modularity, and hence is a key enabled for this type of experimentation. Agile and Lean talk about these concepts and microservices help delivery that vision in a safe way.

Comments

Post a Comment

Popular posts from this blog

What good looks like!!

Image
I think back to my favourite places to work, and wonder what made that place so memorable? What have I been pursuing every since then. Is it great new tech? is it working on interesting problems and solutions? Is it learning off great team member who think differently?  It is all of the above and more. Over the years I have worked at many places and had some hits and some real misses and realised what I really enjoy is working with people who are nice and make it a pleasure to come to work each day. This doesn't just somehow magically happen, this culture is specifically created by the leaders who foster it. I hear a lot about servant leader or empowering teams, but what does this mean exactly? As a leader, if your focus is on the success of the business. The solution comes first, must be the best, we can not compromise on quality, must be done quickly. You may get a good solution built, but you may also burn through people. If people leave, the vision is diluted and the new team m...
Read more

A microservice journey - part 2: what type of micro service are you?

Once we established we were on the microservice path, the question of just how would that look came up? We had established the problems and understood what we were trying to achieve, but what exactly is a microservice, and how do you go about building it? The importance of the first step can not be understated enough. If everyone understands the problem and really gets what we are trying to achieve, then microservices write themselves. Well not really, but by understanding what we are trying to achieve, the solution options can be assessed around a set of questions which would help drive the decisions being made. Can this service be deployed on its own, without affecting other parts of prod? If we had an influx of users using this feature, can we scale it out independent of the other components? If we did need to scale it out? do we need to scale out any other components as well? What happens if we had a network issue? would it still be able to do its job? If this service did ...
Read more

Validation Rules

Every business application must incorporate some kind of business logic and validate the response received from the user. There are a few ways to incorporate this logic through your application, which involve a mix of client and server side validations. This often means you will have some validation in your UI layer as Field validators and some logic in your business tier, where it is just not possible to bring to the UI such as verifying a date range does not overlap another set, or something which needs to access the database to verify against. The Validation Rules library attempt to unify this disparate approach to validation, and to automate some of the simpler validation logic such as required fields and field length validation. Instead of putting validation through the tiers, we can simply implement IValidation on the business object. This interface attaches a meta list of rules to the entity which we can build in the business layer and will propagate through down to th...
Read more