Hiding the Lead
Please log in to watch this conference skillscast.
Information hiding, coupling, and cohesion, microservices-style
The terms coupling and cohesion come from the world of structured programming, but they are also thrown about in the context of microservices. In this session, I look at the applicability of these terms to microservice architecture and also do a deep dive into the different types of coupling to explore how ideas from the 1970s still have a lot of relevance to the types of systems we build today.
Q&A
Question: How does your desire for explicit schemata (certainly for ingested entities) measure up against Postel's Maxim (the Robustness principle)?
Answer: Postel's maxim describes a situation where we have untrusted parties sending us stuff. In a world where I can't trust the APIs I call to be sensible in changing their APIs, I might consider it, but that doesn't describe most microservice organisations I've worked in. The problem is this places the responsibility on the consumer, not the producer, to maintain backwards compatibility. That is to some extent unworkable if you want to ensure you can safely make a change to a microservice and release it. As the maintainer of a microservice, do I have to test the various consumer's tolerant readers? So I'd consider the use of a tolerant reader with untrusted parties, but I wouldn't adopt this as a generic solution to the evolvability of microservice interfaces within an organisation.
Question: I've been looking at Consumer-Driven Contracts for a while, but the tricky bit is semantics. The vast majority of the times we've had issues due to accidental changes it's been semantics not schema/type things. I haven't really found a good way of doing that
Answer: Honestly, it's just tricky. Fundamentally, the idea is simple - you want the consumer to be able to be explicit about the behaviour they expect of your microservice. The more complex your microservice, or the more nuanced it's behaviour, the more cases you might need to cover to catch all cases. Of course, there are other things you can/should consider doing around your release processes to mitigate accidental breakages. Canary rollouts are one example - at least then if you do encounter a breaking change in prod, its impact will likely be limited.
Question: You suggested only exposing the things that your consumers currently need, and avoid exposing everything in case they need it in the future. How do you avoid the problem where as you get more consumers, you end up adding more and more endpoints that effectively do the same thing, but return slightly different data according to the different new consumer’s needs?
Answer: Being consumer first doesn't mean you do everything a consumer asks for. Treat your service interface like a user interface. It isn't graphical, but in terms of how you evolve it, use similar ideas as to how you design a user interface. You need to try and accommodate everyone, but you have to balance the cost. Minor variations on a theme should result in a change to an existing interface, and such a change should be made in a backwards compatible way (e.g. expansion changes).
If a type of consumer needs a radically different service interface - perhaps a fundamentally different style of communication (e.g. you provide request/response, but the consumer wants event-driven), then you might need to deliver a totally different style of endpoint.
There are also use cases where you might want to give different types of consumers different endpoints - e.g. internal vs external APIs, or presenting endpoints that don't expose PII to most consumers, but does expose PII to selected consumers.
Question: Any pointers, references, thoughts on specifics around designing interfaces up front for later backward compatibility changes?
Answer: The best advice I can give here is don't. Designing a service interface without knowing who the consumers will be will unfortunately inevitably lead to you a.) Exposing more than you need and still b.) Delivering an interface which isn't exactly what the consumer wants anyway.
If work is going on in parallel - e.g. you're creating the microservice whilst the consumer isn't ready, at the very least attempt to work with the consumer to do the best job you can of coming up with an interface that they want, and work to that, with the understanding that you might get it wrong.
Question: I particularly liked the call-out that coupling and cohesion are closely related.
It's made me realise that lots of discussions I've had in the past few years have mainly been around coupling in isolation, rather than including cohesion in the discussion. This has resulted in some interesting architectures that didn't really help improve cohesion and independent deployments. I'll definitely be making sure that both are discussed together in future!
Answer: Glad it was useful! I also like a quote I heard from Mike Nygard - "Cohesion is coupling we like". We get to make decisions about what code we choose to group together - or don't. But we should factor into that thinking how our software will be worked on. Hope that helps!
Question: The same ideas hold at a class level as well: high cohesion / low coupling is desirable.
Answer: Yep - that's one of the main points I was hoping would come across in the talk - these ideas are not new, they come from work at the code level. I'm just reframing these ideas for service-based interactions.
Question: Do you think, heavy use of design patterns in a single micro service can be a smell of tight coupling? A sign that we may need to revisit the domain modelling?
Answer: Not necessarily a sign of tight coupling, but the heavy use of very overt language around design patterns (or the overly heavy use of a certain pattern) can sometimes make me worried that someone hasn't thought about the domain, they've just tried throwing patterns at the screen till it works. For me, the domain of the system should be more obvious to us than the patterns used to implement it.
So it's a sign of general concern, rather than a specific concern around tight coupling
Question: When we startup microservice, it is a big challenge to define each of the microservice boundaries. A clear boundary could help us to reach high cohesion and low coupling :) . May I have your thoughts about the boundary of microservice?
Answer: If you're a startup, my general advice is NOT to create microservices. Stick with a simple single-process monolithic deployment. The monolith, as an architecture, is not inherently a bad thing, in fact I think for most people it's likely the most sensible architectural choice. I have specific concerns though about the use of microservices by startups.
These concerns are outlined here: https://samnewman.io/blog/2015/04/07/microservices-for-greenfield/ and here: https://www.youtube.com/watch?v=aAbKULcthw0
but I can summarise the concerns thus:
a. As a startup, you have limited resources. Those limited resources should be focused on finding market fit, not building a distributed system
b. As a startup, what you are building is likely going to change, perhaps drastically. Your domain is highly fluid - that makes finding stable boundaries difficult. Expect lots of breaking changes across microservice boundaries with a highly fluid domain
c. Microservices can be useful to help you scale an application that is already successful. Scale in terms of handling more users, or supporting the concurrent development by a larger dev team. But as a startup, you don't know if you'll be successful, nor if you do become successful can you forecast exactly what constraint will need to be addressed. So knowing what sort of microservice architecture might be required is difficult
Now, there are always exceptions, but I think most startups should keep the software simple, and focus energy on finding market fit and making customers happy. And remember, a good product with "bad" tech will outperform a bad product with "good" tech any day of the week. Once you get successful, you'll have plenty of time to migrate to microservices later.
YOU MAY ALSO LIKE:
Hiding the Lead
Sam Newman
Sam Newman, a techie interested in cloud, continuous delivery and microservices. Aside from other things he's committed sporadically to open source projects, spoke at more than a few conferences, and wrote some things including the book Building Microservices for O'Reilly.