All too often, API designers and developers work to peripheral requirements instead of focusing on exposing the functionality of the underlying system for which they are developing an API.
This creates brittle solutions that are not reusable. In fact, this type of solution amounts to little more than point-to-point integration — using MuleSoft.
Why would technical teams deliberately choose to hobble the Mule this way?
Here are a few reasons and pitfalls to avoid
Mistake #1 — Designing for Project Requirements
This sounds all wrong — right?
Aren’t we supposed to work to project requirements? Of course we are. And yet…
In an integration solution, we have to think more broadly than that.
I’m not saying we shouldn’t work to project requirements; what we need to do is design a System API first, and then ensure that the API supports project requirements.
The problem with working to specific project requirements is this usually puts the cart before the, er — Mule.
If I am told that an API consumer needs:
- a specific set of data
- in a specific shape
- with a specific set of field names
- served in a specific format
- filtered in a specific way
- retrieved at specific times and
- used for a specific purpose
… I have a very tight specification. Armed with this information, a Designer is likely to define an API implementation that conforms to the above requirements.
Developers love tight specifications because there is zero room for ambiguity. They implement to spec. Turnaround is quick and the client is happy. We’re living the dream!
The trouble is that, having now developed an API to the very specific project requirements described above, we now have, well — an API of sorts. Going forward, business stakeholders will expect this asset to be reusable. And why wouldn’t they? After all, isn’t that the purpose of an API?
So, when the next project team comes along and asks for access to your shiny new API, everybody is horrified to learn that it does not support the new project’s requirements.
For the new project:
- the required data is not all present
- the message is in the wrong shape
- fields are misnamed
- the message format is unusable and
- existing filters render data inaccessible
Worse still, we daren’t change this API in any way because that would break the API’s single existing consumer.
In fact, we now discover that our “API” is not an API at all. Instead, we have developed a consumer-facing solution, built for one specific client, that has nothing to do with the underlying system actually being exposed.
While it might have made for a happy project outcome, it is NOT an API.
Solution
When designing an API, we need to consider the underlying system or service we are being asked to expose. We need to consider its functionality holistically and we need to API-enable those features first and foremost.
We then need to consider business security and how we will secure this API in a generic way.
Once that is done, we can then examine our design from a project perspective. It is here that we ask ourselves the question; how will the API support this specific project’s requirements? This may reveal certain shortcomings. For example, filtering requirements. We can then go back to our API and consider how we might filter data in a way that makes sense to our underlying system; not the project. Then we can reconsider this from a project perspective and ensure that the new, sensible, filtering feature supports the specific project requirements.
This approach is far more likely to result in a System-facing API that increases your chances of reusability exponentially.
Mistake #2 — Designing for The Status Quo
The best example of designing for the status quo is a project in which I was asked to expose a sub-set of data within a Data Warehouse via an API. Once again, this was for a specific project. However, I discussed the matter with the tech lead at length. I explained the dangers of Mistake #1 above and we agreed that we would not fall into that trap.
Three projects and three API’s later, I discussed it with the tech lead again.
I asked the question, ‘Why are we creating a new API every time a new consumer comes along? I thought we discussed this.’
It turns out the pressure was not coming from the API consumers but rather from the DBA team that managed the data warehouse. Every time a new consumer asked them for data, they created a custom view for that consumer. It was their knee-jerk response to every new requirement. It was how they had always done things.
This is, by definition, a point-to-point implementation. Only now, they wanted to involve the Mule in their folly as well.
The point of implementing this API was to save the DBA’s the trouble of creating a new view every time a project needed information from their system, not to create more work for the API Integration team.
I instructed the developer that, henceforth, he should refuse to implement any API for a newly created custom view on the underlying database. If the DBA’s insisted on following that course, there was no need for an API; database consumers could create their own DB connections and consume the data directly.
Solution
Once again, the focus should always be first on what value the underlying service offers the business and second, how we support project requirements within that framework.
If existing stakeholders insist on doing things a certain way, then we need to engage with them and understand why that is. There are only two possible outcomes off the back of that conversation:
- Stakeholders will change the way they do things to make their own lives easier.
- Stakeholders offer compelling reasons to maintain the status quo; API’s are not the answer to everything
Mistake #3 — kicking the Can down the road
This is a combination of the above two mistakes. It usually comes in the form of a Project Manager’s instruction.
‘I don’t care about best practice right now. I just want to meet my project timelines,’ comes the weekly howl from the pack of project managers baying at IT’s heels. ‘Give me what I need now, and we can implement it properly later.’
The trouble is later is almost always too late. Once implemented, downstream systems will begin to rely on the cheap and cheerful fix and ensure that it remains brittle for all time. In six months, these same project managers will wonder why your API can’t be more flexible as new requirements come to light.
The longer we delay fixing a poor implementation, the more technical debt we build into a solution. This is as true for MuleSoft as for any other technology. You cannot hobble the Mule and then expect it to leap tall buildings when requirements demand.
Solution
The only solution to this is:
- good governance and
- for tech leads to stand their ground
When project demands place tight deadlines on a team, we need to find other ways to deliver on time. We cannot sacrifice good API design for the sake of short-term convenience
Mistake #4 — ignoring integration experience
This is without doubt the biggest challenge for many businesses when undertaking an integration strategy. Experience counts. And in the integration space, it is Integration Experience that counts the most.
Sure, we can use a team of qualified commodity resources or repurpose our web development team to implement our API landscape. Just remember, if this is their first integration undertaking, they need support. It doesn’t matter how experienced they are in their current field of expertise; when it comes to integration, they don’t know what they don’t know — yet!
Left to their own devices, this team will fall prey to the technology hype-cycle and your project will suffer the consequences.
Solution
Bring in a Mule Whisperer.