API Lessons from APIWorld 2016

Friday, 16 September 2016

Marc Andreessen claimed in 2011 that Software Is Eating the World, and as technology has progressed ever onward it’s now a case of APIs Are Eating the World more specifically or at least APIs Are Fueling the Software That’s Eating the World. Whatever your particular view on the matter, APIs are extremely and increasingly important in software and hence, the world at large. With that in mind, I spent the last few days at APIWorld, the “largest vendor-neutral API conference and expo”. Below are some insightful learnings1 I picked up during a couple of sessions that might be helpful in providing an API – whether you’re creating, maintaining, or actively growing one.

Negotiating an API: Crafting Endpoints So Developers Still Like Each Other link intact

A great talk by Travis Jungroth (@travisjungroth) from HouseCanary which could have been titled as “5 Lessons in Leveraging UX Best Practices to Create a Better API”.

  • Lesson 1: Empathy
    • Design the client side first in order to make the “ideal client”.
    • Write 20% of the calls that get 80% usage. Set the patterns that will dictate how the remaining 80% of the calls will be designed.
    • Take this time to negotiate with yourself the tradeoffs between ease of the provider (you, when creating/maintaining it) and ease of the client (your users, developers using the API for their products).
    • Get feedback early and often.
  • Lesson 2: Be Consistent, But Flexible
    • While seemingly at odds, more specifically, be consistent in your implementation but flexible in function.
    • Simplicity will often get you both at the same time, mostly for free.
  • Lesson 3: Endpoint Design
    • Make sure you’re hitting all the REST super-basics; avoid elaborate and unique implementations.
    • Jungroth noted that he prefers “coding like a lawyer”, preferring to look for and replicate prior art when he’s coding.
    • By following standard conventions & best practices, you’re providing your potential users the best opportunity to understand the API because they’re already familiar with the core basics of it.
  • Lesson 4: Get Feedback
    • It’s best to talk to actual users. When doing this, he prefers to not write it all down and instead make sure that he’s talking to all of the users approached for a few reasons:
      • Logging all feedback with the plan to query it later almost never happens; the feedback just sits somewhere never to be looked at again.
      • When the same person is doing all the user interviews, that person will be able to tune into the most important issues as they will surface simply due to repetition from a large group of users.
      • Usage & Logging Analytics are the #1 feedback source (in addition to endpoint usage), so make sure you’re fully logging errors.
  • Lesson 5: Be Careful When Making Change (the hardest thing to do)
    • An API is essentially a contract with your users; when they choose to use it, they’re putting their trust in you that it will work and work consistently. Breaking the API is breaking their trust.
    • Of course, adding endpoints is OK - the trick is how to remove or change them.
    • You set yourself up for failure if you add versioning in the URL (e.g., http://yourdomain.com/yourapi/v1/…); instead, add versioning info in the headers.
    • Documentation is hugely helpful; he’s a huge fan of Swagger.

Jungroth also shared a few Do’s and Don’ts:

  • Do
    • Try to achieve flat endpoint design.
    • Use user personas when designing the client.
    • When needing to validate, split the call into a second “is_valid”.
    • If you need to provide large files, consider passing through a link for the developer to download directly (such as from Amazon S3).
  • Don’t
    • Use query/endpoint flags that dramatically alter default behavior of the endpoints.
    • Allow endpoints to use different services for different things.

The Fundamentals of Platform Strategy – Creating Genuine Value with APIs link intact

This talk by Steven Willmott (@njyx) from 3scale / Red Hat could’ve benefitted from another 20 minutes or so; fantastic information that unfortunately got occasionally rushed through as the conference ran a pretty tight ship in maintaining the schedule.

While Willmott has provided his deck on Slideshare, here are the points I was able to scribble down:

  • Platforms are both a huge opportunity for value and often a wasted resource.
    • They’re big initiatives, often mission-critical, and are for a genuine need.
    • The challenges: the APIs are not used (or don’t work), and the team responsible often isn’t properly resourced.
  • “The Jeff Bezos Moment” was a defining milestone.
    • But even if you do have one yourself, make sure you’re focused on the value you’re delivering piece by piece.
  • Plan the Value:
    • Individual APIs can have tremendous value.
    • Underpin your products, enable new business channels, provide abstraction layers.
    • Core value almost always revolves around agility.
  • Common Errors:
    • Trying something peripheral to your main business to lower your risk.
    • Replacing multiple core systems all at once.
    • Having your platform team build everything themselves; instead, share the burden among other teams.
    • Top-down designing a uniform approach.
  • Your best time to begin:
    • When you have IT driving an initiative.
    • When IT teams are ready to rethink some infrastructure.
    • When the goal isn’t too abstract.
  • Focus on the true value and identify the true users:
    • …of the API
    • …the end user on the very end of the API
    • Ask the usual Product Management questions:
      • Who is the customer?
      • Where is their value?
      • If we create this, will you use it?
      • What is the complete use case?
  • Deliver value one API and one use case at a time.
  • The Developer Experience is overrated:
    • How do the APIs need to be consumed?
    • Who else might be consuming the API?
    • Where do their effects propagate to?
    • What additional value can be added?
  • The API is not the key thing – the Value it delivers is.
    • The platform’s value is the result of the value between providers and consumers
  • Measuring Value:
    • Delivering a platform is very hard & unglamorous work.
    • It’s not a big bang.
    • People don’t like the rules and restrictions.
    • But it can be hugely rewarding.
    • Recommends measuring on business(es) enabled – number of- and revenue-created-by.

10 Mistakes to Avoid When Building Your API (And How We Learned the Hard Way) link intact

Another great snapshot of learnings from Chris Paul (@idiosynchris) of HelloSign.

  • Mistake 1: Don’t Write the Documentation First
    • Starting with writing the documentation before a line of code helps you avoid inconsistencies created in development.
  • Mistake 2: Don’t Notify Your Users of Change
    • Paul shared a story about a rebrand HelloSign underwent – and didn’t preannounce. While seemingly innocuous and inconsequential to their API users, the rebranding did break the experience for those users that chose to design their UI & UX consistently with HelloSign’s (prior) design.
    • Ultimately, HelloSign’s users really wanted the heads-up and ability to test the changes ahead of time.
  • Mistake 3: Don’t Provide SDKs
    • HelloSign took a while before providing SDKs and found that their API’s adoption rate jumped when they did; many users confirmed the assumption and cited the SDKs as a deciding factor to sign up.
    • If you don’t provide an official SDK, your community will fill the void with their own of varying quality and eventually you will be expected to support them.
  • Mistake 4: Ignore Task Automation Services
    • HelloSign initially didn’t put much thought into supporting task automation services, but eventually co-developed with Zapier their particular implementation.
    • Now they see 1 out of 10 API requests being made through task automation services.
  • Mistake 5: Don’t Use Your Own API
    • Paul joked that HelloSign’s Marketing team requested shorthanding this to “Drinking Your Own Champagne” instead of the common “Eating Your Own Dogfood” analogy.
    • Here Paul also referenced the aforementioned “Jeff Bezos Moment” and pointed out that if HelloSign used their own API it would’ve saved them a lot of development and testing effort.
    • Using your own API will also tell you whether or not your developer experience is first rate.
  • Mistake 6: Build One-Size-Fits-All Rate Limiting
    • In particular, HelloSign found that doing this made upgrading problematic. Further, customers that hit the limit often had outstanding time-sensitive documents that needed signing.
    • Now, the HelloSign support team has the ability to increase individual customer rate limits immediately.
  • Mistake 7: Ignore Duplicate API Requests
    • You can probably file this under “Don’t ever think you know everything your user is going to do with your product”; HelloSign initially took a pretty conservative view on duplicate requests but eventually found they were catching false positives resulting in customer complaints.
    • Instead, let the user define what is a duplicate. Suggested implementing an “item potency key” user-defined parameter, pointing to Stripe’s API as an example.
  • Mistake 8: Don’t Plan for Support
    • HelloSign’s developer team found that they were having a lot of their time being taken by being the ones supporting the API as their Customer Support team was oriented around end-user support.
    • They eventually built out a separate API Support team when one of their CS reps interested in programming stepped up and created the team.
  • Mistake 9: Don’t Provide Your Users Data Insight
    • Users expected this level of service, which resulted in a lot of dev team hours being used answering CS escalations about failures, etc.
  • Mistake 10: Be Inflexible
    • Paul shared a story about one of HelloSign’s customers that was in the business of assisting people pursuing adoption. The original design of the architecture never anticipated their use case of PDFs with a large amount of editable fields and this particular customer was running into time-outs and other errors because their PDFs were exceeding the POST field maximum (each editable PDF field is one POST field). The customer turned out to be very generous with their time to work with HelloSign to come up with an acceptable solution.
    • Paul referenced Postel’s Law: “Be conservative in what you do, liberal in what you accept from others”.

Consequences of an Unhealthy API – What Businesses Need to Know link intact

Rounding out the “API lessons learned” was this talk by Patrick Malatack (@patrickmalatack) of Twilio, although it really should’ve been titled something to the effect of “3 Lessons We Learned to Develop Great APIs” as he never really discussed any consequences that resulted from an unhealthy API.

  • Lesson 1: APIs Are For Humans
    • Make it easy to read (by a human).
    • Opt-in to complexity; introduce complexity to the user at their time of need for it. Malatack showed examples from the Twilio API where a simple message’s straight-forward code was slowly incremented to introduce first an image attachment, then functionality to support international messaging.
    • Hackathons aren’t just for fun; don’t just sponsor them for branding awareness but participate to learn from the contestants too. Sit down with them while they’re working and get their feedback.
  • Lesson 2: API Docs Are Marketing for Developers
    • Malatack also advocates for writing the docs first, before any code as a way to do rapid prototyping; he likens the process as the equivalent of wireframing in Design.
    • Here too, solicit feedback from users early on.
    • “Helper Libraries” are the primary interface: they account for >80% of Twilio’s API requests.
      • This is basically the manifestation of looking where your desired users are and supporting them there; Provide all documentation in every given language/framework.
      • Most importantly: make sure the documentation for the helper libraries are as good as the docs for endpoints.
      • Support standard HTTP status codes; don’t get creative.
  • Lesson 3: Move Fast and Don’t Break Things
    • Users put a lot of trust in you and your API when they choose to use it.
    • A Diamond An API Is Forever”.
    • Never have maintenance windows.
    • Consistency trumps speed:
      • Your API is just one part of what comprises your users’ product; you have no way of knowing the consequences for breaking your API to your users.
      • Measure your P99s.
      • A little latency is probably OK as long as it’s consistent (i.e., your users can expect it and design their products accordingly).
    • “Flags” are your friends:
      • Trilio created “Flag” attributes at the account level that enables them to change anything on a particular account.
      • Once you recognize that a design decision is problematic, you can introduce a new flag to isolate all the affected users and address the issue(s) going forward.
      • Flags also enable you to easily test with subsets of users.
    • Allow for independent releases; version each product/feature independently to minimize risk.

While there are some slight variances here and there, it’s reassuring that there’s a consistent story among these API practitioners: Remember your API is for people and design it for both your developer users and their end-users in mind. Staying user-focused will set a sound foundation for success.

  1. For those looking for something interesting and maybe controversial, you might want to check out Owen Rubel’s The New API Pattern deck, in which he argues for a more robust design pattern to better handle distributed architectures.