[Epic] API is the Product

[nelsonic]

Context 💭

As a person who uses Apps/Services, I hate it when my data is locked up in a "walled garden".

A service might initially solve a problem for me but I have no control over my data, so when it goes bad (e.g. because the megalomaniac sociopath founder wants all your personal data so they can sell you to ads even though there were no ads when you joined in 2007 to stay in touch with friends ...),
I cannot leave the platform without losing all (or most) of my data!
Sure they might provide an API but it's always limited in scope to what they want you to have.

We want to always ensure that all actions that can be performed in the Web UI can also be done via REST API and all real-time notifications/updates can be subscribed to via WebSocket API¹.
That way anyone using our App can always access all of their data and they are not locked-in to using our service(s).

This is in keeping with our Manifesto: manifesto.md#your-data-is-yours

Story 💡

As a person storing lots of important data in the @dwyl App,
I want to be able to access all of my data at any time via an API
So that I can extract, analyse, update or export my data if I chose to.

As people using the @dwyl App ourselves this is what we would expect from a good great App. Ideally we want to have the API from day 1 (MVP) so that people can immediately access and create their data programmatically.

Todo

• Define the scope of the API ("everything" isn't a scope!)
• How will we document our API
• Will we conform to a particular API Spec e.g: JSON Schema What is a JSON Schema and why is it useful? How do I create/use a JSON Schema? learn-json-schema#1
  or are we better off just following the good examples that don't conform to a Spec?
  see: Ask HN: What is the best API documentation you have ever seen? learn-api-design#39 either way I think it's worth updating our knowledge of APIs https://github.com/dwyl/learn-api-design before embarking on building our API.
• Explore Content Negotiation as a way to keep the App and API as the same Phoenix App.
  • Tutorial to Consolidate our Understanding: Create Content Negotiation Tutorial phoenix-content-negotiation-tutorial#1
  • [SPIKE] Create Content Negotiation Plug: [SPIKE] Create Content Negotiation Plug that *Any* Phoenix App can use! content#1
  • Publish content Plug to Hex.pm: https://hex.pm/packages/content

¹ As noted in our "Why Elixir" thread: dwyl/learn-elixir#102 one of our key reasons for selecting Elixir/Phoenix for our backend tech is for Channels which we feel is one of the "super powers" of the language/framework! Having a scalable Realtime API with presence is one of the key features we need to have for teamwork.

[nelsonic]

I am working on Content Negotiation and creating a tutorial/example: https://github.com/dwyl/phoenix-content-negotiation-tutorial

[nelsonic]

I'm fairly happy with the tutorial: https://github.com/dwyl/phoenix-content-negotiation-tutorial
(feedback and improvements very much welcome!)

Now I'm working on generalising the solution in a Plug so that any App can do this with minimal code: dwyl/content#1

[nelsonic]

Twitter (Learn from them!)

One of the reasons for Twitter's early success was their extensive and well-documented API.
All actions that could be performed in the UI (or via SMS which was a way to Tweet in the beginning!)
https://en.wikipedia.org/wiki/Twitter#Developers

This is the success we want to emulate but in the realm of personal/team effectiveness.

What we don't ever want to do is make the mistake Twitter made in removing features from their API. In the run up to their (November 6 2013) IPO, twitter tightened their public API with by restricting queries and rate-limiting which crippled may apps that had relied on it.

The leaders at Twitter put the investors/money ahead of the users/developers. It made them all rich but it alienated many of their early adopters. We want to avoid this. If anyone ever decides that the @dwyl App is not meeting their expectations they are free to export all their data and use an alternative service.

[nelsonic]

I have finished my Content Negotiation SPIKE: dwyl/content#1 ✅
Created: https://github.com/dwyl/content (feedback very much welcome! 🙏)
Published the package: https://hex.pm/packages/content 📦
And demonstrated how to use it: /dwyl/phoenix-content-negotiation-tutorial#part-2 📝

I feel confident that we can use Content Negotiation in our App. 🚀

[nelsonic]

@LuchoTurtle please read through this and confirm your understanding. 👀
Ask follow-up questions so we can move forward with this. 💬 🙏

[LuchoTurtle]

Seeing as this is the "last step" before beginning to develop the Flutter version of the front end, I feel like defining the scope of the API/list of requirements shouldn't be a task I should undertake alone.
I'm happy to implement it (given the work that was already done with learn-insomnia and learn-api-design) and I think I know how to.
But the list of endpoints to be implemented require a list of requirements and that should be something that needs to be discussed so we're all on the same page 🤔

[nelsonic]

@LuchoTurtle don't worry, I wouldn't axsk you to define the scope of the API. 🗻
Just want to make sure you're on the same page as us with regards to the API being "core" to what we are building. 👌
We could do a remote pairing session to plan/roadmap it when you're free. 🗺️