All RPGs and Storygames by Tod Foley are now available at DrivethruRPG and RPGnow. Bring these games to your table!
Note: This article was originally published on August 23, 2017. Following DrupalCon Nashville, we are republishing some of our key articles on decoupled or "headless" Drupal as the community as a whole continues to explore this approach further. Comments from the original will appear unmodified.
Decoupled Drupal has been well understood at a technical level for many years now. While the implementation details vary, most Drupal teams can handle working on decoupled projects. However, we’ve heard the following from many of our clients:
- We want a decoupled site. Why is this web project so expensive compared to sites I worked on in the past?
- Why do our decoupled projects seem so unpredictable?
- If we decide to invest in decoupled technologies, what can we expect in return?
Let’s dive into these questions.Why Can Decoupled Sites Cost More?
Before getting too much into the details of decoupled versus full-stack, I like to ask stakeholders:“What does your website need to do today that it didn't 5 years ago?”
Often, the answer is quite a lot! Live video, authenticated traffic, multiple mobile apps, and additional advertising deals all add to more requirements, more code, and more complexity. In many cases, the costs that are unique to decoupling are quite small compared to the costs imposed by the real business requirements.
However, I have worked on some projects where the shift to a decoupled architecture is fundamentally a technology shift to enable future improvements, but the initial build is very similar to the existing site. In those cases, there are some very specific costs of decoupled architectures.Decoupling means forgoing Drupal functionality
Many contributed modules provide the pre-built functionality we rely on for Drupal site builds. For example, the Quickedit module enables in-place editing of content. In a decoupled architecture, prepare to rewrite this functionality. Website preview (or even authenticated viewing of content) has to be built into every front-end, instead of using the features we get for free with Drupal. Need UI localization? Content translation? Get ready for some custom code. Drupal has solved a lot of problems over the course of its evolution, so you don’t have to—unless you decouple.Decoupling is shorthand for Service Oriented Architectures
For many organizations, a decoupled website is their first foray into Service Oriented Architectures. Most full-stack Drupal sites are a single application, with constrained integration points. In contrast, a decoupled Drupal site is best conceived of as a “content service,” accessed by many disparate consumers.
I’ve found that the “black-boxing” of a decoupled Drupal site is a common stumbling block for organizations and a driver behind the increased costs of decoupling. To properly abstract a system requires up-front systems design and development that doesn’t always fit within the time and budget constraints of a web project. Instead, internal details end up being encoded into the APIs Drupal exposes, or visual design is reflected in data structures, making future upgrades and redesigns much more expensive. Writing good APIs is hard! To do it well, you need a team who is capable of handling the responsibility—and those developers are harder to find and cost more.Scalable systems and network effects
- Drupal for content management
- A CouchDB application to serve content over an API
- A second CouchDB application to support internal content preview
- A React app for the site front-end
- Disqus for commenting
Compared to the sites our clients need, lullabot.com is a simple site. In other words, as you build, expect to be building a web of systems, and not just a “decoupled” website. It’s possible to have a consumer request Drupal content directly, especially in Drupal 8, but expect your tech teams to push for smaller “micro” services as they get used to decoupling.
Building and testing a network of systems requires a lot of focus and discipline. For example, I’ve worked with APIs that expose internal traces of exceptions instead of returning something usable to API consumers. Writing that error handling code on the service is important, but takes time! Is your team going to have the bandwidth to focus on building a robust API, or are they going to be focusing on the front-end features your stakeholders prioritize?
I’ve also seen decoupled systems end up requiring a ton of human intervention in day-to-day use. For example, I’ve worked with systems where not only is an API account created manually, but manual configuration is required on the API end to work properly. The API consumer is supposed to be abstracted from these details, but in the end, simple API calls are tightly coupled to the behind-the-scenes configuration. A manual set up might be OK for small numbers of clients, but try setting up 30 new clients at once, and a bottleneck forms around a few overworked developers.
Another common mistake is not to allow API consumers to test their integrations in “production.” Think about Amazon’s web services—even if your application is working from a QA instance, as far as Amazon is concerned there are only production API calls available. Forcing other teams to use your QA or sandbox instance means that they won’t be testing with production constraints, and they will have production-only bugs. It’s more difficult to think about clients creating test content in production—but if the API doesn't have a good way to support that (such as with multiple accounts), then you’re missing a key set of functionality.
It’s also important to think about error conditions in a self-serve context. Any error returned by an API must make clear if the error is due to an error in the API, or the request made of the API. Server-side errors should be wired up to reporting and monitoring by the API team. I worked with one team where client-side errors triggered alerts and SMS notifications. This stopped the client-side QA team from doing any testing where users entered bad data beyond very specific cases. If the API had been built to validate inbound requests (instead of passing untrusted data through its whole application), this wouldn't have been a problem.
There's a lot to think about when it comes to decoupled Drupal sites, but it’s the only way to build decoupled architectures that are scalable and lead to faster development. Otherwise, decoupling is going to be more expensive and slower, leaving your stakeholders unsatisfied.Why are decoupled projects unpredictable?
When clients are struggling with decoupled projects, we’ve often found it’s not due to the technology at all. Instead, poor team structure and discipline lead to communication breakdowns that are compounded by decoupled architectures.The team must be strong developers and testers
Building decoupled sites means teams have to be self-driving in terms of automated testing, documentation, and REST best practices. QA team members need to be familiar with testing outside of the browser if they are going to test APIs. If any of these components are missing, then sprints will start to become unpredictable. The riskiest scenario is where these best practices are known, but ignored due to stakeholders prioritizing “features.” Unlike one-off, full-stack architectures, there is little room to ignore these foundational techniques. If they’re ignored, expect the team to be more and more consumed by technical debt and hacking code instead of solving the actual difficult business problems of your project.The organizational culture must prioritize reliable systems over human interactions
The real value in decoupled architectures comes not in the technology, but in the effects on how teams interact with each other. Ask yourself: when a new team wants to consume an API, where do they get their information? Is it primarily from project managers and lead developers, or documentation and code examples? Is your team focused on providing “exactly perfect” APIs for individual consumers, or a single reusable API? Are you beholden to a single knowledge holder?
This is often a struggle for teams, as it significantly redefines the role of project managers. Instead of knowing the who of different systems the organization provides, it refocuses on the what - documentation, SDKs, and examples. Contacting a person and scheduling a meeting becomes a last resort, not a first step. Remember, there’s no value in decoupling Drupal if you’ve just coupled yourself to a lead developer on another team.Hosting complexity
- HTTP servers
- Deployment scripts
- Testing and automation tools
- Caching and other performance tools
- Local development for all of the above
What’s important here is to remember that this instability isn't due to decoupling—it’s due to front-end architecture decisions. There’s nothing that stops a team from building a decoupled front-end in PHP with Twig, as another Drupal site, or anything else.If we invest in Decoupled Drupal, what’s the payoff?
With full-stack Drupal, it’s easy to create and show content that is impossible to view on mobile or set-tops apps. By decoupling the Drupal front-end, and using the same APIs as every other app, it forces CMS teams to develop with an API-first mentality. It puts all consumers on an equal playing field, simplifying the development effort in adding a new app or platform. That, on its own, might be a win for your organization.Scaling large teams?
Most large Drupal sites, even enterprise sites, have somewhere between 5-10 active developers at a time. What if your team has the budget to grow to 30 or 50 developers?
In that case, decoupled Drupal is almost the only solution to keep individuals working smoothly. However, decoupled Drupal isn’t enough. Your team will need to completely adopt an SOA approach to building software. Otherwise, you’ll end up paying developers to build a feature that takes them months instead of days.Decoupling with your eyes open
The most successful decoupled projects are those where everyone is on board—developers, QA, editorial, and stakeholders. It’s the attitude towards decoupling that can really push teams to the next level of capability. Decoupling is a technical architecture that doesn't work well when the business isn't buying in as well. It’s worth thinking about your competitors too—because if they are tech companies, odds are they are already investing in their teams and systems to fully embrace decoupling.
Educational psychologist and WWE wrestler Xavier Woods speaks about the overlap between the storytelling of professional wrestling and narrative design in games. ...
During this month's membership campaign, we mention that the average cost of a DrupalCI core test is $0.24-$0.36. Every time a contribution to the Drupal project needs to be tested, DrupalCI spins up a testbot on AWS to test those changes. DrupalCI runs about 5,000 core tests, and 13,000 contrib tests in an average month.
The test runs on Drupal.org are paid for by our generous partners and members. This is just one of the services provided by the Drupal Association as part of our commitment to maintain Drupal.org so you can focus on Drupal development and community building.
You can help sustain the work of the Drupal Association by joining as a member. Thank you!
Want to hear more about the work of the team? Check out the Drupal.org panel session recording at DrupalCon Nashville.
This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.
The Drupal community has done an amazing job organizing thousands of developers around the world. We've built collaboration tools and engineering processes to streamline how our community of developers work together to collectively build Drupal. This collaboration has led to amazing results. Today, more than 1 in 40 of the top one million websites use Drupal. It's inspiring to see how many organizations depend on Drupal to deliver their missions.
What is equally incredible is that historically, we haven't collaborated around the marketing of Drupal. Different organizations have marketed Drupal in their own way without central coordination or collaboration.
In my DrupalCon Nashville keynote, I shared that it's time to make a serious and focused effort to amplify Drupal success stories in the marketplace. Imagine what could happen if we enabled hundreds of marketers to collaborate on the promotion of Drupal, much like we have enabled thousands of developers to collaborate on the development of Drupal.Accelerating Drupal adoption with business decision makers
To focus Drupal's marketing efforts, we launched the Promote Drupal Initiative. The goal of the Promote Drupal Initiative is to do what we do best: to work together to collectively grow Drupal. In this case, we want to collaborate to raise awareness with business and non-technical decision makers. We need to hone Drupal's strategic messaging, amplify success stories and public relation resources in the marketplace, provide agencies and community groups with sales and marketing tools, and improve the Drupal.org evaluator experience.
To make Promote Drupal sustainable, Rebecca Pilcher, Director of MarComm at the Drupal Association, will be leading the initiative. Rebecca will oversee volunteers with marketing and business skills that can help move these efforts forward.Promote Drupal Fund: 75% to goal
At DrupalCon Nashville, we set a goal of fundraising $100,000 to support the Promote Drupal Initiative. These funds will help to secure staffing to backfill Rebecca's previous work (someone has to market DrupalCon!), produce critical marketing resources, and sponsor marketing sprints. The faster we reach this goal, the faster we can get to work.
I'm excited to announce that we have already reached 75% of our goal, thanks to many generous organizations and individuals around the world. I wanted to extend a big thank you to the following companies for contributing $1,000 or more to the Promote Drupal Initiative:
If you can, please help us reach our total goal of $100,000! By raising a final $25,000, we can build a program that will introduce Drupal to an emerging audience of business decision makers. Together, we can make a big impact on Drupal.
Adds the functionality to rollback specific database updates, useful for development and testing.
Integrates with drush to provide the rollbackdb (rbdb) command. Inspired by the migrate:rollback command in Laravel's Artisan command line tool.
Acquia Developer Center Blog: Experience Express in Alicante: Analytics, Security, and Horizons at DrupalCamp Spain
Atop the Castle of Saint Barbara in Alicante, time sometimes seems to slow down, and words that once held grand meaning seem inadequate. I had a similar feeling both during and on the heels of DrupalCamp Spain, organized by the Spanish Drupal Association and held this year at Las Cigarreras cultural center in a seaside city that is one of the crown jewels of not only the Valencian Community but also of Spain.Tags: acquia drupal planet
The module exposes the translation field that can be used to retrieve translated interface texts out of drupal.