Dries Buytaert

Subscribe to Dries Buytaert feed
Updated: 6 hours 19 min ago

The evolution of Acquia's product strategy

11 October 2017 - 1:26pm

Four months ago, I shared that Acquia was on the verge of a shift equivalent to the decision to launch Acquia Fields and Drupal Gardens in 2008. As we entered Acquia's second decade, we outlined a goal to move from content management to data-driven customer journeys. Today, Acquia announced two new products that support this mission: Acquia Journey and Acquia Digital Asset Manager (DAM).

Last year on my blog, I shared a video that demonstrated what is possible with cross-channel user experiences and Drupal. We showed a sample supermarket chain called Gourmet Market. Gourmet Market wants its customers to not only shop online using its website, but to also use Amazon Echo or push notifications to do business with them. The Gourmet Market prototype showed an omnichannel customer experience that is both online and offline, in store and at home, and across multiple digital touchpoints. The Gourmet Market demo video was real, but required manual development and lacked easy customization. Today, the launch of Acquia Journey and Acquia DAM makes building these kind of customer experiences a lot easier. It marks an important milestone in Acquia's history, as it will accelerate our transition from content management to data-driven customer journeys.

Introducing Acquia Journey

I've written a great deal about the Big Reverse of the Web, which describes the transition from "pull-based" delivery of the web, meaning we visit websites, to a "push-based" delivery, meaning the web comes to us. The Big Reverse forces a major re-architecture of the web to bring the right information, to the right person, at the right time, in the right context.

The Big Reverse also ushers in the shift from B2C to B2One, where organizations develop a one-to-one relationship with their customers, and contextual and personalized interactions are the norm. In the future, every organization will have to rethink how it interacts with customers.

Successfully delivering a B2One experience requires an understanding of your user's journey and matching the right information or service to the user's context. This alone is no easy feat, and many marketers and other digital experience builders often get frustrated with the challenge of rebuilding customer experiences. For example, although organizations can create brilliant campaigns and high-value content, it's difficult to effectively disseminate marketing efforts across multiple channels. When channels, data and marketing software act in different silos, it's nearly impossible to build a seamless customer experience. The inability to connect customer profiles and journey maps with various marketing tools can result in unsatisfied customers, failed conversion rates, and unrealized growth.

Acquia Journey delivers on this challenge by enabling marketers to build data-driven customer journeys. It allows marketers to easily map, assemble, orchestrate and manage customer experiences like the one we showed in our Gourmet Market prototype.

It's somewhat difficult to explain Acquia Journey in words — probably similar to trying to explain what a content management system does to someone who has never used one before. Acquia Journey provides a single interface to define and evaluate customer journeys across multiple interaction points. It combines a flowchart-style journey mapping tool with unified customer profiles and an automated decision engine. Rules-based triggers and logic select and deliver the best-next action for engaging customers.

One of the strengths of Acquia Journey is that it integrates many different technologies, from marketing and advertising technologies to CRM tools and commerce platforms. This makes it possible to quickly assemble powerful and complex customer journeys.

Acquia Journey will simplify how organizations deliver the "best next experience" for the customer. Providing users with the experience they not only want, but expect will increase conversion rates, grow brand awareness, and accelerate revenue. The ability for organizations to build more relevant user experiences not only aligns with our customers' needs but will enable them to make the biggest impact possible for their customers.

Acquia's evolving product offering also puts control of user data and experience back in the hands of the organization, instead of walled gardens. This is a step toward uniting the Open Web.

Introducing Acquia Digital Asset Manager (DAM)

Digital asset management systems have been around for a long time, and were originally hosted through on-premise servers. Today, most organizations have abandoned on-premise or do-it-yourself DAM solutions. After listening to our customers, it became clear that large organizations are seeking a digital asset management solution that centralizes control of creative assets for the entire company.

Many organizations lack a single-source of truth when it comes to managing digital assets. This challenge has been amplified as the number of assets has rapidly increased in a world with more devices, more channels, more campaigns, and more personalized and contextualized experiences. Acquia DAM provides a centralized repository for managing all rich media assets, including photos, videos, PDFs, and other corporate documents. Creative and marketing teams can upload and manage files in Acquia DAM, which can then be shared across the organization. Graphic designers, marketers and web managers all have a hand in translating creative concepts into experiences for their customers. With Acquia DAM, every team can rely on one dedicated application to gather requirements, share drafts, consolidate feedback and collect approvals for high-value marketing assets.

On top of Drupal's asset and media management capabilities, Acquia DAM provides various specialized functionality, such as automatic transcoding of assets upon download, image and video mark-up during approval workflows, and automated tagging for images using machine learning and image recognition.

By using a drag-and-drop interface on Acquia DAM, employees can easily publish approved assets in addition to searching the repository for what they need.

Acquia DAM seamlessly integrates with both Drupal 7 and Drupal 8 (using Drupal's "media entities"). In addition to Drupal, Acquia DAM is built to integrate with the entirety of the Acquia Platform. This includes Acquia Lift and Acquia Journey, which means that any asset managed in the Acquia DAM repository can be utilized to create personalized experiences across multiple Drupal sites. Additionally, through a REST API, Acquia DAM can also be integrated with other marketing technologies. For example, Acquia DAM supports designers with a plug in to Adobe Creative Cloud, which integrates with Photoshop, InDesign and Illustrator.

Acquia's roadmap to data-driven customer journeys

Throughout Acquia's first decade, we've been primarily focused on providing our customers with the tools and services necessary to scale and succeed with content management. We've been very successful with helping our customers scale and manage Drupal and cloud solutions. Drupal will remain a critical component to our customer's success, and we will continue to honor our history as committed supporters of open source, in addition to investing in Drupal's future.

However, many of our customers need more than content management to be digital winners. The ability to orchestrate customer experiences using content, user data, decisioning systems, analytics and more will be essential to an organization's success in the future. Acquia Journey and Acquia DAM will remove the complexity from how organizations build modern digital experiences and customer journeys. We believe that expanding our platform will be good not only for Acquia, but for our partners, the Drupal community, and our customers.

Categories: Drupal

Hermès using Drupal

6 October 2017 - 8:40am

Since its founding in 1837, Hermès has defined luxury. Renowned as an iconic brand within the fashion industry, Hermès is now setting the trend for how customers shop online. This week, Hermès launched its new site in Drupal!

Hermès married the abilities of Drupal as a CMS and Magento as an eCommerce engine to provide their customers with highly engaging shopping experience. Hermès' new site is a great example of how iconic brands can use Drupal to power ambitious digital experiences. If you are in the mood for some retail therapy, check out https://www.hermes.com!

Categories: Drupal

Drupal looking to adopt React

2 October 2017 - 10:32am

Last week at DrupalCon Vienna, I proposed adding a modern JavaScript framework to Drupal core. After the keynote, I met with core committers, framework managers, JavaScript subsystem maintainers, and JavaScript experts in the Drupal community to discuss next steps. In this blog post, I look back on how things have evolved, since the last time we explored adding a new JavaScript framework to Drupal core two years ago, and what we believe are the next steps after DrupalCon Vienna.

As a group, we agreed that we had learned a lot from watching the JavaScript community grow and change since our initial exploration. We agreed that today, React would be the most promising option given its expansive adoption by developers, its unopinionated and component-based nature, and its well-suitedness to building new Drupal interfaces in an incremental way. Today, I'm formally proposing that the Drupal community adopt React, after discussion and experimentation has taken place.

Two years ago, it was premature to pick a JavaScript framework

Three years ago, I developed several convictions related to "headless Drupal" or "decoupled Drupal". I believed that:

  1. More and more organizations wanted a headless Drupal so they can use a modern JavaScript framework to build application-like experiences.
  2. Drupal's authoring and site building experience could be improved by using a more modern JavaScript framework.
  3. JavaScript and Node.js were going to take the world by storm and that we would be smart to increase the amount of JavaScript expertise in our community.

(For the purposes of this blog post, I use the term "framework" to include both full MV* frameworks such as Angular, and also view-only libraries such as React combined piecemeal with additional libraries for managing routing, states, etc.)

By September 2015, I had built up enough conviction to write several long blog posts about these views (post 1, post 2, post 3). I felt we could accomplish all three things by adding a JavaScript framework to Drupal core. After careful analysis, I recommended that we consider React, Ember and Angular. My first choice was Ember, because I had concerns about a patent clause in Facebook's open-source license (since removed) and because Angular 2 was not yet in a stable release.

At the time, the Drupal community didn't like the idea of picking a JavaScript framework. The overwhelming reactions were these: it's too early to tell which JavaScript framework is going to win, the risk of picking the wrong JavaScript framework is too big, picking a single framework would cause us to lose users that favor other frameworks, etc. In addition, there were a lot of different preferences for a wide variety of JavaScript frameworks. While I'd have preferred to make a bold move, the community's concerns were valid.

Focusing on Drupal's web services instead

By May of 2016, after listening to the community, I changed my approach; instead of adding a specific JavaScript framework to Drupal, I decided we should double down on improving Drupal's web service APIs. Instead of being opinionated about what JavaScript framework to use, we would allow people to use their JavaScript framework of choice.

I did a deep dive on the state of Drupal's web services in early 2016 and helped define various next steps (post 1, post 2, post 3). I asked a few of the OCTO team members to focus on improving Drupal 8's web services APIs; funded improvements to Drupal core's REST API, as well as JSON API, GraphQL and OpenAPI; supported the creation of Waterwheel projects to help bootstrap an ecosystem of JavaScript front-end integrations; and most recently supported the development of Reservoir, a Drupal distribution for headless Drupal. There is also a lot of innovation coming from the community with lots of work on the Contenta distribution, JSON API, GraphQL, and more.

The end result? Drupal's web service APIs have progressed significantly the past year. Ed Faulkner of Ember told us: "I'm impressed by how fast Drupal made lots of progress with its REST API and the JSON API contrib module!". It's a good sign when a core maintainer of one of the leading JavaScript frameworks acknowledges Drupal's progress.

The current state of JavaScript in Drupal

Looking back, I'm glad we decided to focus first on improving Drupal's web services APIs; we discovered that there was a lot of work left to stabilize them. Cleanly integrating a JavaScript framework with Drupal would have been challenging 18 months ago. While there is still more work to be done, Drupal 8's available web service APIs have matured significantly.

Furthermore, by not committing to a specific framework, we are seeing Drupal developers explore a range of JavaScript frameworks and members of multiple JavaScript framework communities consuming Drupal's web services. I've seen Drupal 8 used as a content repository behind Angular, Ember, React, Vue, and other JavaScript frameworks. Very cool!

There is a lot to like about how Drupal's web service APIs matured and how we've seen Drupal integrated with a variety of different frameworks. But there is also no denying that not having a JavaScript framework in core came with certain tradeoffs:

  1. It created a barrier for significantly leveling up the Drupal community's JavaScript skills. In my opinion, we still lack sufficient JavaScript expertise among Drupal core contributors. While we do have JavaScript experts working hard to maintain and improve our existing JavaScript code, I would love to see more experts join that team.
  2. It made it harder to accelerate certain improvements to Drupal's authoring and site building experience.
  3. It made it harder to demonstrate how new best practices and certain JavaScript approaches could be leveraged and extended by core and contributed modules to create new Drupal features.

One trend we are now seeing is that traditional MV* frameworks are giving way to component libraries; most people seem to want a way to compose interfaces and interactions with reusable components (e.g. libraries like React, Vue, Polymer, and Glimmer) rather than use a framework with a heavy focus on MV* workflows (e.g. frameworks like Angular and Ember). This means that my original recommendation of Ember needs to be revisited.

Several years later, we still don't know what JavaScript framework will win, if any, and I'm willing to bet that waiting two more years won't give us any more clarity. JavaScript frameworks will continue to evolve and take new shapes. Picking a single one will always be difficult and to some degree "premature". That said, I see React having the most momentum today.

My recommendations at DrupalCon Vienna

Given that it's been almost two years since I last suggested adding a JavaScript framework to core, I decided to bring the topic back in my DrupalCon Vienna keynote presentation. Prior to my keynote, there had been some renewed excitement and momentum behind the idea. Two years later, here is what I recommended we should do next:

  • Invest more in Drupal's API-first initiative. In 2017, there is no denying that decoupled architectures and headless Drupal will be a big part of our future. We need to keep investing in Drupal's web service APIs. At a minimum, we should expand Drupal's web service APIs and standardize on JSON API. Separately, we need to examine how to give API consumers more access to and control over Drupal's capabilities.
  • Embrace all JavaScript frameworks for building Drupal-powered applications. We should give developers the flexibility to use their JavaScript framework of choice when building front-end applications on top of Drupal — so they can use the right tool for the job. The fact that you can front Drupal with Ember, Angular, Vue, React, and others is a great feature. We should also invest in expanding the Waterwheel ecosystem so we have SDKs and references for all these frameworks.
  • Pick a framework for Drupal's own administrative user interfaces. Drupal should pick a JavaScript framework for its own administrative interface. I'm not suggesting we abandon our stable base of PHP code; I'm just suggesting that we leverage JavaScript for the things that JavaScript is great at by moving relevant parts of our code from PHP to JavaScript. Specifically, Drupal's authoring and site building experience could benefit from user experience improvements. A JavaScript framework could make our content modeling, content listing, and configuration tools faster and more application-like by using instantaneous feedback rather than submitting form after form. Furthermore, using a decoupled administrative interface would allow us to dogfood our own web service APIs.
  • Let's start small by redesigning and rebuilding one or two features. Instead of rewriting the entirety of Drupal's administrative user interfaces, let's pick one or two features, and rewrite their UIs using a preselected JavaScript framework. This allows us to learn more about the pros and cons, allows us to dogfood some of our own APIs, and if we ultimately need to switch to another JavaScript framework or approach, it won't be very painful to rewrite or roll the changes back.
Selecting a JavaScript framework for Drupal's administrative UIs

In my keynote, I proposed a new strategic initiative to test and research how Drupal's administrative UX could be improved by using a JavaScript framework. The feedback was very positive.

As a first step, we have to choose which JavaScript framework will be used as part of the research. Following the keynote, we had several meetings at DrupalCon Vienna to discuss the proposed initiative with core committers, all of the JavaScript subsystem maintainers, as well as developers with real-world experience building decoupled applications using Drupal's APIs.

There was unanimous agreement that:

  1. Adding a JavaScript framework to Drupal core is a good idea.
  2. We want to have sufficient real-use experience to make a final decision prior to 8.6.0's development period (Q1 2018). To start, the Watchdog page would be the least intrusive interface to rebuild and would give us important insights before kicking off work on more complex interfaces.
  3. While a few people named alternative options, React was our preferred option, by far, due to its high degree of adoption, component-based and unopinionated nature, and its potential to make Drupal developers' skills more future-proof.
  4. This adoption should be carried out in a limited and incremental way so that the decision is easily reversible if better approaches come later on.

We created an issue on the Drupal core queue to discuss this more.

Conclusion Drupal should support a variety of JavaScript libraries on the user-facing front end while relying on a single shared framework as a standard across Drupal administrative interfaces.

In short, I continue to believe that adopting more JavaScript is important for the future of Drupal. My original recommendation to include a modern JavaScript framework (or JavaScript libraries) for Drupal's administrative user interfaces still stands. I believe we should allow developers to use their JavaScript framework of choice to build front-end applications on top of Drupal and that we can start small with one or two administrative user interfaces.

After meeting with core maintainers, JavaScript subsystem maintainers, and framework managers at DrupalCon Vienna, I believe that React is the right direction to move for Drupal's administrative interfaces, but we encourage everyone in the community to discuss our recommendation. Doing so would allow us to make Drupal easier to use for site builders and content creators in an incremental and reversible way, keep Drupal developers' skills relevant in an increasingly JavaScript-driven world, move us ahead with modern tools for building user interfaces.

Special thanks to Preston So for contributions to this blog post and to Matt Grill, Wim Leers, Jason Enter, Gábor Hojtsy, and Alex Bronstein for their feedback during the writing process.

Categories: Drupal

State of Drupal presentation (September 2017)

27 September 2017 - 4:55am

Yesterday, I shared my State of Drupal presentation at DrupalCon Vienna. In addition to sharing my slides, I wanted to provide some more detail on how Drupal is evolving, who Drupal is for, and what I believe we should focus on.

Drupal is growing and changing

I started my keynote by explaining that Drupal is growing. Over the past year, we've witnessed a rise in community engagement, which has strengthened Drupal 8 adoption.

This is supported by the 2017 Drupal Business Survey; after surveying 239 executives from Drupal agencies, we can see that Drupal 8 has become the defacto release for them and that most of the Drupal businesses report to be growing.

While the transition from Drupal 7 to Drupal 8 is not complete, Drupal 8's innovation continues to accelerate. We've seen the contributed modules ecosystem mature; in the past year, the number of stable modules has more than doubled. Additionally, there are over 4,000 modules in development.

In addition to growth, both the vendor and technology landscapes around Drupal are changing. In my keynote, I noted three primary shifts in the vendor landscape. Single blogs, portfolio sites and brochure sites, which represent the low end of the market, are best served by SaaS tools. On the other side of the spectrum, a majority of enterprise vendors are moving beyond content management into larger marketing suites. Finally, the headless CMS market segment is growing rapidly, with some vendors growing at a rate of 500% year over year.

There are also significant changes in the technology landscape surrounding Drupal, as a rising number of Drupal agencies have also started using modern JavaScript technologies. For example, more than 50% of Drupal agencies are also using Node.js to support the needs of their customers.

While evolving vendor and technology landscapes present many opportunities for Drupal, it can also introduce uncertainty. After listening to many people in the Drupal community, it's clear that all these market and technology trends, combined with the long development and adoption cycle of Drupal 8, has left some wondering what this all means for Drupal, and by extension also for them.

Drupal is no longer for simple sites

Over the past year, I've explained why I believe Drupal is for ambitious digital experiences, in both my DrupalCon Baltimore keynote and on my blog. However, I think it would be valuable to provide more detail on what I mean by "ambitious digital experiences". It's important that we all understand who Drupal is for, because it drives our strategy, which in turn allows us to focus our efforts.

Today, I believe that Drupal is no longer for simple sites. Instead, Drupal's sweetspot is sites or digital experiences that require a certain level of customization or flexibility — something I refer to as "richness".

Ambitious is much more than just enterprise

This distinction is important because I often find that the term "ambitious" becomes conflated with "enterprise". While I agree that Drupal is a great fit for the enterprise, I personally never loved that categorization. It's not just large organizations that use Drupal. Individuals, small startups, universities, museums and nonprofits can be equally ambitious in what they'd like to accomplish and Drupal can be an incredible solution for them.

An example of this could be a small business that manages 50 rental properties. While they don't have a lot of traffic (reach), they require integrations with an e-commerce system, a booking system, and a customer support tool to support their business. Their allotted budget is $50,000 or less. This company would not be considered an enterprise business; however, Drupal would be a great fit for this use case. In many ways, the "non-enterprise ambitious digital experiences" represent the majority of the Drupal ecosystem. As I made clear in my presentation, we don't want to leave those behind.

Addressing the needs of smaller organizations

The Drupal ecosystem majority are organizations with sites that require medium-to-high richness, which SaaS builders cannot support. However, they also don't need to scale at the level of enterprise companies. As the Drupal community continues to consider how we can best support this majority, a lot of smaller Drupal agencies and end-users have pointed out that they would benefit from the following two things:

  1. Powerful site building tools. They want easy-to-use site building tools that are simple to learn, and don't require dozens of contributed modules to be installed and configured. They would also prefer to avoid writing a lot of custom code because their clients have smaller budgets. Great examples of tools that would improve site building are Drupal's upcoming layout builder, workspaces and media library. To make some of Drupal's own administrative UIs more powerful and easier to use, I proposed that we add a modern JavaScript to core.
  2. Easier updates and maintenance. While each Drupal 8 site benefits from continuous innovation, it also needs to be updated more often. The new Drupal 8 release cycle has monthly patch releases and 6-month minor releases. In addition, organizations have to juggle ad-hoc updates from contributed modules. In addition, site updates has often become more complex because our dependency on third-party libraries and because not everyone can use Composer. Many smaller users and agencies would benefit tremendously from auto-updates because maintaining and updating their Drupal 8 sites can be too manual, too complex and too expensive.

The good news is that we have made progress in both improving site builder tools and simplifying updates and maintenance. Keep an eye on future blog posts about these topics. In the meantime, you can watch a recording of my keynote (starting at 22:10), or you can download a copy of my slides (56 MB).

Categories: Drupal

Announcing Node.js on Acquia Cloud

20 September 2017 - 7:48am

Today, Acquia announced that it expanded Acquia Cloud to support Node.js, the popular open-source JavaScript runtime. This is a big milestone for Acquia as it is the first time we have extended our cloud beyond Drupal. I wanted to take some time to explain the evolution of Acquia's open-source stack and why this shift is important for our customers' success.

From client-side JavaScript to server-side JavaScript

JavaScript was created at Netscape in 1995, when Brendan Eich wrote the first version of JavaScript in just 10 days. It took around 10 years for JavaScript to reach enterprise maturity, however. Adoption accelerated in 2004 when Google used JavaScript to build the first release of Gmail. In comparison to e-mail competitors like Yahoo! Mail and Hotmail, Gmail showed what was possible with client-side JavaScript, which enables developers to update pages dynamically and reduces full-page refreshes and round trips to the server. The benefit is an improved user experience that is usually faster, more dynamic in its behavior, and generally more application-like.

In 2009, Google invented the V8 JavaScript engine, which was embedded into its Chrome browser to make both Gmail and Google Maps faster. Ryan Dahl used the V8 run-time as the foundation of Node.js, which enabled server-side JavaScript, breaking the language out of the boundaries of the browser. Node.js is event-driven and provides asynchronous, non-blocking I/O — things that help developers build modern web applications, especially those with real-time capabilities and streamed data. It ushered in the era of isomorphic applications, which means that JavaScript applications can now share code between the client side and server side. The introduction of Node.js has spurred a JavaScript renaissance and contributed to the popularity of JavaScript frameworks such as AngularJS, Ember and React.

Acquia's investment in Headless Drupal

In the web development world, few trends are spreading more rapidly than decoupled architectures using JavaScript frameworks and headless CMS. Decoupled architectures are gaining prominence because architects are looking to take advantage of other front-end technologies, most commonly JavaScript based front ends, in addition to those native to Drupal.

Acquia has been investing in the development of headless Drupal for nearly five years, when we began contributing to the addition of web service APIs to Drupal core. A year ago, we released Waterwheel, an ecosystem of software development kits (SDKs) that enables developers to build Drupal-backed applications in JavaScript and Swift, without needing extensive Drupal expertise. This summer, we released Reservoir, a Drupal distribution for decoupled Drupal. Over the past year, Acquia has helped to support a variety of headless architectures, with and without Node.js. While not always required, Node.js is often used alongside of a headless Drupal application to provide server-side rendering of JavaScript applications or real-time capabilities.

Managed Node.js on Acquia Cloud

Previously, if an organization wanted to build a decoupled architecture with Node.js, it was not able to host the Node.js application on Acquia Cloud. This means that the organization would have to run Node.js with a separate vendor. In many instances, this requires organizations to monitor, troubleshoot and patch the infrastructure supporting the Node.js application of their own accord. Separating the management of the Node.js application and Drupal back end not only introduces a variety of complexities, including security risk and governance challenges, but it also creates operational strain. Organizations must rely on two vendors, two support teams, and multiple contacts to build decoupled applications using Drupal and Node.js.

To eliminate this inefficiency, Acquia Cloud can now support both Drupal and Node.js. Our goal is to offer the best platform for developing and running Drupal and Node.js applications. This means that organizations only need to rely on one vendor and one cloud infrastructure when using Drupal and Node.js. Customers can access Drupal and Node.js environments from a single user interface, in addition to tools that enable continuous delivery, continuous integration, monitoring, alerting and support across both Drupal and Node.js.

On Acquia Cloud, customers can access Drupal and Node.js environments from a single user interface. Delivering on Acquia's mission

When reflecting on Acquia's first decade this past summer, I shared that one of the original corporate values our small team dreamed up was to "empower everyone to rapidly assemble killer websites". After ten years, we've evolved our mission to "build the universal platform for the world's greatest digital experiences". While our focus has expanded as we've grown, Acquia's enduring aim is to provide our customers with the best tools available. Adding Node.js to Acquia Cloud is a natural evolution of our mission.

Categories: Drupal

Don't blame open-source software for poor security practices

15 September 2017 - 7:28am

Last week, Equifax, one of the largest American credit agencies, was hit by a cyberattack that may have compromised the personal data of nearly 143 million people, including name, address, social security numbers, birth dates and more. The forfeited information reveals everything required to steal someone's identity or to take out a loan in someone else's name. Considering that the current US population is 321 million, this cyberattack is now considered to be one of the largest and most intrusive breaches in US history.

It's Equifax that is to blame, not open-source

As Equifax began to examine how the breach occurred, many unsubstantiated reports and theories surfaced in an attempt to pinpoint the vulnerability. One such theory targeted Apache Struts as the software responsible for the breach. Because Apache Struts is an open-source framework used for developing Java applications, this resulted in some unwarranted open-source shaming.

Yesterday, Equifax confirmed that the security breach was due to an Apache Struts vulnerability. However, here is what is important; it wasn't because Apache Struts is open-source or because open-source is less secure. Equifax was hacked because the firm failed to patch a well-known Apache Struts flaw that was disclosed months earlier in March. Running an old, insecure version of software — open-source or proprietary — can and will jeopardize the security of any site. It's Equifax that is to blame, not open-source.

The importance of keeping software up-to-date

The Equifax breach is a good reminder of why organizations need to remain vigilant about properly maintaining and updating their software, especially when security vulnerabilities have been disclosed. In an ideal world, software would update itself the moment a security patch is released. WordPress, for example, offers automatic updates in an effort to promote better security, and to streamline the update experience overall. It would be interesting to consider automatic security updates for Drupal (just for patch releases, not for minor or major releases).

In absence of automatic updates, I would encourage users to work with PaaS companies that keep not only your infrastructure secure, but also your Drupal application code. Too many organizations underestimate the effort and expertise it takes to do it themselves.

At Acquia, we provide customers with automatic security patching of both the infrastructure and Drupal code. We monitor our customers' sites for intrusion attempts, DDoS attacks, and other suspicious activity. If you prefer to do the security patching yourself, we offer continuous integration or continuous delivery tools that enable you to get security patches into production in minutes rather than weeks or months. We take pride in assisting our customers to keep their sites current with the latest patches and upgrades; it's good for our customers and helps dispel the myth that open-source software is more susceptible to security breaches.

Categories: Drupal

Don't blame open-source software for poor security practices

15 September 2017 - 7:28am

Last week, Equifax, one of the largest American credit agencies, was hit by a cyber attack that may have compromised the personal data of nearly 143 million people, including name, address, social security numbers, birthdates and more. The forfeited information reveals everything required to steal someone's identity or to take out a loan on someone else's name. Considering that the current US population is 321 million, this cyberattack is now considered to be one of the largest and most intrusive breaches in US history.

It's Equifax that is to blame, not open-source

A security breach of this scale warrants serious concern. As Equifax began to examine how the breach occurred, many unsubstantiated reports and theories surfaced in an attempt to pinpoint the vulnerability. One such theory targeted Apache Struts as the software responsible for the the breach. Because Apache Struts is an open-source framework used for developing Java applications, this resulted in some unwarranted open source shaming.

Yesterday, Equifax confirmed that the security breach was due to an Apache Struts vulnerability. However, here is what is important; it wasn't because Apache Struts is open-source or because open-source is less secure. Equifax was hacked because the firm failed to patch a well-know Apache Struts flaw that was disclosed months earlier in March. Running an old, insecure version of software — open-source or proprietary — can and will jeopardize the security of any site. It's Equifax that is to blame, not open-source.

The importance of keeping software up-to-date

The Equifax breach is a good reminder of why organizations need to remain vigilant about properly maintaining and updating their software, especially when security vulnerabilities have been disclosed. In an ideal world, software would update itself the moment a security patch is released. Wordpress, for example, offers automatic updates in an effort to promote better security, and to streamline the update experience overall. It would be interesting to consider automatic security updates for Drupal (just for patch releases, not for minor or major releases).

In absence of automatic updates, I would encourage users to work with PaaS companies that keep not only your infrastructure secure, but also your Drupal application code. Too many organizations underestimate the effort and expertise it takes to do it themselves.

At Acquia, we provide customers with automatic security patching of both the infrastructure and Drupal code. We monitor our customers sites for intrusion attempts, DDoS attacks, and other suspicious activity. If you prefer to do the security patching yourself, we offer continuous integration or continuous delivery tools that enable you to get security patches into production in minutes rather than weeks or months. We take pride in assisting our customers to keep their sites current with the latest patches and upgrades; it’s good for our customers and helps dispel the myth that open-source software is more susceptible to security breaches.

Categories: Drupal

Who sponsors Drupal development? (2016-2017 edition)

13 September 2017 - 4:46am

Last year, Matthew Tift and I examined Drupal.org's commit data to understand who develops Drupal, how much of that work is sponsored, and where that sponsorship comes from. We published our analysis in a blog post called "Who Sponsors Drupal Development?". A year later, I wanted to present an update. This year's report will also cover additional data, including gender and geographical diversity, and project sponsorship.

Understanding how an open-source project works is important because it establishes a benchmark for project health and scalability. Scaling an open-source project is a difficult task. As an open-source project's rate of adoption grows, the number of people that benefit from the project also increases. Often the open-source project also becomes more complex as it expands, which means that the economic reward of helping to improve the project decreases.

A recent article on the Bitcoin and Ethereum contributor communities illustrates this disparity perfectly. Ethereum and Bitcoin have market capitalizations valued at $30 billion and $70 billion, respectively. However, both projects have fewer than 40 meaningful contributors, and contribution isn't growing despite the rising popularity of cryptocurrency.

According to Bitcoin's GitHub data, Bitcoin has less than 40 active contributors.According to Ethereum's GitHub data, Ethereum has less than 20 active contributors.

Drupal, by comparison, has a diverse community of contributors. In the 12-month period between July 1, 2016 to June 30, 2017 we saw code contributions on Drupal.org from 7,240 different individuals and 889 different companies. This does not mean that Drupal is exempt from the challenges of scaling an open-source project. We hope that this report provides transparency about Drupal project development and encourages more individuals and organizations incentive to contribute. We also will highlight areas where our community can and should do better.

What is the Drupal.org credit system?

In the spring of 2015, after proposing ideas for giving credit and discussing various approaches at length, Drupal.org added the ability for people to attribute their work to an organization or customer in the Drupal.org issue queues. Maintainers of Drupal modules, themes and distributions can award issues credits to people who help resolve issues with code, translations, documentation, design and more.

A screenshot of an issue comment on Drupal.org. You can see that jamadar worked on this patch as a volunteer, but also as part of his day job working for TATA Consultancy Services on behalf of their customer, Pfizer.

Credits are a powerful motivator for both individuals and organizations. Accumulating credits provides individuals with a way to showcase their expertise. Organizations can utilize credits to help recruit developers or to increase their visibility in the Drupal.org marketplace.

While the benefits are evident, it is important to note a few of the limitations in Drupal.org's current credit system:

  • Contributing to issues on Drupal.org is not the only way to contribute. Other activities, such as sponsoring events, promoting Drupal, and providing help and mentorship, are important to the long-term health of the Drupal project. Many of these activities are not currently captured by the credit system. For this post, we chose to only look at code contributions.
  • We acknowledge that parts of Drupal are developed on GitHub and therefore aren't fully credited on Drupal.org. The actual number of contributions and contributors could be significantly higher than what we report.
  • Even when development is done on Drupal.org, the credit system is not used consistently; because using the credit system is optional, a lot of code committed on Drupal.org has no or incomplete contribution credits.
  • Not all code credits are the same. We currently don't have a way to account for the complexity and quality of contributions; one person might have worked several weeks for just one credit, while another person might receive a credit for ten minutes of work. In the future, we should consider issuing credit data in conjunction with issue priority, patch size, etc. We can also reduce the need for trivial credits by automating patch rerolls and automating coding style fixes.
Who is working on Drupal?

For our analysis we looked at all the issues that were marked "closed" or "fixed" in the 12-month period from July 1, 2016 to June 30, 2017. What we learned is that there were 23,238 issues marked "closed" or "fixed", a 22% increase from the 19,095 issues in the 2015-2016 period. Those 23,238 issues had 42,449 issue credits, a 30% increase from the 32,711 issue credits recorded in the previous year. Issue credits against Drupal core remained roughly the same year over year, meaning almost all of this growth came from increased activity in contributed projects. This is no surprise. Drupal development is cyclical, and during this period of the Drupal 8 development cycle, most of the Drupal community has been focused on porting modules from Drupal 7 to Drupal 8. Of the 42,449 issue credits reported this year, 20% (8,619 credits) were for Drupal core, while 80% (33,830 credits) went to contributed themes, modules and distributions.

Compared to the previous year, we also saw an increase in both the number of people contributing and the number of organizations contributing. Drupal.org received code contributions from 7,240 different individuals and 889 different organizations.

The number of individual contributors is up 28% year over year and the number of organizations contributing is up 26% year over year.

While the number of individual contributors rose, a relatively small number of individuals still do the majority of the work. Approximately 47% of individual contributors received just one credit. Meanwhile, the top 30 contributors (the top 0.4%) account for over 17% of the total credits, indicating that these individuals put an incredible amount of time and effort in developing Drupal and its contributed projects:

RankUsernameIssues1jrockowitz5372dawehner4213RenatoG4084bojanz3515Berdir3356mglaman3347Wim Leers3328alexpott3299DamienMcKenna24510jhodgdon24211drunken monkey23812naveenvalecha19613Munavijayalakshmi19214borisson_19115yongt941218916klausi18517Sam15218418miro_dietiker18219Pavan B S18020ajay_reddy17621phenaproxima17222sanchiz16223slashrsm16124jhedstrom15525xjm15126catch14727larowlan14528rakesh.gectcr14129benjy13930dhruveshdtripathi138

Out of the top 30 contributors featured, 19 were also recognized as top contributors in our 2015-2016 report. These Drupalists' dedication and continued contribution to the project has been crucial to Drupal's development. It's also exciting to see 11 new names on the list. This mobility is a testament to the community's evolution and growth.

Next, we looked at both the gender and geographic diversity of Drupal.org code contributors. While these are only two examples of diversity, this is the only available data that contributors can choose to share on their Drupal.org profiles. The reported data shows that only 6% of the recorded contributions were made by contributors that identify as female, which indicates a steep gender gap. Like in most open-source projects, the gender imbalance in Drupal is profound and underscores the need to continue fostering diversity and inclusion in our community.

The gender representation behind the issue credits. Only 6% of the recorded contributions are by women. When measuring geographic diversity, we saw individual contributors from 6 different continents and 116 different countries: The top 20 countries from which contributions originate. The data is compiled by aggregating the countries of all individual contributors behind each commit. Note that the geographical location of contributors doesn't always correspond with the origin of their sponsorship. Wim Leers, for example, works from Belgium, but his funding comes from Acquia, which has the majority of its customers in North America.How much of the work is sponsored?

Drupal is used by more than one million websites. The vast majority of the individuals and organizations behind these Drupal websites never participate in the development of the project. They might use the software as it is or might not feel the need to help drive its development. We have to provide more incentive for these individuals and organizations to contribute back to the project.

Issue credits can be marked as "volunteer" and "sponsored" simultaneously (shown in jamadar's screenshot near the top of this post). This could be the case when a contributor does the minimum required work to satisfy the customer's need, in addition to using their spare time to add extra functionality.

While Drupal started out as a 100% volunteer-driven project, today the majority of the code on Drupal.org is sponsored by organizations. Only 11% of the commit credits that we examined in 2016-2017 were "purely volunteer" credits (4,498 credits), in stark contrast to the 46% that were "purely sponsored". In other words, there were four times as many "purely sponsored" credits as "purely volunteer" credits.

A few comparisons with the 2015-2016 data:

  • The credit system is used more. Between July 1, 2015 and June 30, 2016, 37% of all credits had no attribution while in the more recent period between July 1, 2016 to June 30, 2017, only 28% of credits lacked attribution. More people have become aware of the credit system, the attribution options, and their benefits.
  • Sponsored credits are growing faster than volunteer credits. Both "purely volunteer" and "purely sponsored" credits grew, but "purely sponsored" credits grew faster. There are two reasons why this could be the case: (1) more contributions are sponsored and (2) organizations are more likely to use the credit system compared to volunteers.

No data is perfect, but it feels safe to conclude that most of the work on Drupal is sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal. Maybe most importantly, while the number of volunteers and sponsors has grown year over year in absolute terms, sponsored contributions appear to be growing faster than volunteer contributions. This is consistent with how open source projects grow and scale.

Who is sponsoring the work?

Now that we have established that most of the work on Drupal is sponsored, we want to study which organizations contribute to Drupal. While 889 different organizations contributed to Drupal, approximately 50% of them received four credits or fewer. The top 30 organizations (roughly the top 3%) account for about 48% of the total credits, which implies that the top 30 companies play a crucial role in the health of the Drupal project. The graph below shows the top 30 organizations and the number of credits they received between July 1, 2016 and June 30, 2017:

The top 30 contributing organizations based on the number of Drupal.org commit credits.

While not immediately obvious from the graph above, different types of companies are active in Drupal's ecosystem:

Category Description Traditional Drupal businesses Small-to-medium-sized professional services companies that make money primarily using Drupal. They typically employ fewer than 100 employees, and because they specialize in Drupal, many of these professional services companies contribute frequently and are a huge part of our community. Examples are Chapter Three (shown on graph) and Lullabot (shown on graph). Digital marketing agencies Larger full-service agencies that have marketing-led practices using a variety of tools, typically including Drupal, Adobe Experience Manager, Sitecore, WordPress, etc. They tend to be larger, with the larger agencies employing thousands of people. Examples are Wunderman and Mirum. System integrators Larger companies that specialize in bringing together different technologies into one solution. Example system agencies are Accenture, TATA Consultancy Services, Capgemini and CI&T. Technology and infrastructure companies Examples are Acquia (shown on graph), Lingotek, BlackMesh, Rackspace, Pantheon and Platform.sh. End-users Examples are Pfizer (shown on graph) or NBCUniversal.

A few observations:

  • Almost all of the sponsors in the top 30 are traditional Drupal businesses. Companies like MD Systems (12 employees), Valuebound (34 employees), Chapter Three (27 employees), Commerce Guys (7 employees) and PreviousNext (20 employees) are, despite their size, critical to Drupal's success.

    It's worth highlighting MD Systems, which ranks second in the list of the top 30 contributing organizations, and is the number-one contributor among traditional Drupal businesses. What distinguishes MD Systems from most others is that it has embedded contribution into its corporate philosophy. For every commercial project, MD Systems invests 20% of that project's value back into Drupal. They believe that using commercial projects as the foundation for community contribution leads to more meaningful and healthier contributions for Drupal and a lower total cost of ownership for their customers. This is different from other organizations, where employees are allotted a number of hours per month to contribute outside of customer-facing projects. There is no denying that MD Systems has had a tremendous impact on the Drupal community with contributions that are both frequent and impactful.

  • Compared to these traditional Drupal businesses, Acquia has nearly 800 employees and several full-time Drupal contributors. Acquia's Office of the CTO (OCTO) works to resolve some of the most complex issues on Drupal.org, many of which are not recognized by the credit system (e.g. release management, communication, sprint organizing, and project coordination). However, I believe that Acquia should contribute even more due to our comparative size.
  • No digital marketing agencies show up in the top 30, though some of them are starting to contribute. It is exciting that an increasing number of digital marketing agencies are delivering beautiful experiences using Drupal. As a community, we need to work to ensure that each of these firms are contributing back to the project with the same commitment that we see from firms like Chapter Three, MD Systems or CI&T.
  • The only system integrator in the top 30 is CI&T, which ranked 6th with 664 credits. As far as system integrators are concerned, CI&T is a smaller player with approximately 2,500 employees. However, we do see various system integrators outside of the top 30, including Globant, Capgemini, Sapient and TATA Consultancy Services. Each of these system integrators reported 30 to 70 credits in the past year. Finally, Wipro began contributing this year with 2 credits. We expect, or hope, to see system integrators contribute more and more, especially given the number of Drupal developers they employ. Many have sizable Drupal practices with hundreds of Drupal developers, yet contributing to open source is relatively new and often not well-understood.
  • Infrastructure and software companies play an important role in our community, yet only Acquia appears in the top 30. While Acquia has a professional services division, 75% of the contributions come from the product organization (including the Office of the CTO and the Acquia Lightning team). Other contributing infrastructure companies include Pantheon and Platform.sh, which are both venture-backed platform-as-a-service companies that originated from the Drupal community. Pantheon has 17 credits and Platform.sh has 47 credits. Amazee Labs, who is building an infrastructure business, reported 51 credits. Rackspace is a public company hosting thousands of Drupal sites; they have 48 credits. Lingotek offers cloud-based translation management software and has 94 credits.
  • We saw two end-users in the top 30 corporate sponsors: Pfizer (251 credits, up from 158 credits the year before) and the German company bio.logis (212 credits). Other notable customers outside of the top 30 were Workday, Wolters Kluwer, Burda Media, University of Colorado Boulder, YMCA and OpenY, CARD.com and NBCUniversal.
Sponsored code contributions to Drupal.org from technology and infrastructure companies. The chart does not reflect sponsored code contributions on GitHub, Drupal event sponsorship, and the many forms of value that these companies add to Drupal and other open-source communities.

We can conclude that technology and infrastructure companies, digital marketing agencies, system integrators and end-users are not meaningfully contributing code to Drupal.org today. How can we explain this disparity in comparison to traditional Drupal businesses who contribute the most? We believe the biggest reasons are:

  1. Drupal's strategic importance. A variety of the traditional Drupal agencies have been involved with Drupal for 10 years and almost entirely depend on Drupal to support their business. Given both their expertise and dependence on Drupal, they are most likely to look after Drupal's development and well-being. These organizations are typically recognized as Drupal experts and are sought out by organizations that want to build a Drupal website. Contrast this with most of the digital marketing agencies and system integrators who are sized to work with a diversified portfolio of content management platforms and who are historically only getting started with Drupal and open source. They deliver digital marketing solutions and aren't necessarily sought out for their Drupal expertise. As their Drupal practices grow in size and importance, this could change. In fact, contributing to Drupal can help grow their Drupal business because it helps their name stand out as Drupal experts and gives them a competitive edge with their customers.
  2. The level of experience with Drupal and open source. Drupal aside, many organizations have little or no experience with open source, so it is important that we motivate and teach them to contribute.
  3. Legal reservations. We recognize that some organizations are not legally permitted to contribute, let alone attribute their customers. We hope that will change as open source continues to get adopted.
  4. Tools and process barriers. Drupal contribution still involves a patch-based workflow on Drupal.org's unique issue queue system. This presents a fairly steep learning curve to most developers, who primarily work with more modern and common tools such as GitHub. Getting the code change proposal uploaded is just the first step; getting code changes accepted into an upstream Drupal project — especially Drupal core — is hard work. Peer reviews, gates such as automated testing and documentation, required sign-offs from maintainers and committers, knowledge of best practices and other community norms are a few of the challenges a contributor must face to get code accepted into Drupal.

Consequently, this data shows that the Drupal community can do more to entice companies to contribute code to Drupal.org. The Drupal community has a long tradition of encouraging organizations to share code rather than keep it behind firewalls. While the spirit of the Drupal project cannot be reduced to any single ideology — not every organization can or will share their code — we would like to see organizations continue to prioritize collaboration over individual ownership. Our aim is not to criticize those who do not contribute, but rather to help foster an environment worthy of contribution. Given the vast amount of Drupal users, we believe continuing to encourage organizations and end-users to contribute could be a big opportunity.

There are substantial benefits and business drivers for organizations that contribute: (1) it improves their ability to sell and win deals and (2) it improves their ability to hire. Companies that contribute to Drupal tend to promote their contributions in RFPs and sales pitches. Contributing to Drupal also results in being recognized as a great place to work for Drupal experts.

The uneasy alliance with corporate contributions

As mentioned above, when community-driven open-source projects grow, there is a bigger need for organizations to help drive their development. It almost always creates an uneasy alliance between volunteers and corporations.

This theory played out in the Linux community well before it played out in the Drupal community. The Linux project is 25 years old and has seen a steady increase in the number of corporate contributors for roughly 20 years. While Linux companies like Red Hat and SUSE rank high on the contribution list, so do non-Linux-centric companies such as Samsung, Intel, Oracle and Google. All of these corporate contributors are (or were) using Linux as an integral part of their business.

The 889 organizations that contribute to Drupal (which includes corporations) is more than four times the number of organizations that sponsor development of the Linux kernel. This is significant because Linux is considered "one of the largest cooperative software projects ever attempted". In fairness, Linux has a different ecosystem than Drupal. The Linux business ecosystem has various large organizations (Red Hat, Google, Intel, IBM and SUSE) for whom Linux is very strategic. As a result, many of them employ dozens of full-time Linux contributors and invest millions of dollars in Linux each year.

What projects have sponsors?

In total, the Drupal community worked on 3,183 different projects (modules, themes and distributions) in the 12-month period between July 1, 2016 to June 30, 2017. To understand where the organizations sponsoring Drupal put their money, I've listed the top 20 most sponsored projects:

RankProject nameIssues1Drupal core47452Drupal Commerce (distribution)5263Webform3614Open Y (distribution)3245Paragraphs2316Inmail2237User guide2188JSON API2049Paragraphs collection20010Entity browser19611Diff19012Group17013Metatag15714Facets15515Commerce Point of Sale (PoS)14716Search API14317Open Social (distribution)13318Drupal voor Gemeenten (distribution)13119Solr Search12220Geolocation field118
Who is sponsoring the top 30 contributors? Rank Username Issues Volunteer Sponsored Not specified Sponsors 1 jrockowitz 537 88% 45% 9% The Big Blue House (239), Kennesaw State University (6), Memorial Sloan Kettering Cancer Center (4) 2 dawehner 421 67% 83% 5% Chapter Three (328), Tag1 Consulting (19), Drupal Association (12), Acquia (5), Comm-press (1) 3 RenatoG 408 0% 100% 0% CI&T (408) 4 bojanz 351 0% 95% 5% Commerce Guys (335), Adapt A/S (38), Bluespark (2) 5 Berdir 335 0% 93% 7% MD Systems (310), Acquia (7) 6 mglaman 334 3% 97% 1% Commerce Guys (319), Thinkbean, LLC (48), LivePerson, Inc (46), Bluespark (22), Universal Music Group (16), Gaggle.net, Inc. (3), Bluehorn Digital (1) 7 Wim Leers 332 14% 87% 2% Acquia (290) 8 alexpott 329 7% 99% 1% Chapter Three (326), TES Global (1) 9 DamienMcKenna 245 2% 95% 4% Mediacurrent (232) 10 jhodgdon 242 0% 1% 99% Drupal Association (2), Poplar ProductivityWare (2) 11 drunken monkey 238 95% 11% 1% Acquia (17), Vizala (8), Wunder Group (1), Sunlime IT Services GmbH (1) 12 naveenvalecha 196 74% 55% 1% Acquia (152), Google Summer of Code (7), QED42 (1) 13 Munavijayalakshmi 192 0% 100% 0% Valuebound (192) 14 borisson_ 191 66% 39% 22% Dazzle (70), Acquia (6) 15 yongt9412 189 0% 97% 3% MD Systems (183), Acquia (6) 16 klausi 185 9% 61% 32% epiqo (112) 17 Sam152 184 59% 92% 7% PreviousNext (168), amaysim Australia Ltd. (5), Code Drop (2) 18 miro_dietiker 182 0% 99% 1% MD Systems (181) 19 Pavan B S 180 0% 98% 2% Valuebound (177) 20 ajay_reddy 176 100% 99% 0% Valuebound (180), Drupal Bangalore Community (154) 21 phenaproxima 172 0% 99% 1% Acquia (170) 22 sanchiz 162 0% 99% 1% Drupal Ukraine Community (107), Vinzon (101), FFW (60), Open Y (52) 23 slashrsm 161 6% 95% 3% MD Systems (153), Acquia (47) 24 jhedstrom 155 4% 92% 4% Phase2 (143), Workday, Inc. (134), Memorial Sloan Kettering Cancer Center (1) 25 xjm 151 0% 91% 9% Acquia (137) 26 catch 147 3% 83% 16% Third and Grove (116), Tag1 Consulting (6) 27 larowlan 145 12% 92% 7% PreviousNext (133), University of Technology, Sydney (30), amaysim Australia Ltd. (6), Australian Competition and Consumer Commission (ACCC) (1), Department of Justice & Regulation, Victoria (1) 28 rakesh.gectcr 141 100% 91% 0% Valuebound (128) 29 benjy 139 0% 94% 6% PreviousNext (129), Brisbane City Council (8), Code Drop (1) 30 dhruveshdtripathi 138 15% 100% 0% DevsAdda (138), OpenSense Labs (44)

We observe that the top 30 contributors are sponsored by 46 organizations. This kind of diversity is aligned with our desire not to see Drupal controlled by a single organization. These top contributors and organizations are from many different parts of the world and work with customers large and small. Nonetheless, we will continue to benefit from more diversity.

Evolving the credit system

Like Drupal itself, the credit system on Drupal.org is an evolving tool. Ultimately, the credit system will only be useful when the community uses it, understands its shortcomings, and suggests constructive improvements. In highlighting the organizations that sponsor the development of code on Drupal.org, we hope to elicit responses that help evolve the credit system into something that incentivizes business to sponsor more work and enables more people to participate in our community, learn from others, teach newcomers and make positive contributions. Drupal is a positive force for change and we wish to use the credit system to highlight (at least some of) the work of our diverse community, which includes volunteers, companies, nonprofits, governments, schools, universities, individuals, and other groups.

One of the challenges with the existing credit system is it has no way of "weighting" contributions. A typo fix counts just as much as giving multiple detailed technical reviews on a critical core issue. This appears to have the effect of incentivizing organizations' employees to work on "lower-hanging fruit issues", because this bumps their companies' names in the rankings. One way to help address this might be to adjust the credit ranking algorithm to consider things such as issue priority, patch size, and so on. This could help incentivize companies to work on larger and more important problems and save coding standards improvements for new contributor sprints. Implementing a scoring system that ranks the complexity of an issue would also allow us to develop more accurate reports of contributed work.

Conclusion

Our data confirms Drupal is a vibrant community full of contributors who are constantly evolving and improving the software. While we have amazing geographic diversity, we need greater gender diversity. Our analysis of the Drupal.org credit data concludes that most contributions to Drupal are sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal.

As a community, we need to understand that a healthy open-source ecosystem includes more than traditional Drupal businesses that contribute the most. For example, we don't see a lot of contribution from the larger digital marketing agencies, system integrators, technology companies, or end-users of Drupal — we believe that might come as these organizations build out their Drupal practices and Drupal becomes more strategic for them.

To grow and sustain Drupal, we should support those that contribute to Drupal and find ways to get those that are not contributing involved in our community. We invite you to help us continue to strengthen our ecosystem.

Special thanks to Tim Lehnen and Neil Drumm from the Drupal Association for providing us with the Drupal.org credit system data and for supporting us during our research. I would also like to extend a special thanks to Matthew Tift for helping to lay the foundation for this research, collaborating on last year's blog post, and for reviewing this year's edition. Finally, thanks to Angie Byron, Gábor Hojtsy, Jess (xjm), Preston So, Ted Bowman, Wim Leers and Gigi Anderson for providing feedback during the writing process.

Categories: Drupal

Who sponsors Drupal development? (2016-2017 edition)

13 September 2017 - 4:46am

Last year, Matthew Tift and I examined Drupal.org's commit data to understand who develops Drupal, how much of that work is sponsored, and where that sponsorship comes from. We published our analysis in a blog post called "Who Sponsors Drupal Development?". A year later, I wanted to present an update. This year's report will also cover additional data, including gender and geographical diversity, and project sponsorship.

Understanding how an open-source project works is important because it establishes a benchmark for project health and scalability. Scaling an open-source project is a difficult task. As an open-source project's rate of adoption grows, the number of people that benefit from the project also increases. Often the open-source project also becomes more complex as it expands, which means that the economic reward of helping to improve the project decreases.

A recent article on the Bitcoin and Ethereum contributor communities illustrates this disparity perfectly. Ethereum and Bitcoin have market capitalizations valued at $30 billion and $70 billion, respectively. However, both projects have fewer than 40 meaningful contributors, and contribution isn't growing despite the rising popularity of cryptocurrency.

According to Bitcoin's GitHub data, Bitcoin has less than 40 active contributors.According to Ethereum's GitHub data, Ethereum has less than 20 active contributors.

Drupal, by comparison, has a diverse community of contributors. In the 12-month period between July 1, 2016 to June 30, 2017 we saw code contributions on Drupal.org from 7,240 different individuals and 889 different companies. This does not mean that Drupal is exempt from the challenges of scaling an open-source project. We hope that this report provides transparency about Drupal project development and encourages more individuals and organizations incentive to contribute. We also will highlight areas where our community can and should do better.

What is the Drupal.org credit system?

In the spring of 2015, after proposing ideas for giving credit and discussing various approaches at length, Drupal.org added the ability for people to attribute their work to an organization or customer in the Drupal.org issue queues. Maintainers of Drupal modules, themes and distributions can award issues credits to people who help resolve issues with code, translations, documentation, design and more.

A screenshot of an issue comment on Drupal.org. You can see that jamadar worked on this patch as a volunteer, but also as part of his day job working for TATA Consultancy Services on behalf of their customer, Pfizer.

Credits are a powerful motivator for both individuals and organizations. Accumulating credits provides individuals with a way to showcase their expertise. Organizations can utilize credits to help recruit developers or to increase their visibility in the Drupal.org marketplace.

While the benefits are evident, it is important to note a few of the limitations in Drupal.org's current credit system:

  • Contributing to issues on Drupal.org is not the only way to contribute. Other activities, such as sponsoring events, promoting Drupal, and providing help and mentorship, are important to the long-term health of the Drupal project. Many of these activities are not currently captured by the credit system. For this post, we chose to only look at code contributions.
  • We acknowledge that parts of Drupal are developed on GitHub and therefore aren't fully credited on Drupal.org. The actual number of contributions and contributors could be significantly higher than what we report.
  • Even when development is done on Drupal.org, the credit system is not used consistently; because using the credit system is optional, a lot of code committed on Drupal.org has no or incomplete contribution credits.
  • Not all code credits are the same. We currently don't have a way to account for the complexity and quality of contributions; one person might have worked several weeks for just one credit, while another person might receive a credit for ten minutes of work. In the future, we should consider issuing credit data in conjunction with issue priority, patch size, etc. We can also reduce the need for trivial credits by automating patch rerolls and automating coding style fixes.
Who is working on Drupal?

For our analysis we looked at all the issues that were marked "closed" or "fixed" in the 12-month period from July 1, 2016 to June 30, 2017. What we learned is that there were 23,238 issues marked "closed" or "fixed", a 22% increase from the 19,095 issues in the 2015-2016 period. Those 23,238 issues had 42,449 issue credits, a 30% increase from the 32,711 issue credits recorded in the previous year. Issue credits against Drupal core remained roughly the same year over year, meaning almost all of this growth came from increased activity in contributed projects. This is no surprise. Drupal development is cyclical, and during this period of the Drupal 8 development cycle, most of the Drupal community has been focused on porting modules from Drupal 7 to Drupal 8. Of the 42,449 issue credits reported this year, 20% (8,619 credits) were for Drupal core, while 80% (33,830 credits) went to contributed themes, modules and distributions.

Compared to the previous year, we also saw an increase in both the number of people contributing and the number of organizations contributing. Drupal.org received code contributions from 7,240 different individuals and 889 different organizations.

The number of individual contributors is up 28% year over year and the number of organizations contributing is up 26% year over year.

While the number of individual contributors rose, a relatively small number of individuals still do the majority of the work. Approximately 47% of individual contributors received just one credit. Meanwhile, the top 30 contributors (the top 0.4%) account for over 17% of the total credits, indicating that these individuals put an incredible amount of time and effort in developing Drupal and its contributed projects:

RankUsernameIssues1jrockowitz5372dawehner4213RenatoG4084bojanz3515Berdir3356mglaman3347Wim Leers3328alexpott3299DamienMcKenna24510jhodgdon24211drunken monkey23812naveenvalecha19613Munavijayalakshmi19214borisson_19115yongt941218916klausi18517Sam15218418miro_dietiker18219Pavan B S18020ajay_reddy17621phenaproxima17222sanchiz16223slashrsm16124jhedstrom15525xjm15126catch14727larowlan14528rakesh.gectcr14129benjy13930dhruveshdtripathi138

Out of the top 30 contributors featured, 19 were also recognized as top contributors in our 2015-2016 report. These Drupalists' dedication and continued contribution to the project has been crucial to Drupal's development. It's also exciting to see 11 new names on the list. This mobility is a testament to the community's evolution and growth.

Next, we looked at both the gender and geographic diversity of Drupal.org code contributors. While these are only two examples of diversity, this is the only available data that contributors can choose to share on their Drupal.org profiles. The reported data shows that only 6% of the recorded contributions were made by contributors that identify as female, which indicates a steep gender gap. Like in most open-source projects, the gender imbalance in Drupal is profound and underscores the need to continue fostering diversity and inclusion in our community.

The gender representation behind the issue credits. When measuring geographic diversity, we saw individual contributors from 6 different continents and 116 different countries: The top 20 countries from which contributions originate. The data is compiled by aggregating the countries of all individual contributors behind each commit. Note that the geographical location of contributors doesn't always correspond with the origin of their sponsorship. Wim Leers, for example, works from Belgium, but his funding comes from Acquia, which has the majority of its customers in North America.How much of the work is sponsored?

Drupal is used by more than one million websites. The vast majority of the individuals and organizations behind these Drupal websites never participate in the development of the project. They might use the software as it is or might not feel the need to help drive its development. We have to provide more incentive for these individuals and organizations to contribute back to the project.

Issue credits can be marked as "volunteer" and "sponsored" simultaneously (shown in jamadar's screenshot near the top of this post). This could be the case when a contributor does the minimum required work to satisfy the customer's need, in addition to using their spare time to add extra functionality.

While Drupal started out as a 100% volunteer-driven project, today the majority of the code on Drupal.org is sponsored by organizations. Only 11% of the commit credits that we examined in 2016-2017 were "purely volunteer" credits (4,498 credits), in stark contrast to the 46% that were "purely sponsored". In other words, there were four times as many "purely sponsored" credits as "purely volunteer" credits.

A few comparisons with the 2015-2016 data:

  • The credit system is used more. Between July 1, 2015 and June 30, 2016, 37% of all credits had no attribution while in the more recent period between July 1, 2016 to June 30, 2017, only 28% of credits lacked attribution. More people have become aware of the credit system, the attribution options, and their benefits.
  • Sponsored credits are growing faster than volunteer credits. Both "purely volunteer" and "purely sponsored" credits grew, but "purely sponsored" credits grew faster. There are two reasons why this could be the case: (1) more contributions are sponsored and (2) organizations are more likely to use the credit system compared to volunteers.

No data is perfect, but it feels safe to conclude that most of the work on Drupal is sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal. Maybe most importantly, while the number of volunteers and sponsors has grown year over year in absolute terms, sponsored contributions appear to be growing faster than volunteer contributions. This is consistent with how open source projects grow and scale.

Who is sponsoring the work?

Now that we have established that most of the work on Drupal is sponsored, we want to study which organizations contribute to Drupal. While 889 different organizations contributed to Drupal, approximately 50% of them received four credits or fewer. The top 30 organizations (roughly the top 3%) account for about 48% of the total credits, which implies that the top 30 companies play a crucial role in the health of the Drupal project. The graph below shows the top 30 organizations and the number of credits they received between July 1, 2016 and June 30, 2017:

While not immediately obvious from the graph above, different types of companies are active in Drupal's ecosystem:

Category Description Traditional Drupal businesses Small-to-medium-sized professional services companies that make money primarily using Drupal. They typically employ fewer than 100 employees, and because they specialize in Drupal, many of these professional services companies contribute frequently and are a huge part of our community. Examples are Chapter Three (shown on graph) and Lullabot (shown on graph). Digital marketing agencies Larger full-service agencies that have marketing-led practices using a variety of tools, typically including Drupal, Adobe Experience Manager, Sitecore, WordPress, etc. They tend to be larger, with the larger agencies employing thousands of people. Examples are Wunderman and Mirum. System integrators Larger companies that specialize in bringing together different technologies into one solution. Example system agencies are Accenture, TATA Consultancy Services, Capgemini and CI&T. Technology and infrastructure companies Examples are Acquia (shown on graph), Lingotek, BlackMesh, Rackspace, Pantheon and Platform.sh. End-users Examples are Pfizer (shown on graph) or NBCUniversal.

A few observations:

  • Almost all of the sponsors in the top 30 are traditional Drupal businesses. Companies like MD Systems (12 employees), Valuebound (34 employees), Chapter Three (27 employees), Commerce Guys (7 employees) and PreviousNext (20 employees) are, despite their size, critical to Drupal's success.

    It's worth highlighting MD Systems, which ranks second in the list of the top 30 contributing organizations, and is the number-one contributor among traditional Drupal businesses. What distinguishes MD Systems from most others is that it has embedded contribution into its corporate philosophy. For every commercial project, MD Systems invests 20% of that project's value back into Drupal. They believe that using commercial projects as the foundation for community contribution leads to more meaningful and healthier contributions for Drupal and a lower total cost of ownership for their customers. This is different from other organizations, where employees are allotted a number of hours per month to contribute outside of customer-facing projects. There is no denying that MD Systems has had a tremendous impact on the Drupal community with contributions that are both frequent and impactful.

  • Compared to these traditional Drupal businesses, Acquia has nearly 800 employees and several full-time Drupal contributors. Acquia's Office of the CTO (OCTO) works to resolve some of the most complex issues on Drupal.org, many of which are not recognized by the credit system (e.g. release management, communication, sprint organizing, and project coordination). However, I believe that Acquia should contribute even more due to our comparative size.
  • No digital marketing agencies show up in the top 30, though some of them are starting to contribute. It is exciting that an increasing number of digital marketing agencies are delivering beautiful experiences using Drupal. As a community, we need to work to ensure that each of these firms are contributing back to the project with the same commitment that we see from firms like Chapter Three, MD Systems or CI&T.
  • The only system integrator in the top 30 is CI&T, which ranked 6th with 664 credits. As far as system integrators are concerned, CI&T is a smaller player with approximately 2,500 employees. However, we do see various system integrators outside of the top 30, including Globant, Capgemini, Sapient and TATA Consultancy Services. Each of these system integrators reported 30 to 70 credits in the past year. Finally, Wipro began contributing this year with 2 credits. We expect, or hope, to see system integrators contribute more and more, especially given the number of Drupal developers they employ. Many have sizable Drupal practices with hundreds of Drupal developers, yet contributing to open source is relatively new and often not well-understood.
  • Infrastructure and software companies play an important role in our community, yet only Acquia appears in the top 30. While Acquia has a professional services division, 75% of the contributions come from the product organization (including the Office of the CTO and the Acquia Lightning team). Other contributing infrastructure companies include Pantheon and Platform.sh, which are both venture-backed platform-as-a-service companies that originated from the Drupal community. Pantheon has 17 credits and Platform.sh has 47 credits. Amazee Labs, who is building an infrastructure business, reported 51 credits. Rackspace is a public company hosting thousands of Drupal sites; they have 48 credits. Lingotek offers cloud-based translation management software and has 94 credits.
  • We saw two end-users in the top 30 corporate sponsors: Pfizer (251 credits, up from 158 credits the year before) and the German company bio.logis (212 credits). Other notable customers outside of the top 30 were Workday, Wolters Kluwer, Burda Media, University of Colorado Boulder, YMCA and OpenY, CARD.com and NBCUniversal.
Sponsored code contributions to Drupal.org from technology and infrastructure companies. The chart does not reflect sponsored code contributions on GitHub, Drupal event sponsorship, and the many forms of value that these companies add to Drupal and other open-source communities.

We can conclude that technology and infrastructure companies, digital marketing agencies, system integrators and end-users are not meaningfully contributing code to Drupal.org today. How can we explain this disparity in comparison to traditional Drupal businesses who contribute the most? We believe the biggest reasons are:

  1. Drupal's strategic importance. A variety of the traditional Drupal agencies have been involved with Drupal for 10 years and almost entirely depend on Drupal to support their business. Given both their expertise and dependence on Drupal, they are most likely to look after Drupal's development and well-being. These organizations are typically recognized as Drupal experts and are sought out by organizations that want to build a Drupal website. Contrast this with most of the digital marketing agencies and system integrators who are sized to work with a diversified portfolio of content management platforms and who are historically only getting started with Drupal and open source. They deliver digital marketing solutions and aren't necessarily sought out for their Drupal expertise. As their Drupal practices grow in size and importance, this could change. In fact, contributing to Drupal can help grow their Drupal business because it helps their name stand out as Drupal experts and gives them a competitive edge with their customers.
  2. The level of experience with Drupal and open source. Drupal aside, many organizations have little or no experience with open source, so it is important that we motivate and teach them to contribute.
  3. Legal reservations. We recognize that some organizations are not legally permitted to contribute, let alone attribute their customers. We hope that will change as open source continues to get adopted.
  4. Tools and process barriers. Drupal contribution still involves a patch-based workflow on Drupal.org's unique issue queue system. This presents a fairly steep learning curve to most developers, who primarily work with more modern and common tools such as GitHub. Getting the code change proposal uploaded is just the first step; getting code changes accepted into an upstream Drupal project — especially Drupal core — is hard work. Peer reviews, gates such as automated testing and documentation, required sign-offs from maintainers and committers, knowledge of best practices and other community norms are a few of the challenges a contributor must face to get code accepted into Drupal.

Consequently, this data shows that the Drupal community can do more to entice companies to contribute code to Drupal.org. The Drupal community has a long tradition of encouraging organizations to share code rather than keep it behind firewalls. While the spirit of the Drupal project cannot be reduced to any single ideology — not every organization can or will share their code — we would like to see organizations continue to prioritize collaboration over individual ownership. Our aim is not to criticize those who do not contribute, but rather to help foster an environment worthy of contribution. Given the vast amount of Drupal users, we believe continuing to encourage organizations and end-users to contribute could be a big opportunity.

There are substantial benefits and business drivers for organizations that contribute: (1) it improves their ability to sell and win deals and (2) it improves their ability to hire. Companies that contribute to Drupal tend to promote their contributions in RFPs and sales pitches. Contributing to Drupal also results in being recognized as a great place to work for Drupal experts.

The uneasy alliance with corporate contributions

As mentioned above, when community-driven open-source projects grow, there is a bigger need for organizations to help drive their development. It almost always creates an uneasy alliance between volunteers and corporations.

This theory played out in the Linux community well before it played out in the Drupal community. The Linux project is 25 years old and has seen a steady increase in the number of corporate contributors for roughly 20 years. While Linux companies like Red Hat and SUSE rank high on the contribution list, so do non-Linux-centric companies such as Samsung, Intel, Oracle and Google. All of these corporate contributors are (or were) using Linux as an integral part of their business.

The 889 organizations that contribute to Drupal (which includes corporations) is more than four times the number of organizations that sponsor development of the Linux kernel. This is significant because Linux is considered "one of the largest cooperative software projects ever attempted". In fairness, Linux has a different ecosystem than Drupal. The Linux business ecosystem has various large organizations (Red Hat, Google, Intel, IBM and SUSE) for whom Linux is very strategic. As a result, many of them employ dozens of full-time Linux contributors and invest millions of dollars in Linux each year.

What projects have sponsors?

In total, the Drupal community worked on 3,183 different projects (modules, themes and distributions) in the 12-month period between July 1, 2016 to June 30, 2017. To understand where the organizations sponsoring Drupal put their money, I've listed the top 20 most sponsored projects:

RankUsernameIssues1Drupal core47452Drupal Commerce (distribution)5263Webform3614Open Y (distribution)3245Paragraphs2316Inmail2237User guide2188JSON API2049Paragraphs collection20010Entity browser19611Diff19012Group17013Metatag15714Facets15515Commerce Point of Sale (PoS)14716Search API14317Open Social (distribution)13318Drupal voor Gemeenten (distribution)13119Solr Search12220Geolocation field118
Who is sponsoring the top 30 contributors? Rank Username Issues Volunteer Sponsored Not specified Sponsors 1 jrockowitz 537 88% 45% 9% The Big Blue House (239), Kennesaw State University (6), Memorial Sloan Kettering Cancer Center (4) 2 dawehner 421 67% 83% 5% Chapter Three (328), Tag1 Consulting (19), Drupal Association (12), Acquia (5), Comm-press (1) 3 RenatoG 408 0% 100% 0% CI&T (408) 4 bojanz 351 0% 95% 5% Commerce Guys (335), Adapt A/S (38), Bluespark (2) 5 Berdir 335 0% 93% 7% MD Systems (310), Acquia (7) 6 mglaman 334 3% 97% 1% Commerce Guys (319), Thinkbean, LLC (48), LivePerson, Inc (46), Bluespark (22), Universal Music Group (16), Gaggle.net, Inc. (3), Bluehorn Digital (1) 7 Wim Leers 332 14% 87% 2% Acquia (290) 8 alexpott 329 7% 99% 1% Chapter Three (326), TES Global (1) 9 DamienMcKenna 245 2% 95% 4% Mediacurrent (232) 10 jhodgdon 242 0% 1% 99% Drupal Association (2), Poplar ProductivityWare (2) 11 drunken monkey 238 95% 11% 1% Acquia (17), Vizala (8), Wunder Group (1), Sunlime IT Services GmbH (1) 12 naveenvalecha 196 74% 55% 1% Acquia (152), Google Summer of Code (7), QED42 (1) 13 Munavijayalakshmi 192 0% 100% 0% Valuebound (192) 14 borisson_ 191 66% 39% 22% Dazzle (70), Acquia (6) 15 yongt9412 189 0% 97% 3% MD Systems (183), Acquia (6) 16 klausi 185 9% 61% 32% epiqo (112) 17 Sam152 184 59% 92% 7% PreviousNext (168), amaysim Australia Ltd. (5), Code Drop (2) 18 miro_dietiker 182 0% 99% 1% MD Systems (181) 19 Pavan B S 180 0% 98% 2% Valuebound (177) 20 ajay_reddy 176 100% 99% 0% Valuebound (180), Drupal Bangalore Community (154) 21 phenaproxima 172 0% 99% 1% Acquia (170) 22 sanchiz 162 0% 99% 1% Drupal Ukraine Community (107), Vinzon (101), FFW (60), Open Y (52) 23 slashrsm 161 6% 95% 3% MD Systems (153), Acquia (47) 24 jhedstrom 155 4% 92% 4% Phase2 (143), Workday, Inc. (134), Memorial Sloan Kettering Cancer Center (1) 25 xjm 151 0% 91% 9% Acquia (137) 26 catch 147 3% 83% 16% Third and Grove (116), Tag1 Consulting (6) 27 larowlan 145 12% 92% 7% PreviousNext (133), University of Technology, Sydney (30), amaysim Australia Ltd. (6), Australian Competition and Consumer Commission (ACCC) (1), Department of Justice & Regulation, Victoria (1) 28 rakesh.gectcr 141 100% 91% 0% Valuebound (128) 29 benjy 139 0% 94% 6% PreviousNext (129), Brisbane City Council (8), Code Drop (1) 30 dhruveshdtripathi 138 15% 100% 0% DevsAdda (138), OpenSense Labs (44)

We observe that the top 30 contributors are sponsored by 46 organizations. This kind of diversity is aligned with our desire not to see Drupal controlled by a single organization. These top contributors and organizations are from many different parts of the world and work with customers large and small. Nonetheless, we will continue to benefit from more diversity.

Evolving the credit system

Like Drupal itself, the credit system on Drupal.org is an evolving tool. Ultimately, the credit system will only be useful when the community uses it, understands its shortcomings, and suggests constructive improvements. In highlighting the organizations that sponsor the development of code on Drupal.org, we hope to elicit responses that help evolve the credit system into something that incentivizes business to sponsor more work and enables more people to participate in our community, learn from others, teach newcomers and make positive contributions. Drupal is a positive force for change and we wish to use the credit system to highlight (at least some of) the work of our diverse community, which includes volunteers, companies, nonprofits, governments, schools, universities, individuals, and other groups.

One of the challenges with the existing credit system is it has no way of "weighting" contributions. A typo fix counts just as much as giving multiple detailed technical reviews on a critical core issue. This appears to have the effect of incentivizing organizations' employees to work on "lower-hanging fruit issues", because this bumps their companies' names in the rankings. One way to help address this might be to adjust the credit ranking algorithm to consider things such as issue priority, patch size, and so on. This could help incentivize companies to work on larger and more important problems and save coding standards improvements for new contributor sprints. Implementing a scoring system that ranks the complexity of an issue would also allow us to develop more accurate reports of contributed work.

Conclusion

Our data confirms Drupal is a vibrant community full of contributors who are constantly evolving and improving the software. While we have amazing geographic diversity, we need greater gender diversity. Our analysis of the Drupal.org credit data concludes that most contributions to Drupal are sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal.

As a community, we need to understand that a healthy open-source ecosystem includes more than traditional Drupal businesses that contribute the most. For example, we don't see a lot of contribution from the larger digital marketing agencies, system integrators, technology companies, or end-users of Drupal — we believe that might come as these organizations build out their Drupal practices and Drupal becomes more strategic for them.

To grow and sustain Drupal, we should support those that contribute to Drupal and find ways to get those that are not contributing involved in our community. We invite you to help us continue to strengthen our ecosystem.

Special thanks to Tim Lehnen and Neil Drumm from the Drupal Association for providing us with the Drupal.org credit system data and for supporting us during our research. I would also like to extend a special thanks to Matthew Tift for helping to lay the foundation for this research, collaborating on last year's blog post, and for reviewing this year's edition. Finally, thanks to Angie Byron, Gábor Hojtsy, Jess (xjm), Preston So, Ted Bowman, Wim Leers and Gigi Anderson for providing feedback during the writing process.

Categories: Drupal

Entertainment Tonight using Drupal

30 August 2017 - 6:11pm

Entertainment Tonight is the number one entertainment news magazine in the world, and has been on the air for over 30 years. Fans around the world rely on Entertainment Tonight to receive news and updates on their favorite celebrities and stars. I recently discovered that the newest star featured on Entertainment Tonight was Drupal 8!

Entertainment Tonight's new Drupal 8 website, ETOnline.com, receives 19 million monthly unique visitors, making it the second most visited entertainment news website.

Chapter Three helped build the site. This project really helped Drupal 8 move forward because they ported many modules to Drupal 8 in the process. Check it out at http://www.etonline.com!

Categories: Drupal

Entertainment Tonight using Drupal

30 August 2017 - 6:11pm

Entertainment Tonight is the number one entertainment news magazine in the world, and has been on the air for over 30 years. Fans around the world rely on Entertainment Tonight to receive news and updates on their favorite celebrities and stars. I recently discovered that the newest star featured on Entertainment Tonight was Drupal 8!

Entertainment Tonight's new Drupal 8 website, ETOnline.com, receives 19 million monthly unique visitors, making it the second most visited entertainment news website.

Chapter Three helped build the site. This project really helped Drupal 8 move forward because they ported many modules to Drupal 8 in the process. Check it out at http://www.etonline.com!

Categories: Drupal

Entertainment Tonight using Drupal

30 August 2017 - 6:11pm

Entertainment Tonight is the number one entertainment news magazine in the world, and has been on the air for over 30 years. Fans around the world rely on Entertainment Tonight to receive news and updates on their favorite celebrities and stars. I recently discovered that the newest star featured on Entertainment Tonight was Drupal 8!

Entertainment Tonight's new Drupal 8 website, ETOnline.com, receives 19 million monthly unique visitors, making it the second most visited entertainment news website. Check it out at http://www.etonline.com!

Categories: Drupal

Reservoir, a simple way to decouple Drupal

22 August 2017 - 1:52pm

Headless Drupal seems to be taking the world by storm. I'm currently in Sydney, and everyone I talked to so far, including the attendees at the Sydney Drupal User Group, is looking into headless Drupal. Digital agencies are experimenting with it on more projects, and there is even a new Decoupled Dev Days conference dedicated to the topic.

Roughly eight months ago, we asked ourselves in Acquia's Office of the CTO whether we could create a "headless" version of Drupal, optimized for integration with a variety of applications, channels and touchpoints. Such a version could help us build bridges with other developer communities working with different frameworks and programming languages, and the JavaScript community in particular.

I've been too busy with the transition at Acquia to blog about it in real time, but a few months ago, we released Reservoir. It's a Drupal-based content repository with all the necessary web service APIs needed to build decoupled front-end applications, be it a React application, an Ember front end, a native application, an augmented reality application, a Java or .NET application, or something completely different. You can even front-end it with a PHP application, something I hope to experiment with on my blog.

API-first distributions for Drupal like Reservoir and Contenta are a relatively new phenomenon but seem to be taking off rapidly. It's no surprise because an API-first approach is critical in a world where you have to operate agnostically across any channel and any form factor. I'm convinced that an API-first approach will be a critical addition to Drupal's future and could see a distribution like Reservoir or Contenta evolve to become a third installation profile for Drupal core (not formally decided).

Headless Drupal for both editors and developers The welcome screen after installing Reservoir.

The reason headless Drupal is taking off is that organizations are now grappling with a multitude of channels, including mobile applications, single-page JavaScript applications, IoT applications, digital signage, and content driven by augmented and virtual reality. Increasingly, organizations need a single place to house content.

What you want is an easy but powerful way for your editorial team to create and manage content, including administering advanced content models, content versioning, integrating media assets, translations, and more. All of that should be made easy through a great UI without having to involve a developer. This, incidentally, is aligned with Drupal 8's roadmap, in which we are focused on media management, workflows, layouts, and usability improvements through our outside-in work.

At the same time, you want to enable your developers to easily deliver that content to different devices, channels, and platforms. This means that the content needs to be available through APIs. This, too, is aligned with Drupal 8's roadmap, where we are focused on web services capabilities. Through Drupal's web service APIs, developers can build freely in different front-end technologies, such as Angular, React, Ember, and Swift, as well as Java and .NET. For developers, accomplishing this without the maintenance burden of a full Drupal site or the complexity of configuring standard Drupal to be decoupled is key.

API-first distributions like Reservoir keep Drupal's workflows and editorial UI intact but emphasize Drupal's web service APIs to return control to your developers. But with flexible content modeling and custom fields added to the equation, they also give more control over how editors can curate, combine, and remix content for different channels.

Success is getting to developer productivity faster Reservoir includes side-by-side previews of content in HTML and JSON API output.

The goal of a content repository should be to make it simple for developers to consume your content, including digital assets and translations, through a set of web service APIs. Success means that a developer can programmatically access your content within minutes.

Reservoir tries to achieve this in four ways:

  1. Easy on-boarding. Reservoir provides a welcome tour with helpful guidance to create and edit content, map out new content models, manage access control, and most importantly, introspect the web service APIs you'll need to consume to serve your applications.
  2. JSON API standard. Reservoir makes use of JSON API, which is the specification used for many APIs in JSON and adopted by the Ember and Ruby on Rails communities. Using a common standard means you can on-board your developers faster.
  3. Great API documentation. Reservoir ships with great API documentation thanks to OpenAPI, formerly known as Swagger, which is a specification for describing an API. If you're not happy with the default documentation, you can bring your own approach by using Reservoir's OpenAPI export.
  4. Libraries, references, and SDKs. With the Waterwheel ecosystem, a series of libraries, references, and SDKs for popular languages like JavaScript and Swift, developers can skip learning the APIs and go straight to integrating Drupal content in their applications.
Next steps for Reservoir API documentation auto-generated based on the content model built in Reservoir.

We have a lot of great plans for Reservoir moving forward. Reservoir has several items on its short-term roadmap, including GraphQL support. As an emerging industry standard for data queries, GraphQL is a query language I first highlighted in my 2015 Barcelona keynote; see my blog post on the future of decoupled Drupal for a quick demo video.

We also plan to expand API coverage by adding the ability to programmatically manipulate users, tags, and other crucial content elements. This means that developers will be able to build richer integrations.

While content such as articles, pages, and other custom content types can be consumed and manipulated via web services today, upstream in Drupal core, API support for things like Drupal's blocks, menus, and layouts is in the works. The ability to influence more of Drupal's internals from external applications will open the door to better custom editorial interfaces.

Conclusion

I'm excited about Reservoir, not just because of the promise API-first distributions hold for the Drupal community, but because it helps us reach developers of different stripes who just need a simple content back end, all the while keeping all of the content editing functionality that editorial teams take for granted.

We've put the Reservoir codebase on GitHub, where you can open an issue, create a pull request, or contribute to documentation. Reservoir only advances when you give us feedback, so please let us know what you think!

Special thanks to Preston So for contributions to this blog post and to Ted Bowman, Wim Leers, and Matt Grill for feedback during the writing process.

Categories: Drupal

Reservoir, a simple way to decouple Drupal

22 August 2017 - 1:52pm

Decoupled Drupal seems to be taking the world by storm. I'm currently in Sydney, and everyone I talked to so far, including the attendees at the Sydney Drupal User Group, is looking into decoupled Drupal. Digital agencies are experimenting with it on more projects, and there is even a new Decoupled Dev Days conference dedicated to the topic.

Roughly eight months ago, we asked ourselves in Acquia's Office of the CTO whether we could create a "headless" version of Drupal, optimized for integration with a variety of applications, channels and touchpoints. Such a version could help us build bridges with other developer communities working with different frameworks and programming languages, and the JavaScript community in particular.

I've been too busy with the transition at Acquia to blog about it in real time, but a few months ago, we released Reservoir. It's a Drupal-based content repository with all the necessary web service APIs needed to build decoupled front-end applications, be it a React application, an Ember front end, a native application, an augmented reality application, a Java or .NET application, or something completely different. You can even front-end it with a PHP application, something I hope to experiment with on my blog.

API-first distributions for Drupal like Reservoir and Contenta are a relatively new phenomenon but seem to be taking off rapidly. It's no surprise because an API-first approach is critical in a world where you have to operate agnostically across any channel and any form factor. I'm convinced that an API-first approach will be a critical addition to Drupal's future and could see a distribution like Reservoir or Contenta evolve to become a third installation profile for Drupal core (not formally decided).

Decoupled Drupal for both editors and developers The welcome screen after installing Reservoir.

The reason decoupled Drupal is taking off is that organizations are now grappling with a multitude of channels, including mobile applications, single-page JavaScript applications, IoT applications, digital signage, and content driven by augmented and virtual reality. Increasingly, organizations need a single place to house content.

What you want is an easy but powerful way for your editorial team to create and manage content, including administering advanced content models, content versioning, integrating media assets, translations, and more. All of that should be made easy through a great UI without having to involve a developer. This, incidentally, is aligned with Drupal 8's roadmap, in which we are focused on media management, workflows, layouts, and usability improvements through our outside-in work.

At the same time, you want to enable your developers to easily deliver that content to different devices, channels, and platforms. This means that the content needs to be available through APIs. This, too, is aligned with Drupal 8's roadmap, where we are focused on web services capabilities. Through Drupal's web service APIs, developers can build freely in different front-end technologies, such as Angular, React, Ember, and Swift, as well as Java and .NET. For developers, accomplishing this without the maintenance burden of a full Drupal site or the complexity of configuring standard Drupal to be decoupled is key.

API-first distributions like Reservoir keep Drupal's workflows and editorial UI intact but emphasize Drupal's web service APIs to return control to your developers. But with flexible content modeling and custom fields added to the equation, they also give more control over how editors can curate, combine, and remix content for different channels.

Success is getting to developer productivity faster Reservoir includes side-by-side previews of content in HTML and JSON API output.

The goal of a content repository should be to make it simple for developers to consume your content, including digital assets and translations, through a set of web service APIs. Success means that a developer can programmatically access your content within minutes.

Reservoir tries to achieve this in four ways:

  1. Easy on-boarding. Reservoir provides a welcome tour with helpful guidance to create and edit content, map out new content models, manage access control, and most importantly, introspect the web service APIs you'll need to consume to serve your applications.
  2. JSON API standard. Reservoir makes use of JSON API, which is the specification used for many APIs in JSON and adopted by the Ember and Ruby on Rails communities. Using a common standard means you can on-board your developers faster.
  3. Great API documentation. Reservoir ships with great API documentation thanks to OpenAPI, formerly known as Swagger, which is a specification for describing an API. If you're not happy with the default documentation, you can bring your own approach by using Reservoir's OpenAPI export.
  4. Libraries, references, and SDKs. With the Waterwheel ecosystem, a series of libraries, references, and SDKs for popular languages like JavaScript and Swift, developers can skip learning the APIs and go straight to integrating Drupal content in their applications.
Next steps for Reservoir API documentation auto-generated based on the content model built in Reservoir.

We have a lot of great plans for Reservoir moving forward. Reservoir has several items on its short-term roadmap, including GraphQL support. As an emerging industry standard for data queries, GraphQL is a query language I first highlighted in my 2015 Barcelona keynote; see my blog post on the future of decoupled Drupal for a quick demo video.

We also plan to expand API coverage by adding the ability to programmatically manipulate users, tags, and other crucial content elements. This means that developers will be able to build richer integrations.

While content such as articles, pages, and other custom content types can be consumed and manipulated via web services today, upstream in Drupal core, API support for things like Drupal's blocks, menus, and layouts is in the works. The ability to influence more of Drupal's internals from external applications will open the door to better custom editorial interfaces.

Conclusion

I'm excited about Reservoir, not just because of the promise API-first distributions hold for the Drupal community, but because it helps us reach developers of different stripes who just need a simple content back end, all the while keeping all of the content editing functionality that editorial teams take for granted.

We've put the Reservoir codebase on GitHub, where you can open an issue, create a pull request, or contribute to documentation. Reservoir only advances when you give us feedback, so please let us know what you think!

Special thanks to Preston So for contributions to this blog post and to Ted Bowman, Wim Leers, and Matt Grill for feedback during the writing process.

Categories: Drupal

Reservoir, a simple way to decouple Drupal

22 August 2017 - 1:52pm

Decoupled Drupal seems to be taking the world by storm. I'm currently in Sydney, and everyone I talked to so far, including the attendees at the Sydney Drupal User Group, is looking into decoupled Drupal. Digital agencies are experimenting with it on more projects, and there is even a new Decoupled Dev Days conference dedicated to the topic.

Roughly eight months ago, we asked ourselves in Acquia's Office of the CTO whether we could create a "headless" version of Drupal, optimized for integration with a variety of applications, channels and touchpoints. Such a version could help us build bridges with other developer communities working with different frameworks and programming languages, and the JavaScript community in particular.

I've been too busy with the transition at Acquia to blog about it in real time, but a few months ago, we released Reservoir. It's a Drupal-based content repository with all the necessary web service APIs needed to build decoupled front-end applications, be it a React application, an Ember front end, a native application, an augmented reality application, a Java or .NET application, or something completely different. You can even front-end it with a PHP application, something I hope to experiment with on my blog.

API-first distributions for Drupal like Reservoir and Contenta are a relatively new phenomenon but seem to be taking off rapidly. It's no surprise because an API-first approach is critical in a world where you have to operate agnostically across any channel and any form factor. I'm convinced that an API-first approach will be a critical addition to Drupal's future and could see a distribution like Reservoir or Contenta evolve to become a third installation profile for Drupal core (not formally decided).

Decoupled Drupal for both editors and developers The welcome screen after installing Reservoir.

The reason decoupled Drupal is taking off is that organizations are now grappling with a multitude of channels, including mobile applications, single-page JavaScript applications, IoT applications, digital signage, and content driven by augmented and virtual reality. Increasingly, organizations need a single place to house content.

What you want is an easy but powerful way for your editorial team to create and manage content, including administering advanced content models, content versioning, integrating media assets, translations, and more. All of that should be made easy through a great UI without having to involve a developer. This, incidentally, is aligned with Drupal 8's roadmap, in which we are focused on media management, workflows, layouts, and usability improvements through our outside-in work.

At the same time, you want to enable your developers to easily deliver that content to different devices, channels, and platforms. This means that the content needs to be available through APIs. This, too, is aligned with Drupal 8's roadmap, where we are focused on web services capabilities. Through Drupal's web service APIs, developers can build freely in different front-end technologies, such as Angular, React, Ember, and Swift, as well as Java and .NET. For developers, accomplishing this without the maintenance burden of a full Drupal site or the complexity of configuring standard Drupal to be decoupled is key.

API-first distributions like Reservoir keep Drupal's workflows and editorial UI intact but emphasize Drupal's web service APIs to return control to your developers. But with flexible content modeling and custom fields added to the equation, they also give more control over how editors can curate, combine, and remix content for different channels.

Success is getting to developer productivity faster Reservoir includes side-by-side previews of content in HTML and JSON API output.

The goal of a content repository should be to make it simple for developers to consume your content, including digital assets and translations, through a set of web service APIs. Success means that a developer can programmatically access your content within minutes.

Reservoir tries to achieve this in four ways:

  1. Easy on-boarding. Reservoir provides a welcome tour with helpful guidance to create and edit content, map out new content models, manage access control, and most importantly, introspect the web service APIs you'll need to consume to serve your applications.
  2. JSON API standard. Reservoir makes use of JSON API, which is the specification used for many APIs in JSON and adopted by the Ember and Ruby on Rails communities. Using a common standard means you can on-board your developers faster.
  3. Great API documentation. Reservoir ships with great API documentation thanks to OpenAPI, formerly known as Swagger, which is a specification for describing an API. If you're not happy with the default documentation, you can bring your own approach by using Reservoir's OpenAPI export.
  4. Libraries, references, and SDKs. With the Waterwheel ecosystem, a series of libraries, references, and SDKs for popular languages like JavaScript and Swift, developers can skip learning the APIs and go straight to integrating Drupal content in their applications.
Next steps for Reservoir API documentation auto-generated based on the content model built in Reservoir.

We have a lot of great plans for Reservoir moving forward. Reservoir has several items on its short-term roadmap, including GraphQL support. As an emerging industry standard for data queries, GraphQL is a query language I first highlighted in my 2015 Barcelona keynote; see my blog post on the future of decoupled Drupal for a quick demo video.

We also plan to expand API coverage by adding the ability to programmatically manipulate users, tags, and other crucial content elements. This means that developers will be able to build richer integrations.

While content such as articles, pages, and other custom content types can be consumed and manipulated via web services today, upstream in Drupal core, API support for things like Drupal's blocks, menus, and layouts is in the works. The ability to influence more of Drupal's internals from external applications will open the door to better custom editorial interfaces.

Conclusion

I'm excited about Reservoir, not just because of the promise API-first distributions hold for the Drupal community, but because it helps us reach developers of different stripes who just need a simple content back end, all the while keeping all of the content editing functionality that editorial teams take for granted.

We've put the Reservoir codebase on GitHub, where you can open an issue, create a pull request, or contribute to documentation. Reservoir only advances when you give us feedback, so please let us know what you think!

Special thanks to Preston So for contributions to this blog post and to Ted Bowman, Wim Leers, and Matt Grill for feedback during the writing process.

Categories: Drupal

From buytaert.net to dri.es

14 August 2017 - 5:12pm
I recently was able to obtain the domain name dri.es so I decided to make the switch from buytaert.net to dri.es. I made the switch because my first name is a lot easier to remember and pronounce than my last name. It's bittersweet because I've been blogging on buytaert.net for almost 12 years now. The plan is to stick with dri.es for the rest of the blog's life so it's worth the change. Old links to buytaert.net will automatically be redirected, but if you can, please update your RSS feeds and other links you might have to my website.
Categories: Drupal

From buytaert.net to dri.es

14 August 2017 - 5:12pm
I recently was able to obtain the domain name dri.es so I decided to make the switch from buytaert.net to dri.es. I made the switch because my first name is a lot easier to remember and pronounce than my last name. It's bittersweet because I've been blogging on buytaert.net for almost 12 years now. The plan is to stick with dri.es for the rest of the blog's life so it's worth the change. Old links to buytaert.net will automatically be redirected, but if you can, please update your RSS feeds and other links you might have to my website.
Categories: Drupal

From buytaert.net to dri.es

14 August 2017 - 5:12pm
I recently was able to obtain the domain name dri.es so I decided to make the switch from buytaert.net to dri.es. I made the switch because my first name is a lot easier to remember and pronounce than my last name. It's bittersweet because I've been blogging on buytaert.net for almost 12 years now. The plan is to stick with dri.es for the rest of the blog's life so it's worth the change. Old links to buytaert.net will automatically be redirected, but if you can, please update your RSS feeds and other links you might have to my website.
Categories: Drupal

Niagara Falls by night

31 July 2017 - 7:12am

I had a chance to visit the Niagara Falls last week on a family trip. If you like nature, it's a must see — both during the day and at night when they lit up the falls. I love this photo but it still doesn't capture the majesty and beauty of the Niagara Falls.

Categories: Drupal

Niagara Falls by night

31 July 2017 - 7:12am

I had a chance to visit the Niagara Falls last week on a family trip. If you like nature, it's a must see — both during the day and at night when they lit up the falls. I love this photo but it still doesn't capture the majesty and beauty of the Niagara Falls.

Categories: Drupal

Pages