Planet Drupal

Subscribe to Planet Drupal feed
Drupal.org - aggregated feeds in category Planet Drupal
Updated: 2 hours 18 min ago

OpenSense Labs: Integrating Digital Asset Management with Drupal

2 July 2018 - 5:51am
Integrating Digital Asset Management with Drupal Shankar Mon, 07/02/2018 - 18:21

Governance plays a pivotal role in the business workflow and is significant for the smooth functioning of an organisation which is expanding at a steadfast pace. In this digitised world of organised chaos, demand for agile marketing has resulted in the accretion of digital assets to such an extent that some of them seem superfluous. Digital Asset Management (DAM) and Drupal can come together to smartly strategise the way enterprises handle unimportant assets.

Drupal governs the content on the website whereas Digital Asset Management manages creation, assessment and approval of digital assets.

A superabundance of assets clog up our devices and blot out the productivity. Digital Asset Management can help enterprises in eclipsing outmoded asset management processes and take back control of their digital files. Therefore, it helps in optimising content processes and makes life much easier.

Decoding the Digital Asset Management

What is involved in digital asset management? DAM is a content management and branding automation system which uses the centralised mode of storing and governing voluminous digital files. It accredits enterprises to connect, control, and centralise both the local and the global access to digital files thereby ensuring that digital assets are accessible to everyone whenever they have the dire need of them.

Source: Cognizant

Grid, The Guardian’s image management system, is a very good example of an open source digital asset management system. Its team used pragmatic agile processes to build it quickly. It is now incorporated with its print workflow and is used for almost half of the images used in their digital content.


Does it ring a bell? Definitely! Content Management System (CMS) like Drupal also has its foundation in the provision of a centralised mode of content dissemination. So what’s the difference between CMS and DAM?

Difference between DAM and CMS

Content Management System and Digital Asset Management are two systems that work brilliantly with the digital content but are designed for different tasks.

Digital Asset Management Content Management System

Central repository for handling digital assets

The central system for governing content on the website

Example: Acquia DAM

Example: Drupal

In general, Content Management System, for example, Drupal, is the entire foundation for governing content on the website. The term ‘content’ means anything that is displayed on your site and not just blog posts which are managed by CMS.

In contrast, a Digital Asset Management manages more than just your website. It powers the entire sales and marketing organisation. It acts as the central repository for the approved assets. Moreover, it is the collaborative workflow engine between marketers and designers for the creation, assessment, and approval of those assets.

A Plethora of Benefits

With a superabundance of benefits, Digital Asset Management enhances productivity, improves brand consistency and boosts team collaboration.

Source: Webdam
  • Cuts business costs: It can improve your ROI through quicker creation, retrieval, and dissemination of company content. Do-it-yourself design templates for business cards, banner ads etc. cut both internal and external admin, design, and production costs.
     
  • Security: It provides collective security measures built around your data’s needs and adheres to strict digital rights management (DRM) guidelines. For instance, be it navigating outmoded sharing methods, or supporting teams to establish effective user access permissions, it maps the usage rights vis-à-vis your digital assets. Internal management of brand’s corporate identity: By paving the way for corporate identity guidelines and numerous on-brand digital assets, it offers secure global access to approve useable content.
     
  • Integration scope: It seamlessly integrates with your existing infrastructure. It helps you in improving project management, delegating resources and roles, optimising and targeting assets, and identifying top-performing content by working with CMS, customer relationship management (CRM) and several other marketing, sales and IT solutions.
     
  • Efficient file management: It eliminates the need to find and deliver assets via multiple locations thereby removing any costly errors and inefficiencies incurred through clumsy file management.
     
  • Time-Saving: It provides 24X7 access to company collateral from anywhere. Thus, it removes the need for time-intensive file searches and avoids the need for recreating missing assets.
Choosing the Best Digital Asset Management for your Business

So, how to strategise in order to choose the best Digital Asset Management? It is important that you understand your prerequisites before making the decision of selecting a Digital Asset Management. 

Understanding DAM capabilities

 

  • Repository: One of the first things that you need to consider is a core digital asset repository. A centralised asset repository should on the top of the priority list for efficacy in controlling and governing the assets.
     
  • Creation: Through workflow and collaboration tools,  can help with the creation of assets. If you have multiple departments or have agencies working together to create digital assets, it can make the process of creation of assets faster and makes its easier to assess and approve them. It also ensures that everyone uses the same transparent process. Workflow management tools pre-defines stages and roles so that you can know the status of every asset and their requirements instantly.
     
  • Consumption: It depends on people and systems to determine how these assets are being used. Your Digital Asset Management should be able to provide a way for employees to search for and download assets on their own.

Understanding your Goals

 

  • It should simplify the production of digital assets.
  • It should improve the quality, consistency, and on-brand of assets being created especially in the enterprises with global digital marketing teams.
     
  • It should be able to allow the organisation to keep a record of all the assets without leaving a question mark on where to add them or their approval status.
     
  • It should alleviate manual requests that are being made by the marketing team to provide specific files to several people and groups like logos, product images, campaign assets, sales collateral etc.
     
  • It must master your brand consistency.
     
  • It should decrease the time taken to integrate assets into digital marketing campaigns and user experiences.
Understanding your Users
  • Develop a list of stakeholders, people in your organisation, and even outside of your organisation who will be interacting with Digital Asset Management.
     
  • Stakeholders include digital marketers, designers, sales, partners, IT etc.
     
  • Understand what their access and roles are with the Digital Asset Management.
Understanding your Requirements
  • Get the feedback from each of your stakeholder groups. Know what are their most common challenges and build a list of requirements. Make sure to understand the use cases so that you can prioritise those requirements.
     
  • Digital Asset Management systems have the capability to fulfill the need of basic requirements of small teams to extremely intricate ones for sophisticated large teams. 

Keep a note of typical requirements of organisations utilising Digital Asset Management as mentioned in the tabular column below.

Requirement Description

Asset Approval

Processes required to create a new asset should be well managed

Asset Review

A collaborative asset review cycle for the people to add comments, markups, and feedback to asset proofs

Asset Access (Internal)

A portal for employees to search and download assets. Access control by region or department would be required

Asset Access (External)

A controlled asset access for the external users paving the way for secure sharing of assets with partners, contractors, agencies etc.

Branding guidelines

A self-serve access point to get the approved branding guidelines like product images, logos, color palettes etc.

Structured assets

Assets should be placed in the folders

Keywords/Tags

Assets should be assigned with keywords and tags

Customer fields

Custom metadata fields should be assigned to assets

Reports

There must be reporting and assessment on asset usage

Direct access for marketing system

There must be provision for utilising approved assets for creating web banners, blog posts, email marketing campaigns etc. directly from the Digital Asset Management

Understanding your Deployment Options

Deployment options for Digital Asset Management constitute on-premise, private cloud, and Software-as-as-Service (SaaS). To understand what works best for your business needs, let’s look at the merits and demerits of these deployment options.

On-Premise Private Cloud SaaS Merit
  • Flexibility in integration
  • Low costs
  • Data in cloud
  • Local performance
  • Data in cloud
  • Comparatively fast to implement
  • Great for heavy customisations
  • Accessibility
  • Low Cost
  • On-premise control of data
  • Lessens dependence on IT
  • Low risks involved
 
  • Moderate customisations possible
  • Great multi-location performance
 
  • Security/control
  • Resilience/failover
 
  • Somewhat scalable
  • Scalable
 
  • Very basic resilience/failover
  • Good integration scope
 
  • Quick implementation
  • Automatic upgrades
   
  • Least requirement of IT dependency
Demerit
  • Huge risks involved
  • Dependency on web
  • Dependency on web
  • Not cost effective
  • Usually, web hosting requires third parties
  • Very limited customisation
  • Not scalable
  • Responsible for maintenance and upgrades
  • Data off-premise
  • Very limited accessibility
  • Data off-premise
 
  • Responsible for upgrades and maintenance
   
  • Dependence on IT
   
  • Time-intensive during deployment
    Drupal modules for DAM Integration

Acquia DAM offers a centralised cloud repository to keep all creative assets systematised, tagged and searchable throughout the lifecycle of those assets. Its easy-to-use workflow and collaboration capability lets digital marketers and designers search, review, approve and publish assets faster


It allows seamless integration with Drupal 7 and Drupal 8 for delivering a wonderful digital experience by incorporating captivating rich-media assets for centralised management.

Media: Acquia DAM, Drupal module, helps in the integration of Drupal with Acquia DAM. The integration helps Drupal content editors to browse the Acquia DAM repository using their own credentials via an entity browser plugin. They can search by keyword in order to find the right assets for the usage. WYSIWYG editor allows content authors to add assets easily. The assets marked for the usage are copied to Drupal and are stored as Media Entity references thereby rendering image styles.

Through a cron job, these assets are kept in sync between Drupal and Acquia DAM. Updates made to the assets are automatically reflected in Drupal.

This module provides a media entity provider for importing asset metadata into fields on your entities. Users can view the metadata directly in the entity browser without having to import the assets. It also provides a usage report of an asset within Drupal.

Another very useful digital asset management solution is Bynder which is a cloud-based platform for digital marketing teams to create, find, and use digital content.

Bynder integration Drupal module aids in the integration of Drupal with Bynder thereby providing seamless access to asset bank on your website. Users can import assets from and upload assets to Bynder. Image styles imported from Bynder can be displayed to match the derivatives created.


A digital agency integrated Bynder with Drupal using modules like Entity Browser and Media Entity.

Media entity module made it possible to represent Bynder assets in Drupal without actually copying them over. Its powerful metadata API helped in exposing and using all the information about them in the context of a Drupal website. It offered a central management of assets and their full availability on sites which consume them.

Entity browser module helped in implementing a user-friendly interface for letting the users browse their Bynder assets from the editor’s node of their Drupal site.

DropzoneJS driven widget helped in uploading the right assets into the Bynder DAM directly from the Drupal site.

Case study

Vallabhbhai Jhaverbhai Patel aka Sardar Patel was India’s first Deputy Prime Minister and played a pivotal role in India’s freedom movement. The Sardar Patel website, built on Drupal 8 with the help of an agency, showcases information about him gathered from numerous libraries.

The digital assets of more than 2000 content records related to the man events in the life of Sardar Patel can be viewed in the Sardar Patel website.


Objectives

The main goal behind building a dedicated site was to access books, texts, videos, audios, and similar content about him in multiple languages. By gathering and standardising the available digital assets, everything about him would be present in an easily searchable form. This would, thus, help in preserving the history of India’s influential freedom struggle through digital content for the generations to come.

Drupal 8 to the rescue

The scalability of Drupal 8 helped in migrating 2000+ content from several different sources under a single umbrella.
It also lent a mobile-first user experience.
Drupal’s out-of-the-box capabilities for building a multilingual site helped in translating the content into multiple languages.
Moreover, being the most security focussed CMS, it turned out to be the best choice for the website.

Result

1 TB of data comprising of 1000 images, 800 textual content, numerous audio and video recordings were added as content in addition to keeping the site navigation simple and easy. Most importantly, importing of content in bulk was possible with Drupal 8 thereby streamlining the content migration process and saving a lot of time.

For easy access, the website content was sorted and arranged by the main events or themes in addition to the chronological order of events that transpired in his life.


The site has easy search capability with filter options enabled to zero in on a particular subject.

What the Future holds for DAM?

Mobile applications would be at the epicentre of Digital Asset Management in the coming years. Native applications offer a seamless user experience both online and offline. Connected directly to your asset bank, they provide a mobile marketing portal and interactive features that help in creating, handling, and using the digital content on the fly. No matter what is the location of the user and whether or not they are near their laptop or desktop, they can always work with assets.

Number of mobile phone users worldwide from 2015 to 2020 (in billions) from Statista

Artificial Intelligence technology is becoming more mainstream, affordable and accessible. Digital Asset Management can leverage the benefits of AI for automating processes and centralising tasks. Also, features like facial, location, and image recognition variables; geospatial technology, and deterministic reasoning capabilities will be the key to utilising AI.

Machine learning, as shown in this graph, would require 2 to 5 years for mainstream adoption from Gartner

Machine learning algorithms can be of great use for Digital Asset Management in the future. It can recommend assets for a particular channel or task within a campaign based on analytical data. This would help you in promptly selecting the best asset swiftly without wasting the time thinking over which video, image or banner will best serve your digital marketing pursuits.

Conclusion

With the rapid expansion of businesses, digital assets like images, videos and company documents are piling up to an unimaginable extent. It is imperative that a proper governance is incorporated to handle these increasing digitised assets. Drupal 8 can be a wonderful option to integrate Digital Asset Management to manage your organisation’s increasing online presence and its ever-growing digital assets.

Drupal development is our key service where we are excelling since our birth as a company. Ping us at hello@opensenselabs.com to instill the best asset management solution for your enterprise using Drupal 8.

blog banner blog image Digital Asset Management DAM Drupal Drupal 8 Digital Assets Acquia DAM Bynder Content Management System CMS Machine Learning Artificial Intelligence Mobile marketing User experience Sardar Patel website Agile development methodology Blog Type Articles Is it a good read ? On
Categories: Drupal

Drupal Association blog: It's Time To Vote - Community Elections 2018

2 July 2018 - 2:00am

Voting is now open for the 2018 At-Large Board positions for the Drupal Association!  If you haven't yet, check out the candidate profiles including their short videos found on the profile pages. Get to know your candidates, and then get ready to vote.

Cast Your Vote!

How does voting work? Voting is open to all individuals who have a Drupal.org account by the time nominations open and who have logged in at least once in the past year.

To vote, you will rank candidates in order of your preference (1st, 2nd, 3rd, etc.). The results will be calculated using an "instant runoff" method. For an accessible explanation of how instant runoff vote tabulation works, see videos linked in this discussion.

Election voting will be held from 2 July, 2018 through 13 July, 2018. During this period, you can continue to review and comment on the candidate profiles.

Have questions? Please contact me: Rachel Lawson.

Categories: Drupal

Gizra.com: Using JSON API with WebdriverIO Tests

1 July 2018 - 9:00pm

In Drupal, you can write automated tests with different levels of complexity. If you need to test a single function, or method of a class, probably you will be fine with a unit test. When you need to interact with the database, you can create kernel tests. And finally, if you need access to the final HTML rendered by the browser, or play with some javascript, you can use functional tests or Javascript tests. You can read more about this in the Drupal.org documentation.

So far this is what Drupal provides out of the box. On top of that, you can use Behat or WebDriver tests. This types of tests are usually easier to write and are closer to the user needs. As a side point, they are usually slower than the previous methods.

The Problem.

In Gizra, we use WebdriverIO for most of our tests. This allow us to tests useful things that add value to our clients. But these sort of tests, where you only interact with the browser output, has some disadvantages.

Imagine you want to create an article and check that this node is unpublished by default. How do you check this? Remember you only have the browser output…

One possible way could be this: Login, visit the Article creation form, fill the fields, click submit, and then… Maybe search for some unpublished class in the html:

var assert = require('assert'); describe('create article', function() { it('should be possible to create articles, unpublished by default', function() { browser.loginAs('some user'); browser.url('http://example.com/node/add/article') browser.setValueSafe('#edit-title-0-value', 'My new article'); browser.setWysiwygValue('edit-body-0-value', 'My new article body text'); browser.click('#edit-submit'); browser.waitForVisible('.node-unpublished'); }); });

This is quite simple to understand, but it has some drawbacks.

For one, it depends on the theme to get the status of the node. You could take another approach and instead of looking for a .node-unpublished class, you could logout from the current session and then try to visit the url to look for an access denied legend.

Getting Low-Level Information from a Browser Test

So the problem boils down to this:

How can I get information about internal properties from a browser test?

The new age of decoupled Drupal brings an answer to this question. It could be a bit counterintuitive at first, therefore just try to see is fit for your project.

The idea is to use the new modules that expose Drupal internals, through json endpoints, and use javascript together with a high-level testing framework to get the info you need.

In Gizra we use WDIO tests write end-to-end tests. We have some articles about this topic. We also wrote about a new module called JsonAPI that exposes all the information you need to enrich your tests.

The previous test could be rewritten into a different test. By making use of the JsonAPI module, you can get the status of a specific node by parsing a JSON document:

var assert = require('assert'); describe('create article', function() { it('should be possible to create articles, unpublished by default', function() { browser.loginAs('some user'); browser.url('http://example.com/node/add/article') browser.setValueSafe('#edit-title-0-value', 'My unique title'); browser.setWysiwygValue('edit-body-0-value', 'My new article body text'); browser.click('#edit-submit'); // Use JSON api to get the internal data of a node. let query = '/jsonapi/node/article' += '?fields[node--article]=status' += '&filter[status]=0' += '&filter[node-title][condition][path]=title' += '&filter[node-title][condition][value]=My unique title' += '&filter[node-title][condition][operator]=CONTAINS' browser.url(query); browser.waitForVisible('body pre'); let json = JSON.parse(browser.getHTML('body pre', false)); assert.ok(json[0].id); assert.equals(false, json[0].attributes.content['status']); }); });

In case you skipped the code, don’t worry, it’s quite simple to understand, let’s analyze it:

1. Create the node as usual:

This is the same as before:

browser.url('http://example.com/node/add/article') browser.setValueSafe('#edit-title-0-value', 'My unique title'); browser.setWysiwygValue('edit-body-0-value', 'My new article body text'); browser.click('#edit-submit'); 2. Ask JsonAPI for the status of an article with a specific title:

Here you see the two parts of the request and the parsing of the data.

let query = '/jsonapi/node/article' += '?fields[node--article]=status' += '&filter[status]=0' += '&filter[node-title][condition][path]=title' += '&filter[node-title][condition][value]=My unique title' += '&filter[node-title][condition][operator]=CONTAINS' browser.url(query); 3. Make assertions based on the data:

Since JsonAPI exposes, well, json data, you can convert the json into a javascript object and then use the dot notation to access to a specific level.

This is how you can identify a section of a json document. browser.waitForVisible('body pre'); let json = JSON.parse(browser.getHTML('body pre', false)); assert.ok(json[0].id); assert.equals(false, json[0].attributes.content['status']); A Few Enhancements

As you can see, you can parse the output of a json request directly from the browser.

browser.url('/jsonapi/node/article'); browser.waitForVisible('body pre'); let json = JSON.parse(browser.getHTML('body pre', false));

The json object now contains the entire response from JsonAPI that you can use as part of your test.

There are some drawbacks of the previous approach. First, this only works for Chrome. That includes the Json response inside a XML document. This is the reason why you need to get the HTML from body pre.

The other problem is this somewhat cryptic section:

let query = '/jsonapi/node/article' += '?fields[node--article]=status' += '&filter[status]=0' += '&filter[node-title][condition][path]=title' += '&filter[node-title][condition][value]=My unique title' += '&filter[node-title][condition][operator]=CONTAINS'

The first problem can be fixed using a conditional to check which type of browser are you using to run the tests.

The second problem can be addressed using the d8-jsonapi-querystring package, that allows you to write an object that is automatically converted into a query string.

Other Use Cases

So far, we used JsonAPI to get information about a node. But there are other things that you can get from this API. Since all configurations are exposed, you could check if some role have some specific permission. To make tests shorter we skipped the describe and it sections.

browser.loginAs('some user'); let query = '/jsonapi/user_role/user_role' += '?filter[is_admin]=null' browser.url(query); browser.waitForVisible('body pre'); let json = JSON.parse(browser.getHTML('body pre', false)); json.forEach(function(role) { assert.ok(role.attributes.permissions.indexOf("bypass node access") == -1); });

Or if a field is available in some content type, but it is hidden to the end user:

browser.loginAs('some user'); let query = '/jsonapi/entity_form_display/entity_form_display?filter[bundle]=article' browser.url(query); browser.waitForVisible('body pre'); let json = JSON.parse(browser.getHTML('body pre', false)); assert.ok(json[0].attributes.hidden.field_country);

Or if some specific HTML tag is allowed in an input format:

let query = '/jsonapi/filter_format/filter_format?filter[format]=filtered_html' browser.url(query); browser.waitForVisible('body pre'); let json = JSON.parse(browser.getHTML('body pre', false)); let tag = '<drupal-entity data-*>'; assert.ok(json[0].attributes.filters.filter_html.settings.allowed_html.indexOf(tag) > -1);

As you can see, there are several use cases. The benefits of being able to explore the API by just clicking the different links sometimes make this much easier to write than a kernel test.

Just remember that this type of tests are a bit slower to run, since they require a full Drupal instance running. But if you have some continuous integration in place, it could be an interesting approach to try. At least for some specific tests.

We have found this quite useful, for example, to check that a node can be referenced by another in a reference field. To check this, you need the node ids of all the nodes created by the tests.

A tweet by @skyredwang could be accurate to close this post.

Remember how cool Views have been since Drupal 4.6? #JSONAPI module by @e0ipso is the new "Views".

— Jingsheng Wang (@skyredwang) January 9, 2018

Continue reading…

Categories: Drupal

Wim Leers: Shipping the right thing

30 June 2018 - 5:03pm

Two weeks ago, I stumbled upon a two-part blog post by Alex Russell, titled Effective Standards Work.

The first part (The Lay Of The Land) sets the stage. The second part (Threading the Needle) attempts to draw conclusions.

It’s worth reading if you’re interested in how Drupal is developed, or in how any consensus-driven open source project works (rather than the increasingly common “controlled by a single corporate entity” “open source”).

It’s written with empathy, modesty and honesty. It shows the struggle of somebody given the task and opportunity to help shape/improve the developer experience of many, but not necessarily the resources to make it happen. I’m grateful he posted it, because something like this is not easy to write nor publish — which he also says himself:

I’ve been drafting and re-drafting versions of this post for almost 4 years. In that time I’ve promised a dozen or more people that I had a post in process that talked about these issues, but for some of the reasons I cited at the beginning, it has never seemed a good time to hit “Publish”. To those folks, my apologies for the delay.

Parallels!

I hope you’ll find the incredibly many parallels with the open source Drupal ecosystem as fascinating as I did!

Below, I’ve picked out some of the most interesting statements and replaced only a few terms, and tadaaa! — it’s accurately describing observations in the Drupal world!

Go read those two blog posts first before reading my observations though! You’ll find some that I didn’t. Then come back here and see which ones I see, having been a Drupal contributor for >11 years and a paid full-time Drupal core contributor for >6.

Standards Theory

Design A new Drupal contrib module is the process of trying to address a problem with a new feature. Standardisation Moving a contributed module into Drupal core is the process of documenting consensus.

The process of feature design Drupal contrib module development is a messy, exciting exploration embarked upon from a place of trust and hope. It requires folks who have problems (web developers site builders) and the people who can solve them (browser engineers Drupal core/contrib developers) to have wide-ranging conversations.

The Forces at Play

Feature Drupal module design starts by exploring problems without knowing the answers, whereas participation in Working Groups Drupal core initiatives entails sifting a set of proposed solutions and integrating the best proposals competing Drupal modules. Late-stage iteration can happen there, but every change made without developer site builder feedback is dangerous — and Working Groups Drupal core initiatives aren’t set up to collect or prioritise it.

A sure way for a browser engineer Drupal core/contrib developer to attract kudos is to make existing content Drupal sites work better, thereby directly improving things for users site builders who choose your browser Drupal module.

Essential Ingredients
  • Participation by web developers site builders and browser engineers Drupal core/contrib developers: Nothing good happens without both groups at the table.
  • A venue outside a chartered Working Group Drupal core in which to design and iterate: Pre-determined outcomes rarely yield new insights and approaches. Long-term relationships of WG participants Drupal core developers can also be toxic to new ideas. Nobody takes their first tap-dancing lessons under Broadway’s big lights. Start small and nimble, build from there.
  • A path towards eventual standardisation stability & maintainability: Care must be taken to ensure that IP obligations API & data model stability can be met the future, even if the loose, early group isn’t concerned with a strict IP policy update path
  • Face-to-face deliberation: I’ve never witnessed early design work go well without in-person collaboration. At a minimum, it bootstraps the human relationships necessary to jointly explore alternatives.

    If you’ve never been to a functioning standards Drupal core meeting, it’s easy to imagine languid intellectual salons wherein brilliant ideas spring forth unbidden and perfect consensus is forged in a blinding flash. Nothing could be further from the real experience. Instead, the time available to cover updates and get into nuances of proposed changes can easily eat all of the scheduled time. And this is expensive time! Even when participants don’t have to travel to meet, high-profile groups Drupal core contributors are comically busy. Recall that the most in-demand members of the group Drupal core initiative (chairs Drupal core initiative coordinators, engineers from the most consequential firms Drupal agencies) are doing this as a part-time commitment. Standards work is time away from the day-job, so making the time and expense count matters.
Design → Iterate → Ship & Standardise

What I’ve learned over the past decade trying to evolving the web platform is a frustratingly short list given the amount of pain involved in extracting each insight:

  • Do early design work in small, invested groups
  • Design in the open, but away from the bright lights of the big stage
  • Iterate furiously early on because once it’s in the web Drupal core, it’s forever
  • Prioritize plausible interoperability; if an implementer says “that can’t work”, believe them!
  • Ship to a limited audience using experimental Drupal core modules as soon as possible to get feedback
  • Drive standards stabilization of experimental Drupal core modules with evidence and developer feedback from those iterations
  • Prioritise interop minimally viable APIs & evolvability over perfect specs APIs & data models; tests create compatibility stability as much or more than tight prose or perfect IDL APIs
  • Dot “i”s and cross “t”s; chartered Working Groups Drupal core initiatives and wide review many site builders trying experimental core modules are important ways to improve your design later in the game. These derive from our overriding goal: ship the right thing.

    So how can you shape the future of the platform as a web developer site builder?

The first thing to understand is that browser engineers Drupal core/contrib developers want to solve important problems, but they might not know which problems are worth their time. Making progress with implementers site builders is often a function of helping them understand the positive impact of solving a problem. They don’t feel it, so you may need to sell it!

Building this understanding is a social process. Available, objective evidence can be an important tool, but so are stories. Getting these in front of a sympathetic audience within a browser team of Drupal core committers or Drupal contrib module maintainers is perhaps harder.

It has gotten ever easier to stay engaged as designs experimental Drupal core modules iterate. After initial meetings, early designs are sketched up and frequently posted to GitHub Drupal.org issues where you can provide comments.

“Ship The Right Thing”

These relatively new opportunities for participation outside formal processes have been intentionally constructed to give developers and evidence a larger role in the design process.

There’s a meta-critique of formal standards processes in Drupal core and the defacto-exclusionary processes used to create them. This series didn’t deal in it deeply because doing so would require a long digression into the laws surrounding anti-trust and competition. Suffice to say, I have a deep personal interest in bringing more voices into developing the future of the web platform, and the changes to Chrome’s Drupal core’s approach to standards adding new modules discussed above have been made with an explicit eye towards broader diversity, inclusion, and a greater role for evidence.

I hope you enjoyed Alex’ blog posts as much as I did!

Categories: Drupal

Larry Garfield: PHP: Use associative arrays basically never

30 June 2018 - 1:59pm
PHP: Use associative arrays basically never

The other day I was working on some sample code to test out an idea that involved an object with an internal nested array. This is a pretty common pattern in PHP: You have some simple one-off internal data structure so you make an informal struct using PHP associative arrays. Maybe you document it in a docblock, or maybe you're a lazy jerk and you don't. (Fight me!) But really, who bothers with defining a class for something that simple?

But that got me wondering, is that common pattern really, you know, good? Are objects actually more expensive or harder to work with than arrays? Or, more to the point, is that true today on PHP 7 given all the optimizations that have happened over the years compared with the bad old days of PHP 4?

So like any good scientist I decided to test it: What I found will shock you!

Continue reading this post on Steemit

Larry 30 June 2018 - 4:59pm
Categories: Drupal

Drupal blog: Design 4 Drupal: The future of JavaScript in Drupal

29 June 2018 - 7:59am

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Today, I gave a keynote presentation at the 10th annual Design 4 Drupal conference at MIT. I talked about the past, present and future of JavaScript, and how this evolution reinforces Drupal's commitment to be API-first, not API-only. I also included behind-the-scene insights into the Drupal community's administration UI and JavaScript modernization initiative, and why this approach presents an exciting future for JavaScript in Drupal.

If you are interested in viewing my keynote, you can download a copy of my slides (256 MB).

Thank you to Design 4 Drupal for having me and happy 10th anniversary!

Categories: Drupal

Drupixels: Progressive Web App (PWA) integration with Drupal

29 June 2018 - 5:41am
A Progressive Web App (PWA) is a web app that uses modern web capabilities to deliver an app-like experience to users by combining features offered by most modern browsers with the benefits of mobile experience. Integration of PWA with Drupal makes Drupal inherit the latest web technologies and harness devices capabilities.
Categories: Drupal

Axelerant Blog: Axelerant At Drupal Developer Days Lisbon 2018

29 June 2018 - 3:16am


Drupal Developer Days brings together people who contribute to the progress of Drupal from around the world. There are code sprints, workshops, sessions, BoFs, after parties (and after-after parties) and more.

Categories: Drupal

Dries Buytaert: Design 4 Drupal: The future of JavaScript in Drupal

28 June 2018 - 4:44pm

Today, I gave a keynote presentation at the 10th annual Design 4 Drupal conference at MIT. I talked about the past, present and future of JavaScript, and how this evolution reinforces Drupal's commitment to be API-first, not API-only. I also included behind-the-scene insights into the Drupal community's administration UI and JavaScript modernization initiative, and why this approach presents an exciting future for JavaScript in Drupal.

If you are interested in viewing my keynote, you can download a copy of my slides (256 MB).

Thank you to Design 4 Drupal for having me and happy 10th anniversary!

Categories: Drupal

Angie "webchick" Byron: An update on Drupal 8.6 pre-feature freeze

28 June 2018 - 1:52pm

Greetings, folks! As we head into feature freeze for Drupal 8.6 (the week of July 18), here's a run-down of the various initiatives, and a hit-list of what they're trying to accomplish in the next two weeks. Patch reviews, testing, design, docs, and many more skills are very welcomed!

A couple of caveats here:

1) This is my own personal best understanding of where this stuff is all at, based on reading issue comments, attending meetings, overhearing things from other people who attended meetings, catching the odd Slack snippet of conversation, carrier piegon, etc. And therefore may not be 100% accurate, or even 80% accurate — there's a lot going on! (please clarify in the comments if you see any errors/omissions)
2) Just because something is listed here, there is absolutely no guarantee that it gets reviewed + (truly) RTBCed + committed in time for feature freeze and makes it into 8.6. As you can see, there are lots of issues in the list below, and we're all doing our best to stay on top of them. Worst-case, there's always 8.7. :)
3) This post gets into nitty-gritty "technical audience" details; if you're interested in a more broad overview of initiatives and their aims for 8.6 and beyond, there's the strategic initiatives overview on Drupal.org. I was also recently on a Lullbabot podcast to that effect.

OK, here we go! These are listed in alphabetical order.

Admin UI & JavaScript Modernization

This initiative has some lofty goals indeed, to redesign Drupal's admin experience, and modernize the underlying JavaScript code in Drupal to meet modern standards/best practices. While there's a ton of work actively going on in these areas right now, most of the fruit won't bear until 8.7 or later. If you're planning/able to go, come join the sprint next week at Drupal Developer Days Lisbon!

For 8.6, one of the big accomplishments of this initiative was introducing Nightwatch.js testing framework to core, which allows us to test JavaScript code with (wait for it)... JavaScript (what a concept!). This will be critical in ensuring that the React-ified components work as expected, and our existing JavaScript-rich functionality continues to work solidly as we expand on dynamic functionality in the UI.

Here are the issues this team has surfaced as important for 8.6:

Make Nightwatch testing more generally useful
  • Add login/logout commands to nightwatch [#2973879]
  • Create nightwatch command to install modules [#2974619]
Fix long-standing issues in the JavaScript system

Seriously, check out the five-digit node IDs on these bad boys! :P

  • ajax.js insert command sometimes wraps content in a div, potentially producing invalid HTML and other bugs [#736066]
  • Provide a common API for displaying JavaScript messages [#77245]
Bring JS code up to modern standards
  • Use Prettier for formatting core JavaScript [#2978964]
API-First

This team's 8.6 goals are two-fold: 1) stabilizing and filling gaps in the existing REST API, and 2) attempting to add JSON API to core.

TONS of work has been going on in the JSON API contributed module queue to fix a number of outstanding issues to make it core-worthy. So even if this module doesn't make it in time for 8.6, the entire ecosystem will benefit throughout 8.6's lifecycle by using a much more robust and well-tested contributed module. Additionally, a long-standing gap of file upload support has been added. Huzzah!

For the remainder of 8.6, the team would like to focus on the following:

Unblockers to API-First in general
  • Add DateTimeNormalizer+TimestampNormalizer, deprecate TimestampItemNormalizer: @DataType-level normalizers are reusable by JSON API [#2926508]
  • @DataType=map cannot be normalized, affects @FieldType=link, @FieldType=map [#2895532]
Unblockers to REST
  • EntityResource should add _entity_access requirement to REST routes [#2869426]
  • PATCHing entities validates the entire entity, also unmodified fields, so unmodified fields can throw validation errors [#2821077]
Unblockers to JSON API

These are all issues in the JSON API contrib module, which help unblock "Add experimental JSON API module [#2843147]" for core.

  • [PP-1] Work around core's ill-designed @FieldType-level TimestampItemNormalizer normalization until #2926508 lands [#2929932]
  • JSON API indicates it supports POST/PATCH/DELETE of config entity types, but that's impossible [#2887313]
  • Needs Issue: Module name conflict between contrib/core (what happens when we bring a same-named contrib module to core that sites are actively using?)
  • [>=8.5] Remove JSON API's "file URL" field work-around now that Drupal core 8.5 fixed it [#2926463] - Fixed!
Automatic Updates / Composer in Core

These two initiatives overlap in that we're aiming to build the automatic update functionality around improving core's underlying Composer support.

The Composer team has compiled an excellent plan of attack for how to provide Composer support without jeopardizing the site builder experience. Most of that work will take place in 8.7.

However, one of the pre-requisites for Composer to work well, is adding semantic versioning support for contrib. Support for this would also be tremendously helpful to contrib module authors and site builders, regardless if they use Composer to manage their dependencies or not.

Unblockers to semver for contrib
  • Core version key in module's .info.yml doesn't respect core semantic versioning [#2313917]
  • Module version dependency in .info.yml is ineffective for patch releases [#2641658]
Configuration Management 2.0

This team spent most of the 8.6 cycle forming, brainstorming a list of blockers to configuration awesomeness, and prioritizing those efforts. The hope is for a roadmap to get published after the sprint next week at Drupal Developer Days Lisbon.

One major win in 8.6 is the ability to Allow a site-specific profile to be installed from existing config, which is part of the aim to Allow a site to be installed from existing configuration (basically, moving the capabilities of the Config Installer module into core.)

Unblockers of install from existing configuration
  • Install a site from config if the config directory is set in settings.php [#2980670]
Documentation

The Documentation initiative has a lot on the go right now, from designing a top-level landing page for the new docs system, to taking a holistic look at the existing docs and how to refactor the IA around them, and finally creating a repository around "quick start" guides. None of these have a particular deadline around 8.6, because they're happening independently of core.

On the core side, there's work being done on a new experimental module for overhauling the in-app help system and this work has an 8.6 deadline.

New topic-based core help system
  • Refactor using a plugin system [#2961552]
  • Add experimental module for Help Topics [#2920309]
Extended Security Support

For the plan around this initiative to happen, we need to make several adjustments to core's Update Status module, which currently makes several hard-coded assumptions about the last minor release of Drupal expiring immediately once a new minor release is available.

Update Status Improvements
  • If the next minor version of core has a security release, status still says "Security update required!" even if the site is on an equivalent, secure release already [#2804155]
  • Status report should indicate next minor release date (needs issue)
  • (other issues TBD)
Layout

The Layout team has been hard at work improving upon the experimental Layout Builder functionality that was added to 8.5. The main goal of the team for 8.6 is to gather real-world testing feedback from end users, which they are accomplishing by adding Layout Builder to a new branch of the Lightning distribution. Doing this has uncovered a few holes in the implementation relative to what's possible in contrib right now, and filling those gaps is the focus of the remaining 8.6 time for the team.

Layout Builder gaps
  • Allow the inline creation of non-reusable Custom Blocks in the layout builder [#2957425]
  • Add a validation constraint to check if an entity has a field [#2976356]
  • Determine if Layout Builder should replace entity_view_display for all Entity Types [#2936358]
  • No ability to control "extra fields" with Layout Builder [#2953656]
  • Allow Custom blocks to be set as non-reusable adding access restriction based on where it was used. [#2976334]
Integration with other subsysytems/modules
  • [PP-1] LayoutBuilderEntityViewDisplay::getRuntimeSections() does not delegate to plugins [#2976148]
  • Add EntityContextDefinition for the 80% use case [#2932462]
  • [meta] Decide how Layout Builder should function with Content Moderation and Workspaces modules [#2973382]
  • Layout Builder does not respect translations [#2946333]
  • Track Layout override revisions on entities which support revisioning [#2937199]
Media

Media has made tremendous strides in 8.6, including remote video support and a newly designed media library.

Next, we need to integrate that media library into the node form, and ideally allow people to add from there as well in a more streamlined fashion.

Blockers to media awesomeness
  • Create a field widget for the Media library module [#2962525]
  • (needs issue) Mark Media Library as beta
  • [PP-1] Allow media to be uploaded with the Media Library field widget [#2938116]
  • Any AJAX call disregards machine name verification when AJAX is used and leads to a fatal error [#2557299]
Migrate

The goal of this initiative for 8.6 is to stabilize the migration system which means marking the experimental Migrate Drupal + Migrate UI modules stable. This was also the goal for 8.5. What's making it tricky is multilingual migrations, which are themselves tricky because there are a multitude of ways one might have set up multilingual functionality prior to it being included in core in Drupal 8, which introduces lots of edge cases around making IDs line up and whatnot.

The team is taking a two-pronged approach here:

1) Attempt to close all of the remaining i18n-related issues.
2) Worst-case, split off multilingual migrations to an experimental module, so that the rest of the system that works for 80%+ of sites can be marked stable.

Make Migrate Stable
  • [policy, no patch] Mark Migrate Drupal as stable [#2905736]
  • [policy, no patch] Mark Migrate Drupal UI as stable [#2905491]
  • [META] Multilingual migrations meta issue [#2208401]
  • Experimental migrate_drupal_multilingual module [#2953360]
Out-of-the-Box

The Umami profile was committed (albeit marked hidden) in 8.5, and major efforts have been going on to remove all of the "beta blockers" preventing it from being visible in the UI. The last of these—Install profile in settings.php and mismatch check makes re-installs of Drupal hard [#2975328]—just landed earlier this week!

From here to 8.6, the team is working on stability and accessibility improvements.

Umami awesomesaceness
  • Un-hide Umami in 8.5 to vastly improve Drupal's evaluator experience [#2957464]
  • Improve Umami demo's support for managing field display settings [#2980029]
  • Improve Umami Demo's header layout and responsive behaviour [#2980528]
  • Umami missing some Media "plumbing" found in Standard profile [#2939594]
Workflow

Last, but certainly not least, is the Workflow initiative, which aims to add the Workspace contributed module to core in 8.6 to facilitate content staging and full-site previews. The module was already committed to 8.6 awhile back, but must be brought up to "beta" level stability to remain in the tagged + shipped release.

Because Workspaces can only stage content that's revisionable, there's also a parallel effort to add revision-ability to more types of data in Drupal core.

Blockers to Workspaces Stability
  • WI: Workspace module roadmap [#2732071]
  • Add workspace UI in top dialog [#2949991]
  • Remove the automatic entity update system [#2976035]
MOAR revisionable thingies
  • Convert taxonomy terms to be revisionable [#2880149]
  • Convert custom menu links to be revisionable [#2880152]
  • Convert comments to be revisionable [#2880154]
Anything else?

Whew! That's QUITE a lot. Are there any issues out there that we're missing that you feel are mission-critical to get into Drupal 8.6? Feel free to suggest them, with the caveat that the longer the list is, the more distributed the community's and core committers' focus is.

Thanks for reading!

Tags: drupaldrupal 8drupal 8.6product manager hat
Categories: Drupal

WeKnow: Creating a Custom Ajax Command in Drupal 8

28 June 2018 - 12:37pm
Creating a Custom Ajax Command in Drupal 8

Drupal 8 provides the option to include an Ajax Callback within our applications using the Ajax Framework. There are some existing functions which can be used: Methods to hide/show elements in the html document, attach content to an element, redirect a page after a submit, and so on. Sometimes we need to implement something particular, or a custom JS code. In that case, those out-of-the-box functions are not enough. Fortunately, we can also create our own custom responses. So, let’s start creating a new ajax callback for a custom form submission.

mcastillo Thu, 06/28/2018 - 19:37
Categories: Drupal

OPTASY: What Are Some Quick and Easy Ways to Secure Drupal? 7-Step Security Checklist

28 June 2018 - 8:16am
What Are Some Quick and Easy Ways to Secure Drupal? 7-Step Security Checklist adriana.cacoveanu Thu, 06/28/2018 - 15:16

You have patched your Drupal website, haven't you? If so, then that critical 3-month-old security flaw Drupalgeddon2 can't get exploited on your site. Even so, with the menace of a cryptocurrency mining attack still lurking around the unpatched websites, you legitimately ask yourself: what are some quick and easy ways to secure Drupal?

“Which are the most basic steps to take and the simplest best practices to adopt to harden my Drupal site's security myself?”

Categories: Drupal

Acro Media: Reducing Unnecessary Code Nesting in PHP

28 June 2018 - 7:45am

The majority of Drupal's underlying code is PHP. As a Drupal developer, the better you know PHP, the better your code will be. In this Acro Media Tech Talk video, Drupal developer Rob Thornton discusses code nesting and how you can optimize your code in order to reduce unnecessary nesting. 

Code nesting can basically be described as when a block of code is contained within another block of code. If you're code isn't well thought out, you can potentially end up with deep nesting that is both hard to read and difficult to maintain. Aside from reducing difficult to read code and making your code more maintainable, reducing the amount of nesting helps you find bugs and lets other developers contribute to your code easier. Rob uses a number of examples of common nesting scenarios, walking you through how to find and fix them.

If you liked this video, you might also like these posts too.

Categories: Drupal

Drupal Europe: Drupal, DevOps and Infrastructures in the brave new world of Distributed Systems

28 June 2018 - 6:55am

Distributed systems face incredible challenges — Photo by Dennis van Zuijlekom

With Drupal 8 reaching its maturity and coupling/decoupling from other services — including itself — we have an increasing demand for Drupal sites to shine and make engaged teams thrive with good DevOps practices and resilient Infrastructure. All that done in the biggest Distributed System ever created by humans: the Internet. The biggest challenges of any distributed system are heterogeneity of systems and clients, transparency to the end user, openness to other systems, concurrency to support many users simultaneously, security, scalability on the fly and failure handling in a graceful way. Are we there yet?

We envision, in the DevOps + Infrastructure track, to see solutions from the smallest containers that can grow to millions of services to best practices in the DevOps world that accomplish very specific tasks to support Drupal and teams working on it and save precious human time, by reducing repetitive and automatable tasks.

Questions about container orchestration, virtualization and cloud infrastructure arise every day and we expect answers to come in the track sessions to deal with automation and scaling faster — maybe using applied machine learning or some other forms of prediction or self management. See? We’re really into saving time, by using technology to assist us.

We clearly don’t manage our sites in the same way we did years ago, due to increased complexity of what we manage and how we are managing change in process and culture, therefore it’s our goal at Drupal Europe to bring the best ideas, stories and lessons learned from each industry into the room and share them with the community.

What’s your story?

How is your platform scaling? How do you solve automated testing and continuous integrations? How do you keep your team’s happiness with feature velocity and still maintain a healthy platform? How do you make your website’s perceived performance even faster? What chain of tooling is running behind the scenes and what is controlling this chain? Are you using agentless configuration management or are you resorting to an agent. Are you triggering events based on system changes or do you work with command and control.

Be ready to raise, receive and answer some hard questions and but most of all, inspire people to think from a different angle. What works for a high-high traffic website might not be applicable for maintaining a massive amount of smaller sites. We want operations to inspire development on reliability and for development to inspire operations on any kind of automation. We want security to be always top of mind while still have an impact on business value rapidly and efficiently. And that is just the beginning…

About industry tracks

Drupal Europe’s 2018 program is focused on industry verticals, which means there are tons of subjects to discuss therefore when you submit your session be sure to choose the correct industry track in order to increase the chance of your session being selected.

Please help us to spread the word about this awesome conference. Our hashtag is #drupaleurope.

To recommend speakers or topics please get in touch at program@drupaleurope.org.

About the Drupal Europe Conference

Drupal is one of the leading open source technologies empowering digital solutions in the government space around the world.

Drupal Europe 2018 brings over 2,000 creators, innovators, and users of digital technologies from all over Europe and the rest of the world together for three days of intense and inspiring interaction.

Location & Dates

Drupal Europe will be held in Darmstadtium in Darmstadt, Germany — which has a direct connection to Frankfurt International Airport. Drupal Europe will take place 10–14 September 2018 with Drupal contribution opportunities every day. Keynotes, sessions, workshops and BoFs will be from Tuesday to Thursday.

Drupalcon Nashville — Photo by Amazee Labs

Categories: Drupal

Blair Wadman: How to find which Twig template Drupal is using to output markup

28 June 2018 - 1:14am

One of the many changes in Drupal 8 is that all HTML output is rendered via a Twig template. This means that if you want to override the HTML for a given page, node, region or field, you can copy the Twig template that is being used to your theme and make your changes.

For any given page, node, region or field, there is normally more than one template that Drupal could use and it will choose the most specific one. So the question is, how do you know which template is being used? And if you override a template, how can you verify that your template is now used?

Categories: Drupal

ADCI Solutions: ADCI Solutions Recognized as a Top Web Developer in Russia!

28 June 2018 - 12:08am

Our Drupal team was featured in a press release highlighting leading Eastern Europe B2B companies on Clutch!

We are proud to deliver projects for our clients and keep working hard.

Learn more here

Categories: Drupal

myDropWizard.com: CiviCRM secrets for Drupalers: Fundraising Campaigns

27 June 2018 - 9:38pm

We're Drupalers who only recently started digging deep into CiviCRM and we're finding some really cool things! This series of videos is meant to share those secrets with other Drupalers, in case they come across a project that could use them. :-)

In the screencast below, I'll show how how you can set-up a new Campaign in Roundearth's CiviCRM! The thing about campaigns is that until there is activity, there isn't much to see, but we have to start somewhere! So, here we setup a campaign.

Watch the screencast to see how to use a Campaign with Roundearth:

Video of CiviCRM secrets for Drupalers: Fundraising Campaigns

Some highlights from the video:

  • Set-up a new Campaign Type
  • Set-up a new Campaign
  • Send a Mailing attached to a Campaign!

Please leave a comment below!

Categories: Drupal

myDropWizard.com: Drupal 6 security update for Generate Password!

27 June 2018 - 12:06pm

As you may know, Drupal 6 has reached End-of-Life (EOL) which means the Drupal Security Team is no longer doing Security Advisories or working on security patches for Drupal 6 core or contrib modules - but the Drupal 6 LTS vendors are and we're one of them!

Today, there is a Less Critical security release for the Generate Password module to fix an Insecure Randomness vulnerability.

The Generate Password modules allows administrators to create a new user account without setting a password, allowing the system to automatically generate one. The module doesn't use a strong source of randomness, creating weak and predictable passwords.

See the security advisory for Drupal 7 for more information.

Here you can download the Drupal 6 patch.

If you have a Drupal 6 site using the Generate Password module, we recommend you update immediately! We have already deployed the patch for all of our Drupal 6 Long-Term Support clients. :-)

If you'd like all your Drupal 6 modules to receive security updates and have the fixes deployed the same day they're released, please check out our D6LTS plans.

Note: if you use the myDropWizard module (totally free!), you'll be alerted to these and any future security updates, and will be able to use drush to install them (even though they won't necessarily have a release on Drupal.org).

Categories: Drupal

Lullabot: The Hidden Costs of Decoupling

27 June 2018 - 12:00pm

Note: This article was originally published on August 23, 2017. Following DrupalCon Nashville, we are republishing some of our key articles on decoupled or "headless" Drupal as the community as a whole continues to explore this approach further. Comments from the original will appear unmodified.

Decoupled Drupal has been well understood at a technical level for many years now. While the implementation details vary, most Drupal teams can handle working on decoupled projects. However, we’ve heard the following from many of our clients:

  1. We want a decoupled site. Why is this web project so expensive compared to sites I worked on in the past?
  2. Why do our decoupled projects seem so unpredictable?
  3. If we decide to invest in decoupled technologies, what can we expect in return?

Let’s dive into these questions.

Why Can Decoupled Sites Cost More?

Before getting too much into the details of decoupled versus full-stack, I like to ask stakeholders:

“What does your website need to do today that it didn't 5 years ago?”

Often, the answer is quite a lot! Live video, authenticated traffic, multiple mobile apps, and additional advertising deals all add to more requirements, more code, and more complexity. In many cases, the costs that are unique to decoupling are quite small compared to the costs imposed by the real business requirements.

However, I have worked on some projects where the shift to a decoupled architecture is fundamentally a technology shift to enable future improvements, but the initial build is very similar to the existing site. In those cases, there are some very specific costs of decoupled architectures.

Decoupling means forgoing Drupal functionality

Many contributed modules provide the pre-built functionality we rely on for Drupal site builds. For example, the Quickedit module enables in-place editing of content. In a decoupled architecture, prepare to rewrite this functionality. Website preview (or even authenticated viewing of content) has to be built into every front-end, instead of using the features we get for free with Drupal. Need UI localization? Content translation? Get ready for some custom code. Drupal has solved a lot of problems over the course of its evolution, so you don’t have to—unless you decouple.

Decoupling is shorthand for Service Oriented Architectures

For many organizations, a decoupled website is their first foray into Service Oriented Architectures. Most full-stack Drupal sites are a single application, with constrained integration points. In contrast, a decoupled Drupal site is best conceived of as a “content service,” accessed by many disparate consumers.

I’ve found that the “black-boxing” of a decoupled Drupal site is a common stumbling block for organizations and a driver behind the increased costs of decoupling. To properly abstract a system requires up-front systems design and development that doesn’t always fit within the time and budget constraints of a web project. Instead, internal details end up being encoded into the APIs Drupal exposes, or visual design is reflected in data structures, making future upgrades and redesigns much more expensive. Writing good APIs is hard! To do it well, you need a team who is capable of handling the responsibility—and those developers are harder to find and cost more.

Scalable systems and network effects

Once your team dives into decoupling Drupal, they are going to want to build more than just a single Drupal site and a single JavaScript application. For example, lullabot.com actually consists of five systems in production:

  1. Drupal for content management
  2. A CouchDB application to serve content over an API
  3. A second CouchDB application to support internal content preview
  4. A React app for the site front-end
  5. Disqus for commenting

Compared to the sites our clients need, lullabot.com is a simple site. In other words, as you build, expect to be building a web of systems, and not just a “decoupled” website. It’s possible to have a consumer request Drupal content directly, especially in Drupal 8, but expect your tech teams to push for smaller “micro” services as they get used to decoupling.

Building and testing a network of systems requires a lot of focus and discipline. For example, I’ve worked with APIs that expose internal traces of exceptions instead of returning something usable to API consumers. Writing that error handling code on the service is important, but takes time! Is your team going to have the bandwidth to focus on building a robust API, or are they going to be focusing on the front-end features your stakeholders prioritize?

I’ve also seen decoupled systems end up requiring a ton of human intervention in day-to-day use. For example, I’ve worked with systems where not only is an API account created manually, but manual configuration is required on the API end to work properly. The API consumer is supposed to be abstracted from these details, but in the end, simple API calls are tightly coupled to the behind-the-scenes configuration. A manual set up might be OK for small numbers of clients, but try setting up 30 new clients at once, and a bottleneck forms around a few overworked developers.

Another common mistake is not to allow API consumers to test their integrations in “production.” Think about Amazon’s web services—even if your application is working from a QA instance, as far as Amazon is concerned there are only production API calls available. Forcing other teams to use your QA or sandbox instance means that they won’t be testing with production constraints, and they will have production-only bugs. It’s more difficult to think about clients creating test content in production—but if the API doesn't have a good way to support that (such as with multiple accounts), then you’re missing a key set of functionality.

It’s also important to think about error conditions in a self-serve context. Any error returned by an API must make clear if the error is due to an error in the API, or the request made of the API. Server-side errors should be wired up to reporting and monitoring by the API team. I worked with one team where client-side errors triggered alerts and SMS notifications. This stopped the client-side QA team from doing any testing where users entered bad data beyond very specific cases. If the API had been built to validate inbound requests (instead of passing untrusted data through its whole application), this wouldn't have been a problem.

There's a lot to think about when it comes to decoupled Drupal sites, but it’s the only way to build decoupled architectures that are scalable and lead to faster development. Otherwise, decoupling is going to be more expensive and slower, leaving your stakeholders unsatisfied.

Why are decoupled projects unpredictable?

When clients are struggling with decoupled projects, we’ve often found it’s not due to the technology at all. Instead, poor team structure and discipline lead to communication breakdowns that are compounded by decoupled architectures.

The team must be strong developers and testers

Building decoupled sites means teams have to be self-driving in terms of automated testing, documentation, and REST best practices. QA team members need to be familiar with testing outside of the browser if they are going to test APIs. If any of these components are missing, then sprints will start to become unpredictable. The riskiest scenario is where these best practices are known, but ignored due to stakeholders prioritizing “features.” Unlike one-off, full-stack architectures, there is little room to ignore these foundational techniques. If they’re ignored, expect the team to be more and more consumed by technical debt and hacking code instead of solving the actual difficult business problems of your project.

The organizational culture must prioritize reliable systems over human interactions

The real value in decoupled architectures comes not in the technology, but in the effects on how teams interact with each other. Ask yourself: when a new team wants to consume an API, where do they get their information? Is it primarily from project managers and lead developers, or documentation and code examples? Is your team focused on providing “exactly perfect” APIs for individual consumers, or a single reusable API? Are you beholden to a single knowledge holder?

This is often a struggle for teams, as it significantly redefines the role of project managers. Instead of knowing the who of different systems the organization provides, it refocuses on the what - documentation, SDKs, and examples. Contacting a person and scheduling a meeting becomes a last resort, not a first step. Remember, there’s no value in decoupling Drupal if you’ve just coupled yourself to a lead developer on another team.

Hosting complexity

One of the most common technological reasons driving a decoupled project is a desire to use Node.js, React, or other JavaScript technologies. Of course, this brings in an entire parallel stack of infrastructure that a team needs to support, including:

  • HTTP servers
  • Databases
  • Deployment scripts
  • Testing and automation tools
  • Caching and other performance tools
  • Monitoring
  • Local development for all of the above

On the Drupal side, we’ve seen many clients want to host with an application-specific host like Acquia or Pantheon, but neither of those support running JavaScript server-side. JavaScript-oriented hosts likewise don’t support PHP or Drupal well or at all. It can lead to some messy and fragile infrastructure setups.

All of this means that it’s very difficult for a team to estimate how long it will take to build out such an infrastructure, and maintenance after a launch can be unpredictable as well. Having strong DevOps expertise on hand (and not outsourced) is critical here.

Decoupled often means “use a bunch of new Node.js / JavaScript frameworks”

While server-side JavaScript seems to be settling down towards maturity nicely, the JavaScript ecosystem for building websites is reinventing itself every six months. React of today is not the same React of 18 months ago, especially when you start considering some of the tertiary libraries that fill in the gaps you need to make a real application. That’s fine, especially if your project is expected to take less than 6 months! However, if your timeline is closer to 12-18 months, it can be frustrating to stakeholders to see a rework of components they thought were “done,” simply because some library is no longer supported.

What’s important here is to remember that this instability isn't due to decoupling—it’s due to front-end architecture decisions. There’s nothing that stops a team from building a decoupled front-end in PHP with Twig, as another Drupal site, or anything else.

If we invest in Decoupled Drupal, what’s the payoff?

It’s not all doom and decoupled gloom. I’ve recommended and enjoyed working on decoupled projects in the past, and I continue to recommend them in discoveries with clients. Before you start decoupling, you need to know what your goals are.

A JavaScript front-end?

If your only goal is to decouple Drupal so you can build a completely JavaScript-driven website front-end, then simply doing the work will give you what you want. Infrastructure and JavaScript framework churns are most common stumbling blocks and not much else. If your team makes mistakes in the content API, it’s not like you have dozens of apps relying on it. Decouple and be happy!

Faster development?

To have faster site development in a decoupled context, a team needs to have enough developers so they can be experts in an area. Sure, the best JavaScript developers can work with PHP and Drupal but are they the most efficient at it? If your team is small and a set of “full-stack” developers, decoupling is going to add abstraction that slows everything down. I’ve found teams need to have at least 3 full-time developers to get efficiency improvements from decoupling. If your team is this size or larger, you can significantly reduce the time to launch new features, assuming everyone understands and follows best development practices.

Multichannel publishing?

Many teams I’ve worked with have approached decoupled Drupal, not so much to use fancy JavaScript tools, but to “push” the website front-end to be equal to all other apps consuming the same content. This is especially important when your CMS is driving not just a website and a single app, but multiple apps such as set-top TV boxes, game consoles, and even apps developed completely externally.

With full-stack Drupal, it’s easy to create and show content that is impossible to view on mobile or set-tops apps. By decoupling the Drupal front-end, and using the same APIs as every other app, it forces CMS teams to develop with an API-first mentality. It puts all consumers on an equal playing field, simplifying the development effort in adding a new app or platform. That, on its own, might be a win for your organization.

Scaling large teams?

Most large Drupal sites, even enterprise sites, have somewhere between 5-10 active developers at a time. What if your team has the budget to grow to 30 or 50 developers?

In that case, decoupled Drupal is almost the only solution to keep individuals working smoothly. However, decoupled Drupal isn’t enough. Your team will need to completely adopt an SOA approach to building software. Otherwise, you’ll end up paying developers to build a feature that takes them months instead of days.

Decoupling with your eyes open

The most successful decoupled projects are those where everyone is on board—developers, QA, editorial, and stakeholders. It’s the attitude towards decoupling that can really push teams to the next level of capability. Decoupling is a technical architecture that doesn't work well when the business isn't buying in as well. It’s worth thinking about your competitors too—because if they are tech companies, odds are they are already investing in their teams and systems to fully embrace decoupling.

Categories: Drupal

Drupal Association blog: Where your money goes - DrupalCI tests

27 June 2018 - 9:52am

During this month's membership campaign, we mention that the average cost of a DrupalCI core test is $0.24-$0.36. Every time a contribution to the Drupal project needs to be tested, DrupalCI spins up a testbot on AWS to test those changes. DrupalCI runs about 5,000 core tests, and 13,000 contrib tests in an average month.  

The test runs on Drupal.org are paid for by our generous partners and members. This is just one of the services provided by the Drupal Association as part of our commitment to maintain Drupal.org so you can focus on Drupal development and community building.

You can help sustain the work of the Drupal Association by joining as a member. Thank you!

Want to hear more about the work of the team? Check out the Drupal.org panel session recording at DrupalCon Nashville.

Categories: Drupal

Pages