Skip to Content

Drupal

ERPAL: Drupal security: How to deliver Drupal updates continuously

Planet Drupal - 20 May 2015 - 3:30am

While developing a system to automate Drupal updates and using that technology to fulfill our Drupal support contracts, we ran into many issues and questions about the workflows that integrate the update process into our overall development and deployment cycles. In this blog post, I’ll outline the best practices for handling different update types with different deployment processes – as well as the results thereof.

The general deployment workflow

Most professional Drupal developers work in a dev-stage-live environment. Using feature branches has become a valuable best-practice for deploying new features and hotfixes separately from the other features developed in the dev branch. Feature branches foster continuous delivery, although it does require additional infrastructure to test feature branches in separate instances. Let me sum up the development activity of the different branches.

Dev

This is where the development of new features happens and where the development team commits their code (or in a derived feature branch). When using feature branches, the dev branch is considered stable; features can be deployed forward separately. Nevertheless, the dev branch is there to test the integration of your locally developed changes with the code contributions of other developers, even if the current code of the dev branch hasn’t passed quality assurance. Before going live, the dev branch will be merged into the stage branch to be ready for quality assurance.

Stage

The stage branch is where code that’s about to be release (merged to the master branch and deployed to the live site) is thoroughly tested; it’s where the quality assurance happens. If the stage branch is bug-free, it will be merged into the master branch, which is the code base for the live site. The stage branch is the branch where customer acceptance happens.

Master

The master branch contains the code base that serves the live site. No active changes happen here except hotfixes.

Hotfix branches

Hotfixes are changes applied to different environments without passing through the whole dev-stage-live development cycle. Hotfixes are handled in the same way as feature branches but with one difference: whereas feature branches start from the HEAD of the dev branch, a hotfix branch starts from the branch of the environment that requires the hotfix. In terms of security, a highly critical security update simply comes too late if it needs to go through the complete development cycle from dev to live. The same applies if there’s a bug on the live server that needs to be fixed immediately. Hotfix branches need to be merged back to the branches from which they were derived and all previous branches (e.g. if the hotfix branch was created from the master branch, it needs to be merged back to the master to bring all commits to the live site, and then it needs to be merged back to the stage and dev branch as well, so that all code changes are available for the development team)

Where to commit Drupal updates in the development workflow?

To answer this question we need to consider different types of updates. Security updates (including their criticality) and non-security updates (bug fixes and new features).

If we group them by priority we can derive the branches to which they need to be committed and also the duration of a deployment cycle. If you work in an continuous delivery environment, where you ship code continuously,the best way is to use feature branches derived from the dev branch.

 

 

Low (<=1 month):
- Bug fix updates - Feature updates

These updates should be committed by the development team and analysed for side effects. It’s still important to process these low-prio updates, as high-prio updates assume all previous code changes from earlier updates. You might miss some important quality assurance during high-prio updates to a module that hasn’t been updated for a long time.

Medium (<5 days):
- Security updates that are no critical and not highly critical

These updates should be applied in due time, as they’re related to the site's security. Since they’re not highly critical, we might decide to commit them on the stage branch and send a notification to the project lead, the quality assurance team or directly to you customer (depending on your SLA). Then, as soon as they’ve confirmed that the site works correctly, these updates will be merged to the master branch and back to stage and dev.

High (<4 hours):
- Critical and highly critical security updates

For critical and highly critical security updates we follow a "security first" strategy, ensuring that all critical security updates are applied immediately and as quickly as possible to keep the site secure. If there are bugs, we’ll fix them later! This strategy instructs us to apply updates directly to the master branch. Once the live site has been updated with the code from the master branch, we merge the updates back to the stage and dev branch. This is how we protected all our sites from Drupalgeddon in less than two hours!

Requirements for automation

If you want to automate your Drupal security updates with the Drop Guard service, all you need is the following:

  • Code deployment with GIT
  • Trigger the update of an instance by URL using e.g. Travis.ci, Jenkins CI, DeployHQ or other services to manage your deployment or alternatively execute SSH commands from the Drop Guard server.
Also to keep in mind:
  • Know what patches you’ve applied and don't forget to re-apply them during the update process (Drop Guard helps with its automated patch detection feature)
  • Automated tests reduce the time you spend on quality assurance
Conclusion

Where to commit an update depends on its priority and on the speed with which it needs to be deployed to the live site. Update continuously to ensure the ongoing quality and security of your project and to keep it future-proof. Feature and bug fix updates are less critical but also important to apply in due time.

For those of you interested in Drop Guard to automate the process as described in this blog post, please sign up for the free trial period so you can test all its features – for free – and get a personal on-boarding.

Categories: Drupal

Jim Birch: Using Drupal&#039;s Environment Indicator to help visually manage Dev, Stage, and Production Servers

Planet Drupal - 20 May 2015 - 2:00am

There are days that I work on half a dozen different websites.  I'm sure some of you are in the same boat.  We make client edits and change requests with rapid effieciency.  We work locally, push to staging, test and review, then push to the live server and repeat.  I would be remiss in saying that I never made a change on the live or staging site accidentally.

The Drupal Environment Indicator module allows you to name, color, and configure a multitude of visual queues for each of your different servers, or other variables, like Git branch or path.  It is very easy to install, and can integrate with Toolbar, Admin Menu, and Mobile Friendly Navigation Toolbar for no additional screen space. 

Once installed, set the permissions of the roles you want to give permission to see the indicator.  You can adjust the general settings at /admin/config/development/environment-indicator/settings

While you can create different indicators inside the admin UI, I prefer to set these in the settings.php files on the various servers so they are not overidden when we move databases back from Production back to Staging and Dev.

Read more

Categories: Drupal

Modules Unraveled: 135 Writing the Book Drupal 8 Configuration Management with Anja Schirwinski and Stefan Borchert - Modules Unraveled Podcast

Planet Drupal - 19 May 2015 - 9:40pm
Published: Tue, 05/19/15Download this episodeWriting a Book for D8
  • What’s it like writing a book for a piece of software that isn’t even officially released yet?
  • How long did the writing process take?
    • Packt publishing sent us a proposal to write this book in December of 2013. We got started quickly, sending them an outline of the chapters and an estimated page count in the same month. The original estimated page count was 150, it turned out to be around 120. We received a pretty strict time line, having to finish a chapter every two weeks, starting in December of 2013.
    • We managed to finish most chapters in two weeks, but some of the longer ones took a little longer since we also started one of our biggest projects we had had until then, also in January. That was pretty tough because that project took up way more than a regular full time job, so we ended up having to write all of the chapters late at night and on the weekends. In May, all of our chapters then went to the editors and we didn’t hear back from the publisher for a really long time.
    • We also told them that we will have to rewrite a lot of the chapters since there was so much work in progress with the Configuration Management Initiative and they were changing a lot about how it worked, like going from the file based default to the database default. I think it was in January of 2015 when chapters came back with some feedback and we started rewriting every chapter, which was pretty painful at the time. We were able to update some of the documentation at drupal.org with the changes we found. It felt good to contribution at least a small part, when with our project and the book we had no time left to contribute code to Drupal 8 like we usually do.
    • We spent around 40 days on the book between the two of us.
    • In December, Packt asked the first publisher to review the book. We had recommended them one of our team members at undpaul, Tom, who has a similar amount of Drupal knowledge as Stefan. We really wanted to have someone from CMI to review the book, like Greg Dunlap. They had turned down reviewing the book after the first chapters were written, because too much would still change. Then after the changes went in we kept recommending Greg but I never heard anything back, maybe he was busy or they didn’t bother to ask. At the beginning of this year they told us the book was planned to be published by March. We recommended waiting because we didn’t expect a release candidate before the European Drupalcon and we would have rather had someone like Greg take the time to review, but Packt had another opinion :) Since most of CMI changes were finished, we didn’t feel too uncomfortable about the time of publishing, and it was also kind of nice to finally be done with this thing :) So it took a little over a year from start to finish. It was published on March 24th.
  • Do you expect to need to rewrite anything between now and when 8.0 is released?
The Book: Drupal 8 Configuration Management
  • What do you cover in the book?
    • We start of with a basic introduction to what Configuration Management in Drupal means, because it is a thing in Software development in general, that doesn’t usually refer to what it is in Drupal, where it basically just means that configuration is saved in files which makes deployment easier. In the first chapters, we make sure the reader understands what Configuration Management means and why version control is so important. We mention some best practices and then show how to use it for non-coders as well, since there’s a nice backend non-technical folks can use, even if you don’t use version control (which of course we don’t recommend). We also have a part that describes how managing configuration works in Drupal 7 (Features!) and then dive into code examples, explaining schema files, showing how to add configuration to a custom module, how to upgrade Drupal 7 variables to the new system and cover configuration management for multilingual sites.
  • Who is the target audience of the book?
  • Why did you decide to write about Configuration Management?
    • We have used Features to deploy configuration changes for a very long time, I don’t recall not using it since we started the company 5 years ago. We have talked about it at several DrupalCamps and Drupal User Groups and always tried to convince everyone to use it. We were really excited about the Configuration Management Initiative and thought it was a very good fit for us.
  • Before we started recording, you mentioned that there is a companion website to the book. Can you talk about what content we’ll find there, and what purpose that serves?
  • Are you building any sites in D8 at Undpaul?
Episode Links: Anja on drupal.orgAnja on TwitterStefan on drupal.orgStefan on TwitterWhere to buy the bookThe website for the bookundpaul on Twitterundpaul Instagramundpaul websiteTags: Drupal 8Bookplanet-drupal
Categories: Drupal

Flysystem S3

New Drupal Modules - 19 May 2015 - 5:13pm

Provides an Amazon S3 plugin for Flysystem.

See README.txt for configuration information.

Categories: Drupal

Flysystem Rackspace

New Drupal Modules - 19 May 2015 - 5:02pm

Provides a Rackspace plugin for Flysystem.

See README.txt for configuration information.

Categories: Drupal

Flysystem ZIP

New Drupal Modules - 19 May 2015 - 4:48pm

Provides a ZIP file plugin for Flysystem.

See README.txt for configuration information.

Notes

There is currently a bug in the Flysystem adapter that stops it from working with Druapl.
https://github.com/thephpleague/flysystem-ziparchive/pull/6

Categories: Drupal

Flysystem SFTP

New Drupal Modules - 19 May 2015 - 4:41pm

Provides an SFTP plugin for Flysystem.

See README.txt for configuration information.

Categories: Drupal

Flysystem Dropbox

New Drupal Modules - 19 May 2015 - 4:31pm

Provides a Dropbox plugin for Flysystem.

See README.txt for configuration information.

Categories: Drupal

Gizra.com: Visual regression tests on every commit

Planet Drupal - 19 May 2015 - 2:00pm

As we dive deeper into visual regression testing in our development workflow we realize a sad truth: on average, we break our own CSS every week and a half.

Don't feel bad for us, as in fact I'd argue that it's pretty common across all web projects - they just don't know it. It seems we all need a system that will tell us when we break our CSS.

While we don't know of a single (good) system that does this, we were able to connect together a few (good) systems to get just that, with the help of: Travis-CI, webdriverCSS, Shoov.io, BrowserStack/Sauce Labs, and ngrok. Oh my!

Don't be alarmed by the long list. Each one of these does one thing very well, and combining them together was proven to be not too complicated, nor too costly.

You can jump right into the .travis file of the Gizra repo to see its configuration, or check the webdriverCSS test. Here's the high level overview of what we're doing:

Gizra.com is built on Jekyll but visual regression could be executed on every site, regardless of the underlying technology. Travis is there to help us build a local installation. Travis also allows adding encrypted keys, so even though the repo is public, we were able to add our Shoov.io and ngrok access tokens in a secure way.

We want to use services such as BrowserStack or Sauce-Labs to test our local installation on different browsers (e.g. latest chrome and IE11). For that we need to have an external URL accessible by the outside world, which is where ngrok comes in: ngrok http -log=stdout -subdomain=$TRAVIS_COMMIT 9000 from the .travis.yml file exposes our Jekyll site inside the Travis box to a unique temporary URL based on the Git commit (e.g. https://someCommitHash.ngrok.io).

WebdriverCSS tests are responsible for capturing the screenshots, and comparing them against the baseline images. If a regression is found, it will be automatically pushed to Shoov, and a link to the regression would be provided in the Travis log. This means that if a test was broken, we can immediately see where's the regression and figure out if it is indeed a bug - or, if not, replace the baseline image with the "regression" image.

Visual regression found and uploaded to shoov.io

Continue reading…

Categories: Drupal

Mediacurrent: Contrib Committee Status Review for April, 2015

Planet Drupal - 19 May 2015 - 1:47pm

The fourth month of the year brought reminders that Winter can show up at unexpected times, with snow flurries during the early parts of the month. It also that we can only juggle so much. With many of us involved in organizing regional events and preparing for Drupalcon, our code contributions waned for a second month, down to a rather low 20 hours.

Categories: Drupal

Drupalpress, Drupal in the Health Sciences Library at UVA: executing an r script with bash

Planet Drupal - 19 May 2015 - 1:43pm

Here’s a tangent:

Let’s say you need to randomly generate a series of practice exam questions. You have a bunch of homework assignments, lab questions and midterms, all of which are numbered in a standard way so that you can sample from them.

Here’s a simple R script to run those samples and generate a practice exam that consists of references to the assignments and their original numbers.

## exam prep script ## build hw data j <- 1 hw <- data.frame(hw_set = NA, problm = seq(1:17)) for (i in seq(1:12)) { hw[j,1] <- paste0("hw",j) j <- j+1 } library(tidyr) hw <- expand(hw) names(hw) <- c("problm_set", "problm") ## build exam data j <- 1 exam <- data.frame(exam_num = NA, problm = seq(1:22)) for (i in seq(1:8)) { exam[j,1] <- paste0("exam",j) j <- j+1 } library(tidyr) exam <- expand(exam) names(exam) <- c("problm_set", "problm") ## create practice exam prctce <- rbind(exam,hw) prctce_test <- prctce[sample(1:nrow(prctce), size=22),] row.names(prctce_test) <- 1:nrow(prctce_test) print(prctce_test)

As the last line indicates, the final step of the script is to output a prctce_test … that will be randomly generated each time the script is run, but may include duplicates over time.

Sure. Fine. Whatever.

Probably a way to do this with Drupal … or with Excel … or with a pencil and paper … why use R?

Two reasons: 1) using R to learn R and 2) scripting this simulation let’s you automate things a little bit easier.

In particular, you can use something like BASH to execute the script n number of times.

for n in {1..10}; do Rscript examprep.R > "YOUR_PATH_HERE/practice${n}.txt"; done

That will give you 10 practice test txt files that are all named with a tokenized number, with just one command. And of course that could be written into a shell script that’s automated or processed on a scheduler.

Sure. Fine. Whatever.

OK. While this is indeed a fairly underwhelming example, the potential here is kind of interesting. Our next step is to investigate using Drupal Rules to initiate a BASH script that in turn executes an algorithm written in R. The plan is to also use Drupal as the UI for entering the data to be processed in the R script.

Will document that here if/when that project comes together.

Categories: Drupal

Webform Simple Hierarchical Select

New Drupal Modules - 19 May 2015 - 9:01am

Provides a webform component which renders a shs field.

Sponsored by Kirk Monteux for mediapal.de

Categories: Drupal

Drupal Watchdog: Web API Alphabet Soup

Planet Drupal - 19 May 2015 - 8:56am
Feature

Drupal 8 offers unprecedented support for creating RESTful APIs. This was one of the major goals of the Web Services and Context Core Initiative (WSCCI), and one we've delivered on pretty well. However, as with most things that are worth doing, just because Drupal core “supports” it doesn't mean you'll see good results without an understanding of what's going on. In this article, we'll explore some of these principles, so that when it comes time to design with those systems, you'll know how to think about the problem.

HAL

Drupal 8 ships with support for encoding and representing its entities (and other objects) via the Hypermedia Application Language (HAL) specification. HAL can currently be expressed in JSON or XML, and is a specification for describing resources. As the specification says, HAL is “a bit like HTML for machines.”

What that means is that a HAL API can provide enough data for a machine agent to visit the root ("/") of a website, then navigate its way through the remainder of the system exclusively by following the links provided in responses. Humans do the exact same thing by visiting a page and clicking on links. The notion that machines might also want to do this is a relatively obvious idea, but one that has, until recently, rarely been followed on the web.

Still, though, it's pretty abstract. To really understand why HAL is powerful – and what it does for us in Drupal – it's necessary to go back to the basic constraints and capabilities of the problem space it operates in: HTTP and REST. The crucial documents there are RFC2616 and Roy Fielding's thesis, both well-worth [re-]reading. But a more easily digestible version comes in the form of the Richardson Maturity Model, first laid out by Leonard Richardson in 2008, and since revisited by Martin Fowler and Steve Klabnik.

RMM

The Richardson Maturity Model helpfully suggests a set of four “maturity” levels into which HTTP APIs fall:

Categories: Drupal

Commerce discount cumulative

New Drupal Modules - 19 May 2015 - 7:02am

Commerce discount module does not give cumulative options, so all active discounts matching with conditions will be applied.

What the module does
  • Create field and instances in commerce_discount to choose if discount is cumulative or not.
  • Add condition rule to disallow discounts following settings.
  • Add action rule to remove previous discounts in case of "exclusive" setting.
  • Discounts are applied in the order of rules's weight.
Features available

A discount can be :

Categories: Drupal

Cheeky Monkey Media: Building a custom module part 1

Planet Drupal - 19 May 2015 - 7:00am

This tutorial is written for new drupal developers or php developers who want to learn drupal.

We are going to cover the following in this tutorial:

  • Creating the file structure for the module.
  • Creating a simple table.
  • Register a path to display the custom form.
  • Create a custom form with 4 fields.
  • Capturing and saving the form values in the database.
  • Retrieving and re-populating the form with user's input.

Let's get started.

Step 1: File Structure

Create the structure for our first module. We are...

Categories: Drupal

Commerce discount weight

New Drupal Modules - 19 May 2015 - 6:39am

By default, discounts from Commerce Discounts can be ordered thanks to rules UI, editing each rule.
This is not easy, plus editing rule flag discount as overriden.

Purpose

This module helps you to change order of your discounts.

What the module does
  • This module add a weight field to commerce discount.
  • Saving discount, weight of the discount rule is updated.
  • A view is created to order discounts with drag and drop and is available on admin/commerce/store/discounts/weight as a tab of discounts overview.
Categories: Drupal

Drupal governance announcements: Now accepting nominations for the Aaron Winborn Award!

Planet Drupal - 18 May 2015 - 2:28pm

As mentioned during Dries's DrupalCon LA keynote, the Drupal Community Working Group is now accepting nominations for the Aaron Winborn Award, to honour Drupal community members who demonstrate personal integrity, kindness, and above-and-beyond commitment to the Drupal community.

Nominations are open until Monday 15 June 2015, and the selected recipient will receive a scholarship and stipend to attend DrupalCon with recognition during a plenary session at the event.

Submit your nominations here: https://www.drupal.org/aaron-winborn-award

Categories: Drupal

Acquia: How to Select Drupal Modules: Part 3 - Evaluation Tips

Planet Drupal - 18 May 2015 - 1:26pm

In the previous posts we’ve focused on defining your requirements and the basics of searching for modules. Once you’ve found a Drupal project you’re interested in, now you can make a quick evaluation of the project to determine if you should dig deeper before you test it out.

Evaluation Criteria

Each module you select and install on your site must be maintained. There will be security updates, feature improvements and bug fixes offered on a rolling basis. The update manager within Drupal will notify you when new releases are available. This means you will never miss a key security release.

If a module is actively maintained it will mean that one aspect of your site is more likely to be secure and bug-free. One less thing to worry about! Take a “maintenance first” approach to module selection to limit potential issues arising from compatibility issues or security issues that might arise.

An initial evaluation is something an experienced Drupal developer might do in about one to two minutes, simply to compare two modules to decide which to download and try first. However, let’s tease this apart. There are three useful criteria for evaluating a module.

  1. Reputation: How many maintainers? What other contributions have the maintainers made? Is the individual or company a member of the Drupal Association?
  2. Reach: Is there a community around the module? Are there related modules which integrate with it? What is the total number of installations? Checking the usage over time is there a stable arc?
  3. Currency: Have there been recent commits? Are issues being added by users? Is the maintainer responding? Is there a stable (green/not alpha or beta) release available?

These criteria can give you some indication of the level of effort that is being invested in maintaining the software, and help you interpret information on the project page.

The Project Page

You can determine how a project scores against those criteria based on the information available on the project page itself. A wealth of information is available.

  1. Description: This should provide some basic information about the project and you should be able to tell what requirements the module has.
  2. Project information: Maintenance status and how many reported installations. Just because only two others use a project, doesn’t mean it’s not a good start for a solution for your team.
  3. Downloads: Is there a compatible version available? If it's not recently updated it might be a warning sign, or it might just be a stable, well-used module that just works.
  4. Maintainers: Is there an active team of maintainers? You can look at their profiles which also list other contributions and activities.
  5. What are current issues? The graphs indicate recent activity and also a brief analysis of how responsive the maintenance team is. Keep in mind most of this work is done on a voluntary basis, so if you’re willing to help out, you can often get a better response.
  6. Is documentation available? This will help you in the next step of testing and exploring the module.

The project information provided should be considered in relation to the other information. For example, you might see a project like Bean doesn’t have a Drupal 8 version. This might make you wonder if the solution is future-friendly. In this case, similar functionality has been incorporated into Drupal 8, so it actually makes this module unnecessary.

To give another example, a project with few installations could be just that unique solution you need to connect your Drupal site to an obscure third party application. And as another example, a project managed by a Drupal newcomer who has few contributions could be a great sign that someone is bringing in new skills and experience to the community.

I would never disparage or dismiss a project based on just one of the criteria. Make sure you look at each aspect of the project and balance it with the rest of the information available.

How can I help?

OK, now we’ve whittled down our choices and found a module or two we’d like to try out. In the next blog post, we’ll actually install and test out a module. After that, I’ll show you how to explore and “learn” a new module.

In our Drupal Site Building course we focus on the essential building blocks of Drupal and contributed modules. In fact, some of the contributed modules we use in the Drupal 7 course have become core in Drupal 8, which is a good sign that the community has convened around specific requirements and solutions.

If you’re stuck trying to find a module for X, please leave a comment and I’ll help you find the module you’re looking for.

Tags:  modules drupal 7 site building training learning functionality acquia drupal planet
Categories: Drupal

Drupal for Government: Tracking Charlottesville City Expenses in the Crowd - Part 2 Charts!

Planet Drupal - 18 May 2015 - 12:36pm

After part one aka "getting the data cleaned up and in there" it's time to do a few things to make the pretty pictures.  Using the charts tool is a pretty natural visual drill down tool, and its integration with highcharts means it can handle a ton of data. Attached below is a feature that includes all the content type, view, and feeds, I've also attached the original data in case there's someone out there interested in learning how our city blows its wad ;)

Categories: Drupal

Drupal CMS Guides at Daymuse Studios: Drupal, Content Strategy, and the Prevailing Power of the Play Button

Planet Drupal - 18 May 2015 - 12:36pm

Video is key to increasing user engagement. Learn how to integrate YouTube video with your content in Drupal using the Media and Media: YouTube modules.

Categories: Drupal
Syndicate content


Google+
about seo