WebbyKat: Notes from DrupalCon 2013: Improving the editor experience, better performance testing, CSS architecture for the future, Drush 6, and more
DrupalCon was held in Portland last week. I took detailed notes for (most of) the sessions I attended. Here are the ones that were good enough to share with the internet. They also link to their session pages, where a lot of the videos have been posted.
For more information, and for session videos, check out the official DrupalCon 2013 website.
- Secrets to Awesomizing Your Editor's Back-End Drupal Experience: If I'd only gone to one session, I'd have wanted it to be this one. Mike showed a variety of modules that make an editor's job a little easier. If you want editor buy-in on your Drupal site, that's a huge deal. Presented by @mikeherchel.
- Making Sense of the Numbers: DOs and DON'Ts of Quality Performance Testing: This session covered the terms people throw around with performance testing, who has responsibility for good site performance (spoiler: it's everyone), and straightforward dos and don'ts that help you get usable data out of your performance testing. Presented by @erikwebb and @doogiemac.
- What's New in Drush 6: I love this session for showing me how to upload my module list to Google Docs, as a spreadsheet, with ONE COMMAND. Presented by @weitzman, @msonnabaum, @jhedstro, and @grugnog.
- Scalable and Modular Architecture for CSS: You know that sinking feeling you get when you look at a 4,000 line CSS file and wonder how it happened? This will help you avoid that feeling. Presented by @snookca.
- Multilingual Module Madness: What i18n Modules Do You Really Need?: There are lots of multilingual modules out there. There's no perfect way to know which ones you need (other than a lot of experience), but the aim of this session was to give you a starting point based on your project's requirements. Presented by @kristen_pol.
- What Developers Need to Know About Dev Ops: My notes on this are more haphazard than I'd like, but this session on what developers need to know about the environment their site will live in had great information -- like sample differences between a dev box and a production box, and a list of common pitfalls when you develop without understanding your production environment. Presented by @gchaix, @ramereth, @KenLett, and @basic_.
- Should Have Made a Left Turn at Albuquerque: Building Maps in Drupal - These are really limited notes. I normally wouldn't share them, but there were two fantastic mini-sessions: one gave a "recipe" for building great maps in Drupal, and the other showed impressive interactivity features with D3js.
- Rules Without the UI: This was a very detailed session, with lots of code examples, showing you how to set up rules in your module's code. Presented by @dwkitchen.
As a web developer, you were faced with a unique decision this year. Should you get your ultimate geek on at the 2013 DrupalCon in Portland, Oregon, OR get your ultimate freak on at the 2013 Burning Flipside art and music festival in Austin, Texas?
For many of you, the choice was clear. Drupal seems to be here for the long-haul. If DrupalCon attendance follows its historic growth patterns, it will truly reach some mammoth proportions 5 years from now. It’s not going anywhere. On the other hand, there are a limited number of years that you can get away with attendance at the Burning Flipside festival. Eventually you’re going to turn into the old geezer that’s heckling from the sidelines, or maybe you’ll gain a significant other that doesn’t quite approve of the shenanigans necessary for a good art and music festival. Enjoy it while you can, brother. I salute you.
While you’re enjoying that fantastic Austin weather, I’ve ‘taken one for the team’ this year. I did the responsible thing. I went to DrupalCon 2013. But don’t mistake that to mean that DrupalCon was a bust. I enjoyed a city I had never visited, ate some fantastic food, watched a couple spectacular presentations (and some coma-inducing ones as well), made new business contacts with some groups we’d like to partner with, and generally enjoyed touching base with both old and new acquaintances in the Drupal community. The web team at Interworks truly walked away from the conference stronger than when we arrived.
Now that we’re all back in the office, you don’t have time to watch all of those re-cap videos from the conference you didn’t attend. Once again, I’m here with the executive summary to save you time in getting to the meat of the conference.
MONDAY: The Trip to Portland
My trip to Portland began with an upgraded seat (first class seat 3A) on the plane from OKC to DFW. While this is only a short 30 minute hop, it’s always nice to get that surprise. The rest of my flights were on time and uneventful, and I met up with Scott Siemens at the PDX airport so we could share a ride to the hotel.
Shortly after checking into the hotel, we met up with Dalton Parsons and Javod Khalaj. We immediately tracked down a highly recommended Portland eatery so we could plan our strategy for the week. Jake’s Famous Crawfish earned our patronage and we weren’t disappointed. I have a pretty nasty shellfish allergy so I ordered the steak, but my colleagues raved about the seafood. Even Scott, normally being an exclusive pizza and tacos aficionado, tried the calamari and asparagus, and liked it!
TUESDAY: Fired Up!
Tuesday kicked off with a presentation that I had at the top of my list: How Oregon State University Manages Large Scale Drupal. With all the work that Interworks is involved in at Oklahoma State University, this session was a no brainer. I was really interested to see how they had addressed some of the problems that we are facing. They’ve gotten quite a bit further into their deployment of Drupal, and I’m glad that they illustrated how difficult it’s getting to manage 500+ separate Drupal sites. Their solution is Organic Groups for departments where it makes sense to combine Drupal sites. It was also interesting to see how they had implemented their own version of the Multisite model using symbolic links to a common codebase. The fact that they had used Drupal version numbers as part of their symbolic link path is a great idea since you can immediately see what version of core a site is running from the directory structure. It also illustrated that they’re stuck in the world of Drupal 6 still. I was in a later session where Paul Lieberman asked the speaker a question about upgrading to Drupal 7, and the recommendation was to skip that version and wait until Drupal 8 is released next year. That sounds like a massive undertaking given the size of their deployment. I’ll be interested to see how they proceed over the next year or so.
Asset Management in Drupal 8 made it clear that the core developers are taking away two of my favorite functions: drupal_add_js() and drupal_add_css(). It makes sense that they are removing those, given the negative impact they have on performance and site caching. But it bummed me out nevertheless. Those functions have been my security blankie on past projects.
We ended Tuesday by participating in an emergency code sprint to assist emergency responders with the Moore, OK tornado relief efforts. There was an inspiring turnout of developers for this event, and our main contribution was in the discussions around promoting the site that would eventually be developed to facilitate ride sharing and housing needs for the disaster area. Social media was an obvious avenue but we also made contacts back in Oklahoma to get the support of media outlets and the Oklahoma Drupalers Group just in case there was a need for that.
WEDNESDAY: Does the Rain Stop in this City?
Some of us were better at acclimating to the cold precipitation in Portland than others, but the public transit system really made it easy for us to traverse the city without getting too soaked. That is, when we boarded the correct bus. More about that in a minute.
The first break-out session that I attended was Chris Blow’s Design Ops: A UX Workflow for 2013. Hands down, I give this session the award for being the most entertaining, while being the most devoid of anything that I could write in my notes. I was inspired by the use of expletives in his presentation. Chris truly has an inspiring speaking style. His monotone wit was delivered in a manner very similar to Steven Wright. His thoughts were perfectly timed to flow from one to the next, and he threw no less than an average of 3 slides per minute at us. At the end of the presentation, I’d say his cookie monster slide really summed things up nicely with its “WTF is This?” caption. That being said, I’m pretty sure I understood what Chris was really getting at. Designers should do some coding once in a while. Coders should do some design once in a while. It only serves to make you better at your role on the team when you understand how the other side of the coin works.
I walked away with mixed feelings about Drupal 8’s new templating engine after attending Using Twig: The New Template Engine in Drupal 8. I’m fully behind what they’re trying to do in terms of performance improvements, but it feels like they’re taking away quite a bit of the PHP that I’ve grown familiar and comfortable with. I’m keeping the faith that they’ll make good on their promise to provide equivalent Twig tools that replace how I’ve approached template development in the past. More than anything, I’m interested to see how this is going to affect some of the starter themes that we use.
After the conference closed up for the day, I was really pushing to visit Pok Pok for supper. I had seen the episode of Diners, Drive-Ins, and Dives that highlighted the cuisine, and I couldn’t wait to give the boar collar and Vietnamese wings a try. Our group had pretty much followed Dalton around the city blindly up to this point, so we boarded a bus behind him confident that we were on the road to amazing Thai food. It didn’t take long for our smartphone GPS apps to illustrate that we were generally heading in the right direction, albeit somewhere around 15 blocks askew of our intended bus stop. We exited the bus and began our side-long journey through the downpour of Portland, although I had layered and prepared for a detour like this. We were all soaked to varying degrees by the time we arrived at the restaurant. Dalton’s detours had already earned the nickname ‘forced march’ and this walk gave that term some serious meaning.
There was a 45 minute wait for a table once we arrived, but Pok Pok has a sister café across the street called the Whiskey Soda Lounge. It was pretty cool to sit there and have some drinks (out of the rain) while we waited, and our Whiskey Lounge waiter even told us when our table was ready across the street. We ordered several items and shared everything so we could enjoy a variety and there was nothing that disappointed. If you’re in Portland, track this place down. You won’t find Thai food like this anywhere, period.
THURSDAY: Wrapping Up
One of the major initiatives that I’ve been interested in for Drupal 8 is the new Configuration management, so Thursday’s most exciting session from my perspective was Greg Dunlap’s Using the Drupal 8 Configuration System. The problems that we’re facing at Oklahoma State University with rolling out feature updates and security patches will almost entirely go away once we can move to Drupal 8’s configuration management model. In a nutshell, they’ve moved anything that’s configuration related (permissions, CCK, views settings, etc.) to flat files so you’re not colliding with user content in the database. Not only does this make it easier for us to apply those updates, the flat configuration files are easily versioned now, whereas it was next to impossible to version the database-stored configurations in previous versions of Drupal.
I’m hoping that the plan for configuration management extends to the multisite model for Drupal. I could see some real advantages to having a central configuration that all sites share, while still allowing single configuration files to exist in the sub-sites as an override for the global setup. A good example might be that one of your sites needs to have 10 items in a view instead of the 5 that all other sites display. It would just be epic if it winds up working that way.
Dalton’s bus-finding privileges had been revoked by the time the conference broke up for the day. So we consulted Yelp for an eatery within a few blocks of our hotel. Since our server admin colleagues from Oklahoma State University were also attending the conference, it seemed like a great night to gather and review our thoughts on what we’d seen. We all agreed to visit Karam Lebanese and Syrian Cuisine. We ate the goat. We ate the lamb. We had kabobs. I ordered Baba Ghannouj because it’s just plain fun to say, AND it’s quite tasty. I’ve never liked hummus, but it was REALLY good at Karam. The Labne cheese was a delicious surprise as well. But it was the chicken shawarma that I just couldn’t eat enough of. That stuff was AMAZING. The Lebanese Pilsner was a nice complement to all the food that was served. And I still hated the stuffed grape leaves. I just don’t like those things, I think.
FRIDAY: The Trip Home and Conclusions
Dalton and I traveled from PDX to DFW together on the way home, although I had an additional two hour layover as he boarded his plane to OKC. It turns out that I was in the better position since his flight sat on the runway waiting for the weather to clear while I grabbed a sandwich and explored the airport. In the end, his flight landed in OKC no more than 10 minutes before mine.
It was a whirlwind trip, but it energized my enthusiasm for Drupal and the development community. I had already started formulating the thoughts for this blog post when I realized that next year, there will be no scheduling dilemma. DrupalCon 2014 will be held in Austin, Texas. Coincidence? I think not.
Your humble narrator holding a bucket of doughnuts, because Portland.
Several of us here at LC Advanced travelled to Portland, OR last week to attend Drupalcon. Despite soaking up as much rain as knowledge, we all had an awesome time out there, and left in great spirits. The best part of Drupalcon (other than Voodoo Donuts of course), were the people we met. We spent the whole week hanging out with Drupalists from all over the world, including a bunch from Western Massachussets. You can't beat that.
Oh yeah, and then there's the sessions. For a serious Drupal enthusiast, it can be hard to find quality information relating to your specific interests. Even if you can find a Drupal meetup, you're going to get some blank stares when you try to start a conversation about something obscure like building repeatable infrastructure with Chef. In this regard, Drupalcon sessions are like an all-you-can-eat buffet for a nerdy Drupal guy like myself. There's literally no other place in the world where you'll find hundreds of people jammed into a room to talk about the new REST API's available in Drupal 8.
Here were a few of my favorite sessions from Portland. Follow the links to check them out.
Building for a post-mobile world - Lullabot's Jeff Eaton has to be one of the most entertaining presenters I've ever seen. He was able to communicate the basics of good content strategy in a way that made it interesting, even for my code-crazy developer brain. His argument for getting rid of (or at least severely limiting) the client's reliance on the WYSIWYG struck home.
Devops for Drupal: Why We Cook With Chef - Promet Source put on a great presentation explaining why they made the switch from managing their servers with shell scripts and prayers to using Chef. What this presentation lacks in... uh, presentation, it makes up for in real world explanations and tips for how a busy dev shop can bake saner, more consistent environments into their workflow.
Behat, Behavioral-Driven Development and Selenium in Drupal - KNPLabs Ryan Weaver brings a fresh perspective on the development cycle in talking about testing tools for real people (AKA Behat). For those not in the know, the purpose of BDD is to turn written specifications (like "When I log in as an administrative user, I should see my profile") into executable tests that can be verified programmatically. This is hugely different from Drupal's current testing system, Simpletest, which you may need to be an evil supergenius to use. Despite coming from a Symfony background, Ryan does an awesome job of focusing on how Behat can be used with Drupal.
So anyway, that's a quick snapshot of my week in Portland. If you were there and met any of us, feel free to drop us a line. Look forward to seeing you all in Prague and Austin!Tags: drupalconCategory: Drupal Planet, News
Modules Unraveled: 064 DrupalCon Prague with Rick Nashleanas, Andrew Burcin and Stephanie El-Hajj - Modules Unraveled Podcast
- When is it?
- Where is it?
- Who are the keynote speakers, and what will they talk about?
- What tracks are available?
- Are there any featured sessions you’d like to point out?
- What is the DrupalCon global team?
- Historians, consultants, volunteers who get no credit
- Who are the globals?
- Standing tracks, North American and European leaders,
- What are the qualifications to be a global?
- Previously a local track chair, content expert
- What do you do?
- Herd cats, deal with problems
This is a replacement for the core dblog module.
This module provides allows you to configure whether messages are logged for anonymous users, which error levels are logged, and which types of message are logged.
@todo Use a shutdown function to write all per page log messages in a single db_insert() call at the end of the page load, or optionally push them into a queue to insert later for users running fast queue backends such as mongoDB.
Recently a post appeared which stated Drupal became the Scientology of open source and because it appeared in a reply of a few tweets involving me and it contains my name, I need to answer it.
> The goal of the Drupal project is, by design, one that is perpetual: to build and maintain the Drupal software. It is implicitly assumed that this is so, because the web is always changing.
While the web is indeed changing and Drupal is changing with it, there are many challenges that we tried to address in the past and failed to do so. Much unlike a cult, we are actually self reflecting (see for example the Engineering for the 80%, too presentation) and try to improve.
> The websites that the majority of users now spend their time on—Facebook, YouTube, Twitter, Gmail—bear little resemblance to the mid-2000s web portals that Drupal still creates out of the box, or the tools it offers to manage them.
Well, Drupal is a product that some find useful. Others realize it could be made more modern and help like the 600 or so people who showed up at the sprint last Friday. It didn't seem to me they were given anything for this, aside from a free lunch. Perhaps they are given some social status and eventually bills indeed get paid as the original post claims -- but there is no surety of that. There were many people for whom Drupal is not a dayjob just an interesting hobby.
> I'd like to propose a radically different view: Drupal is perpetual, because there is no real goal and never really has been. It merely sustains itself.
I thought we are building websites. We approaching a million websites reporting back to drupal.org. If that's not a goal, I have no idea what is.
> No global plans were ever drawn up, no requirements to verify.
That's because it's somewhat hard to convince a thousand people contributing to core alone to adhere to a single plan. But it's not impossible to have plans involving a subsystem and execute that. When you show people a convincing idea they will show up and help -- have you seen how many people helped with the Twig conversion which most definitely had a plan?
> Implicitly, it's assumed that everything that currently is, should be.
We have moved modules to contrib in Drupal 7 and Drupal 8 both. This statement is factually false.
> Instead of focusing on the end-result, the focus is on the process of development: to make Drupal better, piece by piece. Better for what?
Hint: a million websites are not made from the same cloth. We are focusing on a product that allows various, oftentimes conflicting visions to be built on top.
> Rather, a user should be presented with the website as an open canvas, able to make any number of changes freely.
Karen McGrane's keynote says "It is time to go to war against the blobs". I have no clue whatsoever whether she is right or this blog post is right. This stark (pun intended) contrast however shows very well what I said above: there's no single master plan except for trying to create a versatile tool.
> See, they're not bringing Drupal into the world of Symfony. They're bringing Symfony into the world of Drupal.
Perhaps because Drupal implements more functionality than Symfony does...?
> There is always an overarching Drupal-way that must be maintained, a sensibility and taste that is superior. Chx said as much: Drupal always did it better.
And the community disagreed with me and changed Drupal in ways I disagree with and yet the sky didn't fall. We sometimes disagree and yet we still build together. And yes, we do not know what we are building. Often, Drupal is compared to Lego and I like this comparison.
> Which then leads to the next release: a giant orgy of enthusiasm, the next best thing, the most amazing Drupal ever. If that's the case, why do developers instantly start developing again?
Let me get this straight: if we managed to produce more interesting Lego cubes, we are not allowed to go on trying to create even more interesting ones? This logic seems flawed.
> Clients are notified, updates are scheduled, new development is requisitioned, and the bills get paid. Until the next release, when things will break again. And remember, If your Drupal site is not usable, you're using Drupal wrong.
Except if a site is not satisfied with Drupal they will move to another software. It has happened and will happen all the time. Even whole development shops left Drupal and we have not striken their names from the scrolls, forbidden their name to be uttered. Rather, we congratulate them when the results of their new ventures show up in Wired.
> For Drupal's mission to succeed, it has to be abandoned in its current form entirely, and be reborn as a dozen different solutions to individual problems, each of which can stand alone. Structure, layout, templating, formatting, visualization, interaction, these are the tools of the publisher. Not database entities and router links.
I am not quite sure what this means but I am just this stuck-in-the-past Drupal cultist. As far as I understand these things, templating/formatting/visualization is performed by Twig -- something we build on? Is it possible that others create smaller blocks we just build on and that's not so bad? Perhaps re-join our community, explain what your vision is and enact change? We do change our ways. Drupal 8 is proof to this: we have changed our development strategies, many of our core values -- and, again 600 or so people showed up helping to further this vision whatever it is. If you have an even better vision for Drupal 9, please help, we will be glad to have an even better set of Legos.
This module enables you to control visibility of nivo_slider slides based on currently selected language.
A drop-down form element is added on each slide allowing the slide to be visible in one of the enabled languages as well as in all languages.
Currently a small patch needs to be applied to nivo slider for this module to work.
See Allow other modules to control visibility of slide for details
Copy nivo_slider_language folder to you modules directory.Apply patch to nivo_slider.
Copy nivo_slider_slider.patch into nivo_slider module folder
Use the following command to pathc nivo slider
patch includes/nivo_slider_slider.inc < nivo_slider_slider.patch
Enable nivo_slider_language module
Tested with nivo_slider 7.x-1.9
WebRTC (Web Real Time Communications) offers web application developers the ability to write rich, realtime multimedia applications (think video chat) on the web, without requiring plugins, downloads or installs.
It's purpose is to help build a strong RTC platform that works across multiple web browsers, across multiple platforms.
This module serves as an API for WebRTC applications.
Drupal is a good choice as a platform for building web apps thanks to devops advances that work around PHP's inadequacies as a modern programming language. Web app builders can benefit from the extensive ecosystem of Drupal modules, and the unrivalled community support, while still complying (almost) with the best practises for web apps set out in the Twelve Factor web app methodology.Introduction
Modern software is delivered as a service, via the web. Drupal is a CMS framework that can be used for building web apps, leveraging the extensive ecosystem of modules, and unrivaled community support. But, Drupal's disadvantage for app developers is that it is built on PHP, which lacks some of the features of modern programming languages, and some features of the Drupal architecture are hard to scale.
You might argue that Drupal is designed for building websites, but I don't really see an important distinction between web app or website. Perhaps for a website the focus is more on content, and for a web app the focus is functionality? Whatever your thoughts on this, I believe the twelve factors still apply, so if you're that way inclined just substitute the word site for the word app in the rest of this article.
The Twelve-Factor App is a methodology for building software-as-a-service apps that offers solid advice for both web app developers and op engineers that deploy or manage web apps. It is based on the authors' extensive experience is the development and deployment of apps via the Heroku platform.
I suggest reading the original manifesto as I am not going to go into detail, nor repeat or explain each of the twelve factors. In this article I will present the advances in Drupal devops and practises I have been using that move us towards complying with each of the twelve factors.
There is one single code repository per app. There is no code in this repo that is shared. Any shared code should be refactored into separate modules that can be linked in using the dependency management.
The implication of this is that your codebase repo does not include the Drupal codebase, it contains only the custom modules and themes that are used in your app.
The solution to this is to develop each app as a separate "install profile" which contains a make file for the dependencies, and any custom modules or themes required by the app. You can read more about this approach in the article Every Drupal Site is an Install Profile.
All dependencies of your app are defined in the Drush Make file. This includes the Drupal Core version that the site is built on, and any contrib modules, themes, and your own shared libraries references via Git.
Do not write any custom code (or use any contrib module) that relies implicitly on the existence of system-wide packages. All dependencies must be defined explicitly in the Drush Make file.
I would suggest refactoring code that relied implicitly on a system-wide package into an external resource (see IV Backing Services) and explicitly define the library/module to connect to it as a dependency. For example, rather than writing transcoding code that refers to ffmpeg, you should create a Backing Service for performing the transcode operations. This not only removes the implicit dependency, but has the added advantage of giving you extra scaling options.
To be truly twelve-factor you should use *dependency isolation* to ensure that no implicit dependencies "leak in" from the environment. Ruby and Python offer tools for this, but it's not usually possible in PHP. I don't have a fully worked out answer for this, but I am experimenting with defining system dependencies in a manifest, such as PHP version and required PHP extensions, and then via the magic of Nginx and PHPFPM allowing multiple process to use different PHP versions.
This factor relates to the the configuration that varies between deployments. This is a function of the environment in which the build runs so the config should come from the environment.
You can see an example of how to do this by looking at Aegir where Drupal uses a settings.php file in which the app config is set by reading from environment variables. These variables are set in the server vhost configuration for the deployment. This means you can have separate settings (e.g. Database settings) for dev, staging and production.
You should be able to set any Drupal variable by setting values in the $conf array in settings.php and reading the values from the environment. Some configuration is not stored in variables, but in (ctools) exportable objects in the database in this case you need some bridging code to set the configuration values in the exportable objects saved in code.
IV. Backing Services
Every Drupal app will have at least a database, an usually other "resources" such as queueing systems, email sending, and caches. Each distinct service is a resource, and there is a loose coupling. They can be swapped without code changes (just change settings).
V. Build, Release, Run
The Twelve-factor approach to deployment has three phases. Build, release and run. This translates to Drupal as follows:
1) Build - basically running Drush Make to download Drupal core and the dependencies as defined in your make file.
2) Release - a scripted mechanism for combining the "build" with the correct config for the deploy into a "release"
3) Run - the server processes are repointed to the new release. This could by as simple as changing the symlink for the Apache vhost record, but more likely involves a bit more complexity than this in the case you're running a more advanced nginx setup with multiple backend workers.
A "release" (i.e. combination of build and config) js never changed. You can keep a folder of "releases" named by either an incrementing ID, or timestamp, that means you can, for example, simplify the task of rolling back by repointing the vhost directory to a different release folder.
In a more complex hosting environment (like the one I've been implementing recently) you have multiple workers per site. This allows you to easily scale your app (up and down) as required. This allows for robustness as the site stays up if one worker dies, and performance as you balance the load across multiple workers.
To configure Drupal to comply with this factor you first have to be using a separate cache backend (e.g. Redis or Memcache) for sessions that can be shared by all the workers. You also need to think about storing the files folder where it can be shared among the workers. There are better solutions than NFS for this but perhaps that's a topic for later. Drupal has issues when working with NFS due the excessive use of include_once, although increasing the realpath cache helps. Thanks to Paul Reeves for the heads up on this one.
VII. Port binding
This factor is harder to implement in PHP as it's not usual to link in a web server library and run in "user space" as part of the process. The usual approach in PHP is to run as a module inside Apache or another web server daemon.
As a compromise I have started working with nginx and PHP-FPM. This setup gives me much more flexibility as to how to configure the system for multiple workers.
There is a project on GitHub called phttpd that promises to embed nginx and php-fpm into an isolated install to allow it to run in userspace. I've not looked at this yet, but it's could be interesting.
The twelve-factor app suggests architecting the app by assigning work to a process type, so having HTTP requests handled by a "web process" and having background tasks handled by "worker" processes.
I had success with this on the MTV project I was involved in recently, where certain administration or editorial tasks would be queued for execution rather than attempted straight away by using the Drupal Queue system. The queue items are sent to a queue (use an external system like beanstalkd) and then executed by separate background "queue-runner" processes.
By offloading tasks for asynchronous processing and using separate workers this gives two dimensions for how you can scale your app. You can scale up by adding more process of each type, or you can scale out by increasing workload diversity amongst more worker types. Having more worker types means you can scale up just the type of work that needs to be scaled, while staying lean on process that don't need it.
The twelve-factor app warns that processes should never daemonize themselves. If you've tried demonising PHP you'll know this can be tricky. To comply with this factor I think it is possible using something like supervisord and rely on the operating system process manager.
In the context of a Drupal app, this factor relates to having multiple workers that can be stopped and started on demand easily. This means keeping HTTP requests times short (helped by the actions taken in #VIII Concurrency).
In the case of background worker tasks (for example, queue runners) they should return any unprocessed items back to the Drupal Queue before shutting down. The web server container should stop accepting requests and shutdown when it's finished serving active requests - which should be quick because you've kept HTTP request times down, right?
X. Dev/prod parity
In order to comply with this factor we need to address the three areas where gaps between dev and production occur.
The time gap between when a developer commits a change and it is deployed should be as small as possible. If you've got the build/release/run process working well, and done as much as you can to remove other sources of friction in the process (e.g. by doing automated testing), then you're on your way with this one.
Reducing the personnel gap in Drupal means empowering developers with tools to get involved in deployments, and monitoring the app in production.
And to address the parity between the tools used in production and dev, basically, you need to be using Vagrant. If you're not then you're doing it wrong. It's not hard to find information on using Vagrant for Drupal development, but you will have to do some work to get the repeatable config that matches your production environment.
The Drupal watchdog is an essential debugging tool for any Drupal dev, plus in production it provides essential information on the operation, health, and performance of the app, and useful information for auditing.
There are flexible options for logging in Drupal 7 including the built in watchdog database log, but I would urge that you disable this and use a separate central repository for all logs. The monolog module for Drupal will allow you to send logs anywhere, for example a GrayLog2 server or Splunk, but a better solution is to log to the system logs using the syslog module in Drupal core and use a log router to centralise your logging.
XII. Admin processes
Drush provides a REPL shell for Drupal, and it's fairly straightforward to create drush commands for admin tasks and deploy them with your codebase as part of a release. Running database updates from modules and core should be done via the Drush update command against a deployed release using a worker, and against a test release first before altering the production DB.Conclusion
I hope in posting this comparison between advances in Drupal devops and the manifesto of the Twelve-Factor App, I might have at least raised awareness of the importance of devops in the Drupal community a little (for anyone who's not already aware!).
Drupal is awesomely powerful, it's fun to work with, and we can work around the disadvantages with great tools and practises. It is possible to build modern web apps and scale them in the cloud with maximum portability, maximum agility, and minimise the effects of software erosion.
Other popular posts...
Enables hiding and showing of descriptions and help texts on admin pages and forms
Note: Though we have tested it properly and carefully, we decided to release this module as Dev version. The reason is we believe later release would be much more improved, and your feedback on this module is extremely important to us.
TB Mega Menu allows you to create a mega menu with an innovative back-end user interface and synchronized with Drupal core menu.Overview
TB Mega Menu is developed following Drupal standards and best practices. Along with those must-have functions of a mega menu: display not just menu items but image, slideshow, video, article and customize any of them upon your favour...; below are most noticeable features that TB Mega Menu has:
- Synchronized with Drupal core menu, auto updated if there’s any change with Drupal core menu
- Bootstrap framework
- Responsive ready
- Multiple built-in styles designed with years of theming experience
- Visual and friendly user interface - what you see is what you get
- Custom style for each element of your mega menu, just input extra css class name
- Multiple built-in CSS3 animated effects
- Work independently with themes
- Add/Edit links to menu directly on TB Mega Menu UI and have Drupal core menu synchronized with TB Megamenu
- More menu styles coming
- Multi instances of one Drupal core menu for different styles/appearance
- Drag and drop feature
- Multi-blocks in one column
- Tuning usability, UI
- and based on community suggestion/feedback
Demo for TB Megamenu is built on TB Sirate - a fully responsive free theme.
To download the profile installation of TB Sirate with TB Mega Menu built-in, visit: http://drupal.org/project/tb_sirate_starter
This module is sponsored and developed by ThemeBrain. ThemeBrain is a professional Drupal theme providers with the only principle in mind - bringing Drupal to everyone. This module with innovative user interface is our effort to make it happen.
Temporary sandbox for the 7.x-1.x branch of new relic rpm.
A client recently asked if form labels can be displayed within a form element. More specifically, they wanted the field labels on the user login and registration page to be placed inside of the field or overlaid.
In this article I'll show you how to implement both modules and it should help you decide which one to use.
DrupalCon Portland was action-packed and exhausting for me. I contributed to fifteen architectural discussions with core developers, presented at three core conversations and participated in four others, and helped organize the biggest sprint in DrupalCon history (with something over 600 attendees, as well as fifty or so mentors in our bright green shirts). The part I'm most proud of, though, is our session early in the week on how to run a sprint. I think it's the most scalable thing I've done.Running coaches wanted! Contribution sprints and trainings
Two years ago, I kicked a pebble for a few miles, and I am amazed and humbled by the avalanche it started once I found a hill. :) Now it's your turn. Get rolling.
This module provides a way for roles with the appropriate permission to define Codes which users can then redeem to get userpoints.
Module depends on Drupal 7 version of http://drupal.org/project/userpoints
- Codes can be disabled and re-enabled.
- Codes currently can only be redeemed once by each user.
- A user can redeem a code by visiting the "userpoints/redeem-code" page.
A similar module exists http://drupal.org/project/commerce_coupon_userpoints but depends on Commerce.
Our use-case required Codes to be able to be given to users offline or by other means (with no Commerce system involved) and then required to be entered on the site by a link provided on pages across the site.
- Allow codes to expire.
- Allow admins to administer more aspects of Userpoints. (message etc)
- Allow codes to be used more than once.
It is that time of year. Everything is getting a little greener. The days are warming up. Drupalcamp Colorado is almost upon us! Last year the local community in Colorado decided to skip hosting what is one of the best Drupalcamps around. Why? Because we put on Drupalcon and there was a fair bit of fatigue following the event. Well, not so this year. The camp site is up and running! It is ready for registrations, session proposals, and payments!
Keynotes will be Jeff Eaton (Senior Drupal Architect at Lullabot) and Felicia Pride (Media Strategist, Content Producer, and Educator from Pride Collaborative).
Registration is a mere $10 - and is well worth it!drupaldrupalcamp colorado2013
Blogging tools and user experience have moved on quite fast in recent years. After spending a lot of time automating and markdownifying my coding and authoring workflows it suddenly felt kludgy to to fill small webforms and fighting with the markup filters to get something decently formatted and published. Consider also the time spent on spam fighting -- it was clearly the time to move on to a new platform.
In such cases usually two approaches are considered -- either moving to a hosted blogging platform such as Tumblr, Squarespace or Medium or using a Markdown-based static site generator. I picked the latter.
As a NodeJS convert I first started to research NodeJS-based generators. It turns out that most of them are still in too early stages (sure, there are some good tools out there such as Wintersmith, Blacksmith and Hexo). So I decided to fall back to the de facto standard generator Jekyll, learning from the experiences of the Drupal old-timers such as Walkah, UnConeD and Development Seed.The move
Although Jekyll offers Drupal migration, it did not worked well since I was using Texy markup that was hard to convert to Markdown directly. To overcome the problem, I wrote a Drush script what fetches content from Drupal database, converts it to HTML and saves it to Markdown files. Such "U-turn" approach allows to convert all the typographic details and also tooks care of image references (with direct database migration I would have left with raw image tokens).
I also had to rehash my Twitter strategy. Previously I fetched and displayed my tweets from Twitter RSS, but as Twitter has shut down its feeds, I wrote a NodeJS utility that fetches latest tweets using official API and saves them to Markdown files.
For image previews, I wrote a simple shell script and Jekyll plugin to convert full-resolution images to thumbnail images. Right now it's a simple image replacement, but could be turned into a responsive image approach if needed.
While at it I also refreshed the look of the site (hello Montserrat and Lato), made it responsive, updated CV, created proper podcast feed, replaced old VPS with the new shiny and got a real domain name.The wrapup
Drupal early years were heavily influenced by blog technology developments (remember Radio Userland and Drupal 4 Bloggers ?) and traces of that movement are still left in the Drupal codebase (xmlrpc.module, aggregator.module). But in year 2013 Drupal feels too heavyweight for light blogging, especially filling nerdy workflow needs.
It's not you, Drupal, it's me. I will move on and you better shape up for building those next-gen cloud publishing platforms!