Drupal

Autotrader CSV

New Drupal Modules - 26 February 2018 - 7:18am

This will be the new home of a simple module that creates CSVs on a regular basis in the format(s) required by Autotrader for uploading cars, automobiles, water crafts, and non-passenger vehicles (NPV) data to their FTP, ultimately for listing on the Autotrader website.

Categories: Drupal

Webform Ip Tracking

New Drupal Modules - 26 February 2018 - 7:15am

This module provides feature for sending location details such as city, state, country,etc through email, when a web form is submitted.

This is implemented using custom tokens.

Categories: Drupal

2bits: Optimizing Drupal Views: Query Time and Rendering Time

Planet Drupal - 26 February 2018 - 6:56am

A recent client performance assessment consulting project showed that on their site, the main page that logged in users would browse is slow. Tuning the server for memory and disk throughput helped somewhat, but did not fully eliminate the issue.

Looking at the page, it was a view, and the total time was around 2.75 seconds.

The main query was not efficient, with lots of left joins, and lots of filtering criteria:

SELECT node.nid AS nid,
... AS ...
... AS ...
'node' AS field_data_field_aaa_node_entity_type,
'node' AS field_data_field_bbb_node_entity_type,
'node' AS field_data_field_ccc_node_entity_type,
... AS ...
FROM node
INNER JOIN ... ON node.uid = ...
LEFT JOIN ... ON ... = ...  AND ... = ...
LEFT JOIN ... ON ... = ... AND (... = '12'
OR ... = '11'
OR ... = '15'
OR ... = '24')
WHERE (( (node.status = '1')
AND (node.type IN ('something'))
AND (... <> '0')
AND ((... <> '1') )
AND ((... = '4'))
AND (... IS NULL ) ))
ORDER  BY  ... DESC
LIMIT  51   OFFSET 0

That caused the first pass to sift through over 24,000 rows, while using both file sort and temporary tables. Both operations are disk intensive.

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: ...
   partitions: NULL
         type: range
possible_keys: PRIMARY,...
          key: rid
      key_len: 8
          ref: NULL
         rows: 24039
     filtered: 100.00
        Extra: Using where; Using index; Using temporary; Using filesort
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: ...
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,status
          key: PRIMARY
      key_len: 4
          ref: test43....
         rows: 1
     filtered: 50.00
        Extra: Using where
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: node
   partitions: NULL
         type: ref
possible_keys: uid,status,type,node_status_type
          key: uid
      key_len: 4
          ref: test43....
         rows: 5
     filtered: 12.18
        Extra: Using index condition; Using where
*************************** 4. row ***************************
           id: 1
  select_type: SIMPLE
        table: ...
   partitions: NULL
         type: ref
possible_keys: PRIMARY,...
          key: PRIMARY
      key_len: 4
          ref: test43....
         rows: 2
     filtered: 54.50
        Extra: Using where; Not exists; Using index

But here is the puzzle: this query took 250 to 450 milliseconds at most.

Where did the rest of the 2,750 milliseconds go?

To find out, we use xhprof, the profiler for PHP.

In the screenshot below, you can see that the total page processing time (Total Inc. Wall Time, top right) is 2,732 milliseconds.

Out of that, 85% is in database queries (252 total queries, totaling 2,326 milliseconds, Excl.Wall Time).

What are these queries?

They are queries to other tables in the database to retrieve fields for each row.

For example, if you have a product view, with certain criteria, the result still has to get the product name, its price, its image, ...etc.

All these queries add up, specially when you are loading 50 of them. The time needed to retrieve each field, and rendering it for output is multiplied by the number of rows retrieved.

So, how do you mitigate that overhead? There are several ways:

  • Reduce the number of rows returned by the view. For example, instead of 50, make it 25. That would half the number of queries (and processing) needed to produce the page.
  • If the query is the same for all logged in users, then enable views caching (under Advanced when you edit the view), and enable both Query Result and Rendered Output caching. Use time based caching, for as long as practical to your site (e.g. if you add products or change prices only once a day, then you can cache the results for 20 hours or more).
  • Use a fast caching layer, such as the memcache module, instead of the default database caching, which will be slow for a site with many logged in users.
  • Use View Lite Pager to eliminate COUNT queries from being performed.
  • Consider alternate approaches to views, such as Apache Solr Faceted Search, which has much better performance than MySQL based solutions, because they do build an efficient index.

By implementing all the above for the client in question, except the last one, we were able to bring the view page from 2,730 milliseconds, down to 700-800 milliseconds of response time.

Scalability was much better, with the same server could handle more logged in users.

Contents: Tags: 
Categories: Drupal

Twig Drupal Children Extension

New Drupal Modules - 26 February 2018 - 6:39am

Access children element from Twig.

Usage

Enable the module, then, in your Twig files, you'll be able to do:

{% for item in content.field_something|children %} {{ item.get('value').getValue() }} {% endfor %} Inspired from

#2776307

Categories: Drupal

Schemaless

New Drupal Modules - 26 February 2018 - 6:26am

Provides schemaless configuration management for developers

Categories: Drupal

Field Group as Class

New Drupal Modules - 26 February 2018 - 5:23am

Field Group as Class module is a formatter for field_group.

Description

This formatter get the node field value (list or list_string field_types)
to display it to the field_group class.

Categories: Drupal

DrupalEasy: DrupalEasy Podcast 206 - Heather Rodriguez - Previewing the "Being Human" track at DrupalCon Nashville

Planet Drupal - 26 February 2018 - 5:06am

Direct .mp3 file download.

Heather Rodriguez, (hrodrig), Solutions Analyst for Digital Services Georgia and track chair for DrupalCon Nashville's "Being Human" track joins Mike Anello to talk about some of the sessions accepted for the track, why the track is important, and heavy metal music.

Interview Drupal News DrupalEasy News Sponsors
  • Drupal Aid - Drupal support and maintenance services. Get unlimited support, monthly maintenance, and unlimited small jobs starting at $99/mo.
  • WebEnabled.com - devPanel.
Follow us on Twitter Subscribe

Subscribe to our podcast on iTunes, Google Play or Miro. Listen to our podcast on Stitcher.

If you'd like to leave us a voicemail, call 321-396-2340. Please keep in mind that we might play your voicemail during one of our future podcasts. Feel free to call in with suggestions, rants, questions, or corrections. If you'd rather just send us an email, please use our contact page.

Categories: Drupal

NTB import

New Drupal Modules - 26 February 2018 - 4:44am

Import articles from Norwegian News Agency (Norsk Telegrambyrå; abbreviated NTB, wikipedia page: https://en.wikipedia.org/wiki/Norwegian_News_Agency) to Drupal nodes.
You need an account in NTB in order to use this module.

Categories: Drupal

Commerce guest registration

New Drupal Modules - 26 February 2018 - 3:41am

The Commerce guest registration is an add-on module for the Commerce 2.x. The module won't have any configuration and permission implementations. This module help to create the user account in the system while they are placing orders as guest users. The user email id is considering the unique parameter to create the account.

Categories: Drupal

Tim Millwood: Deprecation fails when testing Drupal 8

Planet Drupal - 26 February 2018 - 12:57am
Deprecation fails when testing Drupal 8

With each release of Drupal 8 more and more things are being deprecated, which is awesome. It shows innovation, forward thinking, and a thought for backwards compatibility. However throwing notices or warnings when deprecated code is used can cause tests to fail. We already counter this a little by adding to phpunit.xml.dist in core.

To quote the Symfony documentation:

By using the weak_vendors value, deprecations that are triggered outside the vendors directory will make the test suite fail, while deprecations triggered from a library inside it will not, giving you the best of both worlds.

This shows that deprecations within Drupal will still cause test fails, to combat that simply update the SYMFONY_DEPRECATIONS_HELPER setting to "weak" or "disabled", which would ignore the deprecations or disable the deprecation helper.

However if you're testing with run-test.sh this will interfere with the settings in your phpunit.xml file and use always use weak_vendors unless you have the suppress-deprecations argument set. Therefore use run-tests.sh --suppress-deprecations and the SYMFONY_DEPRECATIONS_HELPER setting will be set to "disabled". See the simpletest_script_run_one_test() function in run-tests.sh for more context.

Hopefully this helps, we found this really useful when testing Drupal modules with Travis, where we needed tests passing for Drupal 8.4 and 8.5.

timmillwood Mon, 26/02/2018 - 08:57 Tags drupal planet drupal-planet drupal drupal8 drupal 8 testing drupal testing Add new comment
Categories: Drupal

Config Override Core Fields

New Drupal Modules - 25 February 2018 - 9:59pm

Provides hints to how form elements map to configuration objects.

This module does not expose any functionality on its own. You may have been asked by another module to install this module.

Recommended for use with Config Override Inspector.

Categories: Drupal

Form Mode Routing

New Drupal Modules - 25 February 2018 - 9:16pm

This module provides a way to give form modes a route and access check.

currently you need to define these in your own custom module routing file,
this module creates a new config entity that once you create a form mode you can associate that from mode with a path and user access roles.

Workflow
- 1) create a form mode
- 2) go to the content type and add the new from display
- 3) go to this module admin and give it a path and check which roles can access it.

Categories: Drupal

Posting my phone's battery status to my site

Dries Buytaert - 25 February 2018 - 5:56pm

Earlier this month, I uninstalled Facebook from my phone. I also made a commitment to own my own content and to share shorter notes on my site. Since then, I shared my POSSE plan and I already posted several shorter notes.

One thing that I like about Facebook and Twitter is how easy it is to share quick updates, especially from my phone. If I'm going to be serious about POSSE, having a good iOS application that removes friction from the publishing process is important.

I always wanted to learn some iOS development, so I decided to jump in and start building a basic iOS application to post notes and photos directly to my site. I've already made some progress; so far my iOS application shares the state of my phone battery at https://dri.es/status. This is what it looks like:

This was inspired by Aaron Parecki, who not only tracks his phone battery but also tracks his sleep, his health patterns and his diet. Talk about owning your own data and liking tacos!

Sharing the state of my phone's battery might sound silly but it's a first step towards being able to publish notes and photos from my phone. To post the state of my phone's battery on my Drupal site, my iOS application reads my battery status, wraps it in a JSON object and sends it to a new REST endpoint on my Drupal site. It took less than 100 lines of code to accomplish this. More importantly, it uses the same web services approach as posting notes and photos will.

In an unexpected turn of events (yes, unnecessary scope creep), I decided it would be neat to keep my status page up-to-date with real-time battery information. In good tradition, the scope creep ended up consuming most of my time. Sending periodic battery updates turned out to be difficult because when a user is not actively using an iOS application, iOS suspends the application. This means that the applications can't listen to battery events nor send data using web services. This makes sense: suspending the application helps improve battery life, and allows iOS to devote more system resources to the active application in the foreground.

The old Linux hacker in me wasn't going to be stopped though; I wanted my application to keep sending regular updates, even when it's not the active application. It turns out iOS makes a few exceptions and allows certain categories of applications – navigation, music and VOIP applications – to keep running in the background. For example, Waze continues to provide navigation instructions and Spotify will play music even when they are not the active application running in the foreground.

You guessed it; the solution was to turn my note-posting-turned-battery application into a navigation application. This requires the application to listen to location update events from my phone's GPS. Doing so prevents the application from being suspended. As a bonus, I can use my location information for future use cases such as geotagging posts.

This navigation hack works really well, but the bad news is that my application drains my phone battery rather quickly. The good news is that I can easily instruct Drupal to send me email notifications to remind me to charge my phone.

Scope creep aside, it's been fun to work with Drupal 8's built-in REST API and to learn some iOS application development. Now I can post my phone's battery on my site, posting notes or photos shouldn't be much more difficult. I'm guessing that a "minimum viable implementation" will require no more than another 100 lines of code and four hours of my time. My current thinking is to go ahead with that so I can start posting from my phone. That gives me more time to evaluate if I want to use the Micropub standard (probably) and the Indigenous iOS application (depends) instead of building and maintaining my own.

Categories: Drupal

Dries Buytaert: Posting my phone's battery status to my site

Planet Drupal - 25 February 2018 - 5:56pm

Earlier this month, I uninstalled Facebook from my phone. I also made a commitment to own my own content and to share shorter notes on my site. Since then, I shared my POSSE plan and I already posted several shorter notes.

One thing that I like about Facebook and Twitter is how easy it is to share quick updates, especially from my phone. If I'm going to be serious about POSSE, having a good iOS application that removes friction from the publishing process is important.

I always wanted to learn some iOS development, so I decided to jump in and start building a basic iOS application to post notes and photos directly to my site. I've already made some progress; so far my iOS application shares the state of my phone battery at https://dri.es/status. This is what it looks like:

This was inspired by Aaron Parecki, who not only tracks his phone battery but also tracks his sleep, his health patterns and his diet. Talk about owning your own data and liking tacos!

Sharing the state of my phone's battery might sound silly but it's a first step towards being able to publish notes and photos from my phone. To post the state of my phone's battery on my Drupal site, my iOS application reads my battery status, wraps it in a JSON object and sends it to a new REST endpoint on my Drupal site. It took less than 100 lines of code to accomplish this. More importantly, it uses the same web services approach as posting notes and photos will.

In an unexpected turn of events (yes, unnecessary scope creep), I decided it would be neat to keep my status page up-to-date with real-time battery information. In good tradition, the scope creep ended up consuming most of my time. Sending periodic battery updates turned out to be difficult because when a user is not actively using an iOS application, iOS suspends the application. This means that the applications can't listen to battery events nor send data using web services. This makes sense: suspending the application helps improve battery life, and allows iOS to devote more system resources to the active application in the foreground.

The old Linux hacker in me wasn't going to be stopped though; I wanted my application to keep sending regular updates, even when it's not the active application. It turns out iOS makes a few exceptions and allows certain categories of applications – navigation, music and VOIP applications – to keep running in the background. For example, Waze continues to provide navigation instructions and Spotify will play music even when they are not the active application running in the foreground.

You guessed it; the solution was to turn my note-posting-turned-battery application into a navigation application. This requires the application to listen to location update events from my phone's GPS. Doing so prevents the application from being suspended. As a bonus, I can use my location information for future use cases such as geotagging posts.

This navigation hack works really well, but the bad news is that my application drains my phone battery rather quickly. The good news is that I can easily instruct Drupal to send me email notifications to remind me to charge my phone.

Scope creep aside, it's been fun to work with Drupal 8's built-in REST API and to learn some iOS application development. Now I can post my phone's battery on my site, posting notes or photos shouldn't be much more difficult. I'm guessing that a "minimum viable implementation" will require no more than another 100 lines of code and four hours of my time. My current thinking is to go ahead with that so I can start posting from my phone. That gives me more time to evaluate if I want to use the Micropub standard (probably) and the Indigenous iOS application (depends) instead of building and maintaining my own.

Categories: Drupal

aleksip.net: Improving Drupal’s minor version upgrade experience

Planet Drupal - 25 February 2018 - 10:43am
Twice a year the new Drupal upgrade model makes me both excited about new features and worried about the possibility that the sites I maintain will break. I think there are some ways how the upgrade experience could be made better.
Categories: Drupal

Spinning Code: Using Composer for Drupal Modules and Private Bitbucket Repos

Planet Drupal - 25 February 2018 - 10:07am

The next installment in my ongoing set of posts to create a public record for things I couldn’t learn in one Google search is a process for using composer to track a Drupal 8 module in a private repository.

It’s pretty common for Drupal agencies to have a small collection of modules they have built in-house and use on nearly all client site, or to have a module built for one client that has many sites. We are all becoming adept at managing our projects with Composer, but the vast majority of resources are focused on managing publicly available code via packagist. But there are times the kinds of internally shared modules cannot be made fully public (for example they may contain IP belonging to the client). We have one such client that needs a module deployed to dozens of sites, and so I sat down a few weeks ago to figure out a solution.

We use Bitbucket for our private repositories, I am sure there is a similar solution using GitHub, but I haven’t worked out its details.

  1. Create private repo for module on Bitbucket.
  2. Clone that repo locally, and structure it to match Drupal.org’s conventions (this probably isn’t required, but should allow your module to blend into the rest of the project more smoothly).
  3. Create Oauth token for your account in Bitbucket. Make sure to include a dumy callback URL; you can literally use http://www.example.com. If you see references to auth.json, don’t worry about that part yet.
  4. Add a composer.json file to the module’s repo (it only requires module name, type, and the branch alias, but it’s good to include the rest): { "name": "client/client_private_module", "type": "drupal-module", "description": "A very important module to our very important client.", "keywords": ["Drupal"], "homepage": "https://www.bitbucket.org/great_agency/client_private_module", "license": "proprietary", "minimum-stability": "dev", "extra": { "branch-alias": { "8.x-1.x": "1.x-dev" } }, "require": { "drupal/diff": "~1.0", } },
  5. Add reference to project composer.json repositories section: { "type": "package", "package": { "name": "client/client_private_module", "version": "dev", "type": "drupal-module", "dist": { "url": "https://www.bitbucket.org/great_agency/client_private_module/get/8.x-1.x.zip", "type": "zip" } } }
  6. Now just run composer require client/client_private_module, and provide the oauth creds from step 3 (note: the first time you do this composer will create the needed ~/.composer/auth.json)
Categories: Drupal

Pages

Subscribe to As If Productions aggregator - Drupal