Drupal

Eelke Blok: Fighting content spam on Drupal with Akismet

Planet Drupal - 15 August 2019 - 7:15am

Once, the Drupal community had Mollom, and everything was good. It was a web service that would let you use an API to scan comments and other user-submitted content and it would let your site know whether it thought it was spam, or not, so it could safely publish the content. Or not. It was created by our very own Dries Buytaert and obviously had a Drupal module. It was the service of choice for Drupal sites struggling with comment spam. Unfortunately, Mollom no longer exists. But there is an alternative, from the WordPress world: Akismet.

Categories: Drupal

Paragraph Handler

New Drupal Modules - 15 August 2019 - 5:16am

Paragraph handler allows you to provide "Paragraph Handler" classes that can handle preprocessing and altering the render arrays of Paragraph entities when they are displayed. See example below:

Categories: Drupal

OSTraining: How to Group Entity Fields in Tabs in Drupal 8

Planet Drupal - 14 August 2019 - 10:46pm

Extensive nodes (or other types of entities) with many text fields, such as biographies, often remain unread because of the huge (and discouraging) amount of text.

The Drupal 8 "Field Group" module allows you to group fields and to present them in containers like vertical or horizontal tabs, accordions or just plain wrappers. It lets you group fields in the frontend of your site, and in the backend as well.

Keep reading to learn how to use this module!

Categories: Drupal

OSTraining: How to Group Entity Fields in Tabs in Drupal 8

Planet Drupal - 14 August 2019 - 10:46pm

Extensive nodes (or other types of entities) with many text fields, such as biographies, often remain unread because of the huge (and discouraging) amount of text.

The Drupal 8 "Field Group" module allows you to group fields and to present them in containers like vertical or horizontal tabs, accordions or just plain

wrappers. Field Group lets you group fields in the Frontend of your site, and in the Backend as well.

Keep reading to learn how to use this module!

Categories: Drupal

Twig Attributes

New Drupal Modules - 14 August 2019 - 7:25pm

Twig Attributes allows developers to set HTML attributes (such as classes or an id) in a parent Twig template to elements in a child template, eliminating the need to create a template override or implement a preprocess hook just to add an attribute. It is particularly useful when working with fields that do not by default support attributes on certain HTML elements, such as links. Twig Attributes includes specific support for a number of templates provided by Drupal core, including ones used for image fields, responsive image fields, and links.

Categories: Drupal

Migrate Git

New Drupal Modules - 14 August 2019 - 3:37pm
Introduction

Provides:

Categories: Drupal

MPay

New Drupal Modules - 14 August 2019 - 12:03pm

MPay is an all-in-one payment processing and e-commerce solution for Minter Blockchain Network.

For more information visit our website: mpay.ms

Categories: Drupal

Agaric Collective: Migrating addresses into Drupal

Planet Drupal - 14 August 2019 - 11:55am

Today we will learn how to migrate addresses into Drupal. We are going to use the field provided by the Address module which depends on the third-party library commerceguys/addressing. When migrating addresses you need to be careful with the data that Drupal expects. The address components can change per country. The way to store those components also varies per country. These and other important consideration will be explained. Let’s get started.

Getting the code

You can get the full code example at https://github.com/dinarcon/ud_migrations The module to enable is UD address whose machine name is ud_migrations_address. The migration to execute is udm_address. Notice that this migration writes to a content type called UD Address and one field: field_ud_address. This content type and field will be created when the module is installed. They will also be removed when the module is uninstalled. The demo module itself depends on the following modules: address and migrate.

Note: Configuration placed in a module’s config/install directory will be copied to Drupal’s active configuration. And if those files have a dependencies/enforced/module key, the configuration will be removed when the listed modules are uninstalled. That is how the content type and fields are automatically created and deleted.

The recommended way to install the Address module is using composer: composer require drupal/address. This will grab the Drupal module and the commerceguys/addressing library that it depends on. If your Drupal site is not composer-based, an alternative is to use the Ludwig module. Read this article if you want to learn more about this option. In the example, it is assumed that the module and its dependency were obtained via composer. Also, keep an eye on the Composer Support in Core Initiative as they make progress.

Source and destination sections

The example will migrate three addresses from the following countries: Nicaragua, Germany, and the United States of America (USA). This makes it possible to show how different countries expect different address data. As usual, for any migration you need to understand the source. The following code snippet shows how the source and destination sections are configured:

source: plugin: embedded_data data_rows: - unique_id: 1 first_name: 'Michele' last_name: 'Metts' company: 'Agaric LLC' city: 'Boston' state: 'MA' zip: '02111' country: 'US' - unique_id: 2 first_name: 'Stefan' last_name: 'Freudenberg' company: 'Agaric GmbH' city: 'Hamburg' state: '' zip: '21073' country: 'DE' - unique_id: 3 first_name: 'Benjamin' last_name: 'Melançon' company: 'Agaric SA' city: 'Managua' state: 'Managua' zip: '' country: 'NI' ids: unique_id: type: integer destination: plugin: 'entity:node' default_bundle: ud_address

Note that not every address component is set for all addresses. For example, the Nicaraguan address does not contain a ZIP code. And the German address does not contain a state. Also, the Nicaraguan state is fully spelled out: Managua. On the contrary, the USA state is a two letter abbreviation: MA for Massachusetts. One more thing that might not be apparent is that the USA ZIP code belongs to the state of Massachusetts. All of this is important because the module does validation of addresses. The destination is the custom ud_address content type created by the module.

Available subfields

The Address field has 13 subfields available. They can be found in the schema() method of the AddresItem class. Fields are not required to have a one-to-one mapping between their schema and the form widgets used for entering content. This is particularly true for addresses because input elements, labels, and validations change dynamically based on the selected country. The following is a reference list of all subfields for addresses:

  1. langcode for language code.
  2. country_code for country.
  3. administrative_area for administrative area (e.g., state or province).
  4. locality for locality (e.g. city).
  5. dependent_locality for dependent locality (e.g. neighbourhood).
  6. postal_code for postal or ZIP code.
  7. sorting_code for sorting code.
  8. address_line1 for address line 1.
  9. address_line2 for address line 2.
  10. organization for company.
  11. given_name for first name.
  12. additional_name for middle name.
  13. family_name for last name:

Properly describing an address is not trivial. For example, there are discussions to add a third address line component. Check this issue if you need this functionality or would like to participate in the discussion.

Address subfield mappings

In the example, only 9 out of the 13 subfields will be mapped. The following code snippet shows how to do the processing of the address field:

field_ud_address/given_name: first_name field_ud_address/family_name: last_name field_ud_address/organization: company field_ud_address/address_line1: plugin: default_value default_value: 'It is a secret ;)' field_ud_address/address_line2: plugin: default_value default_value: 'Do not tell anyone :)' field_ud_address/locality: city field_ud_address/administrative_area: state field_ud_address/postal_code: zip field_ud_address/country_code: country

The mapping is relatively simple. You specify a value for each subfield. The tricky part is to know the name of the subfield and the value to store in it. The format for an address component can change among countries. The easiest way to see what components are expected for each country is to create a node for a content type that has an address field. With this example, you can go to /node/add/ud_address and try it yourself. For simplicity sake, let’s consider only 3 countries:

  • For USA, city, state, and ZIP code are all required. And for state, you have a specific list form which you need to select from.
  • For Germany, the company is moved above first and last name. The ZIP code label changes to Postal code and it is required. The city is also required. It is not possible to set a state.
  • For Nicaragua, the Postal code is optional. The State label changes to Department. It is required and offers a predefined list to choose from. The city is also required.

Pay very close attention. The available subfields will depend on the country. Also, the form labels change per country or language settings. They do not necessarily match the subfield names. Moreover, the values that you see on the screen might not match what is stored in the database. For example, a Nicaraguan address will store the full department name like Managua. On the other hand, a USA address will only store a two-letter code for the state like MA for Massachusetts.

Something else that is not apparent even from the user interface is data validation. For example, let’s consider that you have a USA address and select Massachusetts as the state. Entering the ZIP code 55111 will produce the following error: Zip code field is not in the right format. At first glance, the format is correct, a five-digits code. The real problem is that the Address module is validating if that ZIP code is valid for the selected state. It is not valid for Massachusetts. 55111 is a ZIP code for the state of Minnesota which makes the validation fail. Unfortunately, the error message does not indicate that. Nine-digits ZIP codes are accepted as long as they belong to the state that is selected.

Finding expected values

Values for the same subfield can vary per country. How can you find out which value to use? There are a few ways, but they all require varying levels of technical knowledge or access to resources:

  • You can inspect the source code of the address field widget. When the country and state components are rendered as select input fields (dropdowns), you can have a look at the value attribute for the option that you want to select. This will contain the two-letter code for countries, the two-letter abbreviations for USA states, and the fully spelled string for Nicaraguan departments.
  • You can use the Devel module. Create a node containing an address. Then use the devel tab of the node to inspect how the values are stored. It is not recommended to have the devel module in a production site. In fact, do not deploy the code even if the module is not enabled. This approach should only be used in a local development environment. Make sure no module or configuration is committed to the repo nor deployed.
  • You can inspect the database. Look for the records in a table named node__field_[field_machine_name], if migrating nodes. First create some example nodes via the user interface and then query the table. You will see how Drupal stores the values in the database.

If you know a better way, please share it in the comments.

The commerceguys addressing library

With version 8 came many changes in the way Drupal is developed. Now there is an intentional effort to integrate with the greater PHP ecosystem. This involves using already existing libraries and frameworks, like Symfony. But also, making code written for Drupal available as external libraries that could be used by other projects. commerceguys\addressing is one example of a library that was made available as an external library. That being said, the Address module also makes use of it.

Explaining how the library works or where its fetches its database is beyond the scope of this article. Refer to the library documentation for more details on the topic. We are only going to point out some things that are relevant for the migration. For example, the ZIP code validation happens at the validatePostalCode() method of the AddressFormatConstraintValidator class. There is no need to know this for a migration project. But the key thing to remember is that the migration can be affected by third-party libraries outside of Drupal core or contributed modules. Another example, is the value for the state subfield. Address module expects a subdivision as listed in one of the files in the resources/subdivision directory.

Does the validation really affect the migration? We have already mentioned that the Migrate API bypasses Form API validations. And that is true for address fields as well. You can migrate a USA address with state Florida and ZIP code 55111. Both are invalid because you need to use the two-letter state code FL and use a valid ZIP code within the state. Notwithstanding, the migration will not fail in this case. In fact, if you visit the migrated node you will see that Drupal happily shows the address with the data that you entered. The problems arrives when you need to use the address. If you try to edit the node you will see that the state will not be preselected. And if you try to save the node after selecting Florida you will get the validation error for the ZIP code.

This validation issues might be hard to track because no error will be thrown by the migration. The recommendation is to migrate a sample combination of countries and address components. Then, manually check if editing a node shows the migrated data for all the subfields. Also check that the address passes Form API validations upon saving. This manual testing can save you a lot of time and money down the road. After all, if you have an ecommerce site, you do not want to be shipping your products to wrong or invalid addresses. ;-)

Technical note: The commerceguys/addressing library actually follows ISO standards. Particularly, ISO 3166 for country and state codes. It also uses CLDR and Google's address data. The dataset is stored as part of the library’s code in JSON format.

Migrating countries and zone fields

The Address module offer two more fields types: Country and Zone. Both have only one subfield value which is selected by default. For country, you store the two-letter country code. For zone, you store a serialized version of a Zone object.

What did you learn in today’s blog post? Have you migrated address before? Did you know the full list of subcomponents available? Did you know that data expectations change per country? Please share your answers in the comments. Also, I would be grateful if you shared this blog post with others.

This blog post series, cross-posted at UnderstandDrupal.com as well as here on Agaric.coop, is made possible thanks to these generous sponsors. Contact Understand Drupal if your organization would like to support this documentation project, whether it is the migration series or other topics.

Read more and discuss at agaric.coop.

Categories: Drupal

Webform Remote Handlers

New Drupal Modules - 14 August 2019 - 7:41am

This module allows to send Webform submission results to third party through Web Services (REST/SOAP), implementing two Webform Handler plugins (REST and SOAP). These plugins allow to define endpoint configurations (including endpoint URL and HTTP method), JSON payload (with tokens), optional base64 configurations and type of authentication (basic or oauth).

Categories: Drupal

Drudesk: Drupal 8 for marketers: key benefits & useful modules

Planet Drupal - 14 August 2019 - 6:51am

It‘s easy and enjoyable to create marketing campaigns, drive leads, and tell your brand’s story to the world if your website is on the right CMS. Drupal 8’s benefits will definitely impress any marketer. So let’s take a closer look at the greatness of Drupal 8 for marketers, see what makes it so valuable, and name a few useful modules.

Categories: Drupal

Content Sanitizer

New Drupal Modules - 14 August 2019 - 6:47am
Categories: Drupal

Entity Unified access

New Drupal Modules - 14 August 2019 - 6:39am

This module is for developers only and unifying hook_entity_access and hook_entity_query.

Categories: Drupal

Active Form

New Drupal Modules - 14 August 2019 - 5:30am
Categories: Drupal

Merge Duplicate Files (MDF)

New Drupal Modules - 14 August 2019 - 3:53am

The Merge Duplicate Files (MDF) module extend filehash module functionally and allow to reuse uploaded files automatically if user upload the existed file. Module compare files by hash and prevent duplicate files, replace it with exists file.

Categories: Drupal

Devportal FAQ

New Drupal Modules - 14 August 2019 - 2:32am
Categories: Drupal

Devportal API Reference

New Drupal Modules - 14 August 2019 - 2:29am
Categories: Drupal

Devportal API Bundle

New Drupal Modules - 14 August 2019 - 2:25am
Categories: Drupal

Built By

New Drupal Modules - 14 August 2019 - 1:45am

Built By provides a simple agency copyright block with a configurable message and link.

For SEO purposes, the link also needs to have 'rel=follow' attribute on the home page, and 'rel=nofollow' on all non-home pages, and the logic and caching for this is included in the module.

The need for this module arose from having to create a reusable solution for many sites.

Categories: Drupal

Dropsolid: Debugging segmentation faults in Drupal

Planet Drupal - 14 August 2019 - 1:15am
14 Aug

Segfaults in Drupal are not a common occurrence - but when they do pop up, they can pose some tricky challenges... Our DevOps Engineer Mattias Michaux sheds some light on how to debug segmentation faults.
 

As a Drupal web developer, it can be frightening to encounter a so-called segmentation fault or segfault. This is a type of failure raised by hardware with memory protection, notifying you that the software has attempted to access a restricted area of memory. This kind of fault is common in languages with low-level memory management, such as C. Coding in a scripting language like PHP usually implies that you will be spared from segfaults, but on rare occasions these kind of errors can still pop up. And if they do, they tend to leave no stack trace or meaningful error clue in the PHP log. This is because segfault errors originate in the layers located below the PHP engine. Let me talk you through a recent consulting project that involved a curious segmentation fault.
 

Segfault case study

Not so long ago, one of our clients started experiencing seemingly random ‘503 service unavailable’ errors, at random times on random pages. That’s plenty of randomness, without much of a clue to start from.
 

The segfaults had started occurring after an update from PHP 5.6 to PHP 7.2 and the switch from mod_php - PHP is executed as a module within the Apache process -  to PHP-FPM. Just to clarify: PHP runs as a standalone service that Apache connects to.
 

We tried to reproduce the errors on a copy of the site and infrastructure of the project. The segfault was only reproducible when we ran a scraping tool on the site. There was no apparent connection between the pages it occurred on, and the error came up with less than 1% of the total requests. Looking at the logs, there seemed to be no direct cause for this error.

 

 

[proxy_fcgi:error] [pid 10272] AH01067: Failed to read FastCGI header [proxy_fcgi:error] [pid 10272] (104)Connection reset by peer: AH01075: Error dispatching request to ****

Example of the unhelpful Apache error message

 

 

The message above only tells us that something went wrong, but it provides no indication as to what the cause might be. Next, the PHP-FPM log indicated a segmentation fault:

WARNING: [pool web] child 3824 exited on signal 11 (SIGSEGV) after 3.353763 seconds from start

 

The syslog entry didn’t turn out to be very helpful either:

kernel: [4734894.041892] traps: php-fpm7.2[3760] general protection ip:555ce4cb6342 sp:7ffccab418c8 error:0 kernel: [4734894.041897] in php-fpm7.2[555ce4a51000+411000]

 

To find out what the root cause might be, we needed to recompile PHP with debugging enabled. This allowed us to produce core dumps, which contain a full backtrace. A backtrace is a summary of how the program got to a particular point. It displays one line per frame for many frames, starting with the frame that is currently being executed. I suggest finding a guide like this one on how to compile your php with debugging enabled.
 

After compiling PHP, we copied the php.ini and pool config from the original PHP version to the freshly compiled one. Next, we altered the config, so now the PHP-FPM pool used a separate name and socket path, and we changed that socket in the vhost. After starting the new PHP-FPM instance and running the crawler to reproduce the issue, we quickly saw the same 503 errors showing up, but this time with a core dump. We started comparing the dumps and tried to find a pattern. Fortunately, most cases pointed to the same thing: the unserialize() function of large objects. This made us look for specific PHP bugs that involved unserializing or memory allocation. We found an interesting one - in fact, someone had encountered almost the same issue before.
 

We decided to try the same approach by disabling garbage collection in the settings file:

ini_set('zend.enable_gc', 0);

 

We reran the scraper for multiple hours and no issues occurred. There are other documented cases where PHP’s garbage collection interferes with Drupal’s processing of huge amounts of data and objects.  Because disabling the garbage collection globally could have negative effects on the resource consumption of the live site, we also looked for a way to only disable garbage collection partially. This is indeed possible by patching  includes/cache.inc so _cache_get_object doesn’t run gc while fetching data.
 

This puzzling problem took several hours and two people to solve, but in the end we managed to diagnose correctly and we implemented a solid solution. An interesting case that led us to explore unusual parts of Drupal - and possibly something to bear in mind next time your Drupal environment produces a similar type of error.
 

gdb /opt/php/7.2/sbin/php-fpm /tmp/coredump-php-fpm.28651 -ex "source /opt/php-src/php7.2-build/php-7.2.17/.gdbinit" -ex "zbacktrace" --batch | grep "\[0x" [0x7f2c2b4220e0] unserialize("O:4:"view":49:{s:8:"db_table";s:10:"views_view";s:10:"base_table";s:18:"commerce_line_item";s:10:"base_field";s:3:"nid";s:4:"name";s:25:"commerce_reports_products";s:3:"vid";s:0:"";s:11:"description";s:0:"";s:3:"tag";s:16:"commerce_reports";s:10:"human_nam...") [internal function] [0x7f2c2b421f40] DrupalDatabaseCache->prepareItem(object[0x7f2c2b421f90]) /home/project/www/includes/cache.inc:520 [0x7f2c2b421d80] DrupalDatabaseCache->getMultiple(reference) /home/project/www/includes/cache.inc:433 [0x7f2c2b421cf0] cache_get_multiple(reference, "cache_views") /home/project/www/includes/cache.inc:113 [0x7f2c2b421a70] _ctools_export_get_defaults_from_cache("views_view", array(24)[0x7f2c2b421ad0]) /home/project/www/sites/all/modules/contrib/ctools/includes/export.inc:746 [0x7f2c2b421390] _ctools_export_get_defaults("views_view", array(24)[0x7f2c2b4213f0]) /home/project/www/sites/all/modules/contrib/ctools/includes/export.inc:649 [0x7f2c2b4211a0] _ctools_export_get_some_defaults("views_view", array(24)[0x7f2c2b421200], array(1)[0x7f2c2b421210]) /home/project/www/sites/all/modules/contrib/ctools/includes/export.inc:783 [0x7f2c2b420210] ctools_export_load_object("views_view", "names", array(1)[0x7f2c2b420280]) /home/project/www/sites/all/modules/contrib/ctools/includes/export.inc:493 [0x7f2c2b420080] ctools_export_crud_load("views_view", "search_results") /home/project/www/sites/all/modules/contrib/ctools/includes/export.inc:81 [0x7f2c2b41ff50] views_get_view("search_results") /home/project/www/sites/all/modules/contrib/views/views.module:1683 [0x7f2c2b41fb50] views_block_view("-exp-search_results-page") /home/project/www/sites/all/modules/contrib/views/views.module:772 [0x7f2c2b41fa60] module_invoke("views", "block_view", array(1)[0x7f2c2b41fad0]) /home/project/www/includes/module.inc:934 [0x7f2c2b41f410] _block_render_blocks(array(0)[0x7f2c2b41f460]) /home/project/www/modules/block/block.module:911 [0x7f2c2b41f2d0] block_list("branding") /home/project/www/modules/block/block.module:690 [0x7f2c2b41f200] block_get_blocks_by_region("branding") /home/project/www/modules/block/block.module:319 [0x7f2c2b41eee0] block_page_build(reference) /home/project/www/modules/block/block.module:270 [0x7f2c2b41ecf0] drupal_render_page(reference) /home/project/www/includes/common.inc:5914 [0x7f2c2b41e570] drupal_deliver_html_page(array(1)[0x7f2c2b41e5c0]) /home/project/www/includes/common.inc:2761 [0x7f2c2b41e3d0] drupal_deliver_page(array(1)[0x7f2c2b41e420], "") /home/project/www/includes/common.inc:2634 [0x7f2c2b41e100] menu_execute_active_handler() /home/project/www/includes/menu.inc:542 [0x7f2c2b41e030] (main) /home/project/www/index.php:21

 

dropsolid8
Categories: Drupal

Web Wash: Use Taxonomy Terms as Webform Options in Drupal 8

Planet Drupal - 13 August 2019 - 6:30pm

Webform has a pretty robust system for managing lists of options. When you create a select box, you can define its options just for that element, or use a predefined list. If you go to Structure, Webforms, Configurations and click on Options, you can see all the predefined options and you can create your own.

If you want to learn how to create your own predefined options check out our tutorial; How to Use Webform Predefined Options in Drupal 8.

One thing to be aware of is that all of these options are stored as config files, which makes perfect sense, it is configuration.

But what if you want editors to manage the options?

Depending on how you deploy Drupal sites if you change an option only on the production site, your change will be overridden the next time you deploy to production because you import all new configuration changes.

To work around this, you could look at using Webform Config Ignore.

Another way of managing options is by using the Taxonomy system. An editor would simply manage all the terms from the Taxonomy page and nothing will be stored in config files.

In this tutorial, you’ll learn how to create a select element which uses a taxonomy vocabulary instead of the standard options.

Categories: Drupal

Pages

Subscribe to As If Productions aggregator - Drupal