Newsfeeds

Twig Tools

New Drupal Modules - 26 February 2019 - 8:33pm

The Twig Tools module provides additional Twig filters and functions.

Current Filters:
clean_class_array
Sanitizes all strings in an array for use as valid class names.
array_unique
Filters all non-unique values from an array.
remove_empty
Removes all falsy values from an array.
Categories: Drupal

Promet Source: Web Accessibility Overlays: True Fix or False Pretense?

Planet Drupal - 26 February 2019 - 7:50pm
With the increased number of accessibility lawsuits for inaccessible websites, it's no wonder that offers for quick fixes are a hot commodity. Unfortunately, the saying, “You get what you pay for” may not apply to accessibility overlay solutions. So, what do you do? First, let’s take look at how quick-fix web accessibility overlay solutions actually work.
Categories: Drupal

Jacob Rockowitz: Open email asking organizations to back the Webform module and Drupal-related Open Collectives

Planet Drupal - 26 February 2019 - 1:03pm

Following up from my previous blog post, "Asking organizations to back a Drupal-related Open Collective."

Below is the email I am sending to organizations within the Drupal community asking them to become a $10 monthly backer of the Webform module and Drupal-related Open Collectives. This email will be sent to people I have spoken to directly at Drupal camps and meetups, as well as organizations listed on the Drupal marketplace.

Read More
Categories: Drupal

Don't Miss: Most players won't finish your game - and that's not a bad thing!

Social/Online Games - Gamasutra - 26 February 2019 - 12:18pm

Most players won't play to the end of your game. That's not a tragedy -- that's a feature of video games' design landscape. ...

Categories: Game Theory & Design

The duo behind the Running Man challenge are suing Epic over a Fortnite dance

Social/Online Games - Gamasutra - 26 February 2019 - 11:44am

Epic Games has another lawsuit on its hands over a dance emote featured in Fortnite, this time over The Running Man. ...

Categories: Game Theory & Design

Acquia DAM Asset Importer

New Drupal Modules - 26 February 2019 - 11:28am

This module allows you to bulk import the contents of folders from Acquia DAM into your Drupal site. This extends the functionality provided by Media: Acquia DAM, allowing you to have assets in Drupal without having to first associate them to another Drupal entity.

Categories: Drupal

OPTASY: 3 Types of Content Management Systems to Consider in 2019: Traditional CMS vs Headless CMS vs Static Site Generators

Planet Drupal - 26 February 2019 - 10:37am
3 Types of Content Management Systems to Consider in 2019: Traditional CMS vs Headless CMS vs Static Site Generators radu.simileanu Tue, 02/26/2019 - 18:37

Kind of stuck here? One one hand, you have all these software development technologies that are gaining momentum these days —  API, serverless computing, microservices — while on the other hand, you have a bulky "wishlist" of functionalities and expectations from your future CMS.  So, what are those types of content management systems that are and will be relevant many years to come and that cover all your feature requirements?

And your list of expectations for this "ideal" enterprise-ready content infrastructure sure isn't a short one:
 

Categories: Drupal

Webform User Registration

New Drupal Modules - 26 February 2019 - 9:40am

Description
-----------
This module adds a new webform handler to start a user session upon form submit.
The configuration is almost identical to the core user register form and also allows
for updating existing user values.

GitHub
-----------
This project is also maintained on GitHub

Installation
------------
To install this module, do the following:

Categories: Drupal

Drupal blog: Regarding critical security patches, we hear your pain.

Planet Drupal - 26 February 2019 - 9:37am

This post was created jointly by Michael Hess of the Security Working Group, and Tim Lehnen, Executive Director of the Drupal Association.

Last year, with the security release of SA-CORE-2018-002, the most significant security vulnerability since 2014, we heard the pain of site owners and development teams around the world staying up at all hours waiting for the complex security release process to complete and the patch to drop. We heard the pain of agencies and end-user organizations required to put teams on late shifts and overtime. We heard from some users who simply couldn't respond to patch their sites on the day of release, because of lack of resources or entrenched change management policies.

We've heard calls from the community for rotating the timezones for security advisories from release to release, or for having more on-call support from security team members across the globe, or simply for a longer horizon between the release of PSA and SA.

Yet at the same time, we're cognizant that these solutions would put increased burden on a security team composed of dedicated volunteers and contributors. There are a number of generous organizations who sponsor many of the members of the security team, but relying on their altruism alone is not a sustainable long-term solution—especially if we consider expanding the role of the security team to address the larger pain points above.

Last week, with the release of SA-CORE-2019-003, we heard these concerns for site owners and the sustainability of the security team echoed again.

The Security Team and the Drupal Association have been developing solutions for this issue for well over a year.

The goals are simple:

  • Provide a new service to the Drupal community, from small site owners to enterprise-scale end users, to protect their sites in the gap from security release to the time it takes them to patch.
  • Create a new model for sustainability for the Security Team, generating funding that 1) covers the operating costs of the program 2) can support security team operations and 3) can support additional Drupal Association programs.

Although the execution will take care and careful partnership, we are happy to announce that we've found a solution.

We're tentatively calling this: Drupal Steward. It is a service to be provided by the Drupal Association, the Security team, and carefully vetted hosting partners.

Drupal Steward will offer sites a form of mitigation through the implementation of web application firewall rules to prevent mass exploitation of some highly critical vulnerabilities (not all highly critical vulnerabilities can be protected in this fashion, but a good many can be - this method would have worked for SA-CORE-2018-002 for example).

It will come in three versions:

  • Community version - for small sites, low-budget organizations, and non-profits, we will offer a community tier, sold directly by the DA. This will be effectively at cost.
  • Self hosted version - for sites that are too large for the community tier but not hosted by our vendor partners.
  • Partner version - For sites that are hosted on vetted Drupal platform providers, who have demonstrated a commitment of contribution to the project in general and the security team in particular, protection will be available directly through these partners.
Next Steps

The Drupal Association and Security Team are excited to bring this opportunity to the Drupal Community.

We believe that the program outlined above will make this additional peace of mind accessible to the broadest base of our community possible, given the inherent costs, and are hopeful that success will only continue to strengthen Drupal's reputation both for one of the most robust security teams in open source, and for innovating to find new ways to fund the efforts of open source contributors.

We will announce more details of the program over the coming weeks and months as we get it up and running.

If you are a hosting company and are interested in providing this service to your customers, please reach out to us at drupalsteward@drupal.org.

Please also join us at DrupalCon for any questions about this program.

If you are a site owner and have questions you can join us in slack #drupalsteward.

For press inquiries, please contact us at: security-press@drupal.org

Categories: Drupal

N1ED

New Drupal Modules - 26 February 2019 - 5:19am

N1ED - a powerful WYSIWYG editor for your Drupal 8 and 7 website.
This is more than yet another CKEditor build enhanced with js.plus plugins. It is also a control panel for effortless configuration.
Bootstrap 4 support can be enabled or disabled with one click making N1ED a full-featured Bootstrap structure editor.
The editor features its own image uploader that does not require any additional configuring.
All of your existing content is compatible with N1ED (any HTML content is supported).

Categories: Drupal

Flufftopia. One year later. Numbers and thoughts about a (probably) dead project - by Daniel da Silva

Gamasutra.com Blogs - 26 February 2019 - 5:16am
A postmortem for the game Flufftopia. Focus on numbers!
Categories: Game Theory & Design

Remembering Advance Wars - by Josh Bycer

Gamasutra.com Blogs - 26 February 2019 - 5:15am
2018 would have marked the 30th anniversary of the Advance Wars franchise, and the recent success of Wargroove made me nostalgic to talk about a great strategy series that you may have missed.
Categories: Game Theory & Design

Development of Difficulty in Games - by Kaleb Eberhart

Gamasutra.com Blogs - 26 February 2019 - 5:14am
This post dives into the general development of game difficulty starting from the first display of a difficulty curve designed on accident to dynamic difficulty adjustment, mainly focusing on Japanese game design.
Categories: Game Theory & Design

Accessible Media Embed

New Drupal Modules - 26 February 2019 - 4:31am

This module aims to bridge the gap between embedding media within a WYSIWYG such as ckeditor and having a fully accessible website using context sensitive alt tags for that embedded media.

What's the problem?

Currently you add an alt tag to an image when adding an image to the media library and this alt tag is used each time the media is embedded within some content.

Categories: Drupal

Capers Review

Gnome Stew - 26 February 2019 - 4:30am

If you are like me, you have probably shot straight up in the middle of the night, sweating, thinking, “what if Johnny Dangerously or Crimewave happened in the same universe as the X-Men? What if anyone other than me had actually seen Crimewave?”

If you aren’t like me, this has never happened to you, so please disregard. However, since I am me, I actually received a response to my fever dream in the form of Capers — a 1920s based roleplaying game about the clash between law enforcement and organized crime, with superpowers thrown in for good measure.

What Kind of Glad Rags Are These?

 This review is based on the PDF of the product. The game comes in at 165 pages, including a one-page rules summary, an 8-page appendix, and two pages of Kickstarter backers at the beginning.

The book is full color, with various headers and sidebars to call out new sections and to highlight optional rules and intent. The chapter heading and sidebars use lettering and flourishes that are reminiscent of the time period, and do a good job of setting the mood for the material in the book. There is also a four-page inset between the Player and GM sections, with a full-color comic depicting a typical sequence of events in the setting, which is a nice nod both to the time period and the comic book inspirations of the game.

 A Word About Setting

 Capers is set in a world where many NPCs are a different gender or ethnicity than the historical figure that inspired that character. Given the world of 1920s organized crime and law enforcement, you aren’t going to see much in the way of diversity without taking a step like this. Despite these changes, the only real historical divergence is the emergence of super-powered individuals after World War I, and many references are made to the existing tensions of the time regarding race, ethnicity, and religious intolerance.

As a cis white male that cares about diversity, I like this on the surface, but I also worry that just because it makes me “comfortable” with the era as a setting, that may be allowing me to live in a comfort zone that I should instead be examining. I’m not saying this is a good or a bad thing, just that it’s a complicated issue that I don’t have an answer for, and I’m not factoring in this choice as a good or a bad thing overall. It is something I want to spend more time exploring and discussing in the future.

Player Section

 The Player Section is broken up into smaller chapters that cover The Game, Character Creation, Rules System, Perks, Powers, Goods and Services, and Trembium.

The Game

The Game is really a section on what the game is about. What is the setting, and what are characters expected to do? In this case, the focus of the game is on the Prohibition Era, and playing either law enforcement officials or criminals in this time period. There is a nice bullet-pointed list spelling out the kinds of actions that PCs may take, as well as how the core mechanic, which is resolved with cards instead of dice, plays into the feel of the game.

I really like that the game is deliberate about its scope, explaining that it’s very focused on 1920s “Cops and Crooks” gameplay. I also really appreciate that this section introduces a sidebar on safety very early on. It’s easy to fall into the over the top action of the setting and then trip over concepts like gang violence, racial tensions, and addiction. The sidebar doesn’t dive deeply into any one of these topics, but does make it clear that difficult topics should not be introduced if those topics push anyone’s boundaries at the table, and it mentions the importance of having discussions and safety tools available.

Character Creation

Character creation is the next chapter, and the process is one that should feel familiar if you have played any traditional roleplaying games. Characters are either Regulars, Exceptionals, or Capers, although PCs are most likely Exceptionals or Capers. Exceptionals can pick up perks that aren’t available to Capers, but they fall short of being superhuman abilities. Capers don’t get access to perks, but they get to choose powers.

Characters pick three anchors — an identity, a virtue, and a vice. These all have triggers written into their descriptions for gaining moxie, a currency in the game that has multiple effects. If you don’t have anything in mind, there are card values assigned to these anchors to allow for random draws. Characters then pick traits and skills, which are added together to determine the number of cards a character can draw when they attempt an action. The next step is picking perks and powers (and with the GM’s permission, access to special gear that simulates powers).

Characters are assigned a level, but the main function of level is to summarize how many advancement points the character has. After spending initial advancement points, each time a character levels up, they gain two more advancement points.

The Rules System

The Rules System chapter delves into how to make those things you picked in the last chapter work during the game. Character creation feels a lot like what you might do in a dice pool style game, adding skills to traits to determine how many randomizers are used in a pool for that action. The difference is, instead of using all of your randomizers, you choose how many of your card draws to use, one at a time. The value of the card will determine the success or failure, but the suit of the card will determine if there is an ancillary good or bad side effect to that success or failure. You might draw a high-value card that is likely to give you a success, but you may still be tempted to draw another card to attempt a success without a complication.

Some powers are passive effects that modify other aspects of the character (like having superhuman strength, bypassing the normal limit on the trait), while others have their own power level that determines the number of cards a player can draw when attempting to activate or use the power.

Moxie can be spent to increase card counts, reduce damage taken, use a previously drawn card, add an element to the narrative, reshuffle the deck (normally done at the end of a scene), or to take damage for another character.

Initiative is a fairly standard skill test, with ranked results determining order, and held actions triggering a change in the order. There is also a special section on how to resolve a standoff, that uses a three-round game of “keep or draw” with the cards to simulate a tense situation before the action breaks out.

Perks and Powers

The next two chapters are on Perks and Powers, but the Perks chapter is only a single page. Perks are simple mechanics that let a non-powered character be a little tougher, more skillful, or resistant to superpowers, without being super-powered themselves.

If you have seen a wide variety of games with superpowers, you may have seen that some games favor discreet rules packets similar to spells in a game like D&D to describe powers in the game, while others, like Champions or Mutants and Masterminds, include game effects that can be slotted together to assemble a recognizable superpower. Capers leans towards the former, with subsections explaining powers like “Cold Beam” having its own set of rules, as well as some upgrades that can be purchased later. Some have persistent maintainable effects that cause a character to reduce their card count while the effect is active.

The powers touch on a lot of comic book staples, but definitely stay grounded in what sometimes gets categorized as “street level” powers. Picking up a car is impressive for super strength, and mental powers lean more towards “super short term hypnotism” rather than a deep exploration of telepathic abilities.

Goods and Services

Goods and Services is another short chapter, coming in at two pages. It gives the most common prices for 1920s goods that would matter to a game centered around law enforcement and organized crime, and gives stats for items like explosives or weapons. It also covers the price of real estate, for when you need options for a new safehouse or expanding your business holdings.

Trembium

Trembium is the final chapter in this section, and it’s about three pages long. Trembium is the element that has caused the upswing in superpowers. It can be used to create gear that mimics superpowers, and can be used in experiments to make super soldiers. While it is the default origin of superpowers in the setting, the rules surrounding it are optional, so it feels a little odd to have it in the Player’s Section. It’s an understandable compromise, however, since if it is allowed in the game, a lot of player options need to reference the rules.

In particular, the idea of nullifying or granting powers via direct injection of Trembium got a lot of creative juices flowing in my mind, and Trembium in general has a lot of potential if the Capers world opens up to other historical periods.

 GM Section

 The GM Section contains chapters on GM Guidelines, Backdrops, Law Enforcement, Friends and Enemies, and a GM Toolbox.

GM Guidelines

The GM Guidelines section has advice on how to structure adventures and encounters, player agency, and examples of encounter difficulty. There are also rules for advancing NPCs, statting out animals, designing your own powers, and a whole host of other eclectic, GM specific items that might come up in a campaign.

What I like best about this section is that encounter and adventure building is very much focused on having an idea of why you want to include something, and the structure for how to move from point A to point B. It’s not deeply philosophical, as games go, but it is very functional, and a clear indication of how the game is expected to run.

Backdrops

The Backdrops section is one of my favorites in the book, because it does what I wish a lot more books that include a setting would do. It gives you the basics of what makes an area unique, summarizes the important people and things, and then gives you sample adventure starters and rumors that are in line with the setting that has just been presented. It is very focused on how to use the information at the table.

Atlantic City, Chicago, and New York get the deepest treatment, with several pages of setting information and adventure ideas, city maps, and a few pages for stats of unique NPCs. Atlanta, Boston, Cincinnati, Detroit, Kansas City, Louisville, Miami, Milwaukee, New Orleans, and Philadelphia all get single page treatments, which save space by cutting out maps and suggesting “generic” NPC stat blocks for notable NPCs that are mentioned.

I like the degree to which all of the backdrops are “table ready,” and I enjoy seeing the subtle differences in the interplay between law enforcement and criminal elements in all of these mini-settings. Knowing how it works in one place helps to reinforce an intentional tone for other settings. If anything, it feels like more of the larger setting NPCs could have been summarized in a similar manner to the smaller setting characters, but that’s a minor quibble.

Law Enforcement

The Law Enforcement chapter goes into about two pages of details about how federal and local law enforcement is organized in the 1920s, then presents some NPC Feds for use in the game. Friends and Enemies follows, and presents a stat block and a descriptive paragraph for various NPCs that the PCs may encounter in the game. For the most part, there are about four stat blocks to a page, starting with unique NPCs that can be inserted anywhere, then moving to more general stat blocks like “Cop” or “Gambler.”

GM Toolbox

The GM Toolbox has a lot of wide-ranging material that might be of use to a GM that is looking to throw another curve into the standard campaign expectations. There are guidelines for alternate Earths, super-powered animals, and “events” where the number of people with superpowers around the world spikes, shifting the assumptions of the setting for a period of time.

I’m a comic book geek, and I love alternate reality tropes, so that’s my favorite section in this chapter. There are outlines for a world where robots are commonplace, a “mirror universe” style Earth, and one where Earth has suffered from an alien invasion, with stat blocks of example NPCs, and even some guidelines on playing characters native to those worlds.

 Reference Section

 The Reference Section includes appendices that cover a wide range of materials. This includes 1920s slang, a power index, common names in the 1920s, an NPC index, a list of inspirational material, a full index, and a one-page rules summary as well as a character sheet.

Like much of the rest of the book, this material is very functional. Having slang and names handy for improvisation is great, and I really like the one-page rules summary, as it manages to hit on some of the biggest items that would normally tempt players and GMs to halt the game and look up how a given rule works.

 It takes a fairly standard structure for how to resolve actions, and swaps in the card-based resolution that shifts expectations of how and when you should keep pushing your luck. Share1Tweet1+11Reddit1Email Successful Delivery

 The card-based mechanics add an element of gambling to a character’s actions that trigger a different risk versus reward evaluation than simple numbers can evoke. Using the suit to add dimensionality to success or failure is a great use of cards as a randomizer. The various settings for the game are summarized in a manner that makes them very useable at the table, and manages to bring out some of the differences between these regions as settings for different ongoing games.

 It’s a Raid!

 In a game where the actions flow freely without a lot of granular tactical movement or tracking, listing movement rates and ranges of powers in absolute feet feels strange to me. Initiative order and how readied actions work also feel like holdovers from older game design that doesn’t highlight the best aspects of the system. I like the anchors, but some of the items that trigger gaining moxie feel a little soft — I personally like more pointed, yes or no questions when it comes to awards based on character traits.

Recommended — If the product fits in your broad area of gaming interests, you are likely to be happy with this purchase.

The strength of Capers is that it pushes just enough to make something familiar into something fun and new. It takes a known time period and genre, Prohibition-era cops and crooks, and adds in superpowers. It takes a fairly standard structure for how to resolve actions, and swaps in the card-based resolution that shifts expectations of how and when you should keep pushing your luck.

In a few places, I feel like it played the traditional structure a little too safe, but overall, it has a great dynamic energy that is engaging. Not only am I interested in this particular implementation, I want to see how this world develops, and how these rules can be used to express other genres in other points in history.

Have you been a fan of Prohibition-era gaming in the past? Are you a fan of adding superpowers to different time periods? What are your favorite games that have done this? We would love to hear from you, so please leave a comment below. Thanks!

Categories: Game Theory & Design

Dropsolid: Machine Learning for optimizing search results with Drupal & Apache Solr

Planet Drupal - 26 February 2019 - 3:00am
26 Feb

Recently, a client of ours shared with us their frustration that their website’s internal site search results didn’t display the most relevant items when searching for certain keywords. They had done their homework and provided us with a list of over 150 keywords and the expected corresponding search result performance. Their site was built with Drupal 7 and relied on Search API & Search API Solr for content indexing and content display.
 

Language Processing

Since the website was in Dutch, we started by applying our usual best practices for word matching. No instance of Apache Solr search will work properly if the word matching and language understanding haven’t been appropriately configured. (see also this blog post)

For this particular website, many of the best practices for Apache Solr Search hadn’t been followed. For example, you always need to make sure to store the rendered output of your content and additional data (such as meta tags) in one single field in the index. This way, it’s a lot easier to search all the relevant data. It will make your optimizations easier further down the road and it will render your queries a lot smaller & faster. Also make sure to filter out any remaining HTML code, as it does not belong in the index. Last but not least, make sure to index this data using as little markup as possible, and get rid of the field labels. You can do this by assigning a specific view mode to each content type. If you’re having trouble with these basics, just get in touch with us so we can give you a hand.
 

Boosting

Once this has been fixed, you can use the boosting functionality in Apache Solr to prioritize certain hypotheses on how much more important a certain factor will be in the overall generation of results. The client website that I mentioned earlier, for example, had some custom code written into it to boost the content based on the content type during the time of indexing.

Apache Solr works with relevance scores to determine where a result should be positioned in relation to all of the other results. Let’s dig into this a bit deeper.

http://localhost:8983/solr/drupal7/select? q=ereloonsupplement& // Our term that we are searching for defType=edismax& // The eDisMax query parser is designed to process simple phrases (without complex syntax) entered by users and to search for individual terms across several fields using different weighting (boosts) based on the significance of each field while supporting the full Lucene syntax. qf=tm_search_api_aggregation_1^1.0& // Search on an aggregated Search Api field containing all content of a node. qf=tm_node$title^4.0& // Search on a node title qf=tm_taxonomy_term$name^4.0& // Search on a term title fl=id,score& // Return the id & score field fq=index_id:drupal7_index_terms& // Exclude documents not containing this value fq=hash:e3lda9& // Exclude documents not containing this value rows=10& // Return top 10 items wt=json& // Return it in JSON debugQuery=true // Show me debugging information

When looking at the debugging information, we can see the following:

"e3lda9-drupal7_index_terms-node/10485": "\n13.695355 = max of:\n 13.695355 = weight(tm_search_api_aggregation_1:ereloonsupplement in 469) [SchemaSimilarity], result of:\n 13.695355 = score(doc=469,freq=6.0 = termFreq=6.0\n), product of:\n 6.926904 = idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:\n 10.0 = docFreq\n 10702.0 = docCount\n 1.9771249 = tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:\n 6.0 = termFreq=6.0\n 1.2 = parameter k1\n 0.75 = parameter b\n 248.69698 = avgFieldLength\n 104.0 = fieldLength\n",


This information shows that item scores are calculated based on the boosting of the fields that our system has had to search through. We can further refine it by adding other boost queries such as:

bq={!func}recip(rord(ds_changed),1,1000,1000)


This kind of boosting has been around in the Drupal codebase for a very long time and it even dates back to Drupal 6. It is possible to boost documents based on the date they were last updated, so that more recent documents will end up with higher scores.
 

Solved?

Sounds great, right? Not quite for the customer, though!

As the client is usually going back and forth with the development company for small tweaks, every change you make as a developer to the search boosting requires a full check on all the other search terms. This needs to be done to make sure the boosting you are introducing doesn’t impact other search terms. It’s a constant battle - and it’s a frustrating one. Even more so because in the above scenario the result that was displayed at the top wasn’t the one that the client wanted to show up in the first place. In the screenshot you can see that for this particular query, the most relevant result according to the customer is only ranked as number 7. We’ve had earlier instances where the desired result wouldn’t even be in the top 50!

To tackle this, we use an in-house application that allows end users to indicate which search results are ‘Not Relevant’,  ‘Somewhat Relevant’, ‘Relevant’ or ‘Highly Relevant’, respectively. The application sends direct queries to Solr and allows the client to select certain documents that are relevant for the query. Dropsolid adjusted this so that it can properly work with Drupal Search API Solr indexes for Drupal 7 and 8.

We’ve used the application from the screenshot to fill in all the preferred search results when it comes down to search terms. In the background, it translates this to a JSON document that lists the document IDs per keyword and their customer relevance score from 0 to 3, with 3 being the most relevant.

This is a very important step in any optimization process, as it defines a baseline for the tests that we are going to perform.
 

Footnote: This image is a separate application in Python based on a talk of Sambhav Kothari from Bloomberg Tech at Fosdem.
 

Learning To Rank

We quote from the Apache Solr website: “In information retrieval systems, Learning to Rank is used to re-rank the top N retrieved documents using trained machine learning models. The hope is that such sophisticated models can make more nuanced ranking decisions than standard ranking functions like TF-IDF or BM25.

Using the baseline that we’ve set, we can now calculate how well our original boosting impacts the search results. What we can calculate, as shown in the screenshot above, is the F-Score, the recall & precision of the end result when using our standard scoring model.

Precision is the ratio of correctly predicted positive observations to the total predicted positive observations.
The question that Precision answers is the following: of all results that labeled as relevant, how many actually surfaced to the top? High precision relates to the low false positive rate.” The higher, the better, with a maximum of 10.
Src: https://blog.exsilio.com/all/accuracy-precision-recall-f1-score-interpretation-of-performance-measures/

Recall is the ratio of correctly predicted positive observations to the all observations in actual class.

The question recall answers is: Of all the relevant results that came back, what is the ratio compared to all documents that were labeled as relevant? The higher the better, with a maximum of 1.
Src: https://blog.exsilio.com/all/accuracy-precision-recall-f1-score-interpretation-of-performance-measures/

If we just look at the top-5 documents, the combined F-Score is 0.35, with a precision of 0.28 and a recall of 0.61. This is quite bad, as only 60% of our relevant documents appear in the top 5 of all search queries. The precision tells us that from the top-5 documents, only 30% has been selected as relevant.
 

Training our model

Before all of this can work, we have to let Solr know what possible features exist that it might use to decide the importance of each feature based on feedback. An example of such a feature could be the freshness of a node - based on the changed timestamp in Drupal -, or it could just as well be the score of the query against a specific field or data such as meta tags. For reference, I’ve listed them both below:

{ "name":"freshnessNodeChanged", "class":"org.apache.solr.ltr.feature.SolrFeature", "params":{ "q":"{!func}recip( ms(NOW,ds_node$changed), 3.16e-11, 1, 1)" }, "store":"_DEFAULT_" }, { "name":"metatagScore", "class":"org.apache.solr.ltr.feature.SolrFeature", "params":{ "q":"{!edismax qf=tm_metatag_description qf=tm_metatag_keywords qf=tm_metatag_title}${query}" }, "store":"_DEFAULT_" }

Using the RankLib library (https://sourceforge.net/p/lemur/wiki/RankLib/), we can train our model and import it into Apache Solr. There are a couple of different models that you can pick to train - for example Linear or Lambdamart - and you can further refine the model to include the number of trees and metrics to optimize for.
You can find more details at https://lucene.apache.org/solr/guide/7_4/learning-to-rank.html
 

Applying our model

We can, using the rq parameter, apply our newly trained model and re-rank the first 100 results according to the model.

rq={!ltr+efi.query=ereloonsupplement+model=lambdamart-NDCG@10-100-2019-02-11-12:24+reRankDocs=100}

If we look at the actual result, it shows us that the search results that we’ve marked as relevant are suddenly surfacing to the top. Our model assessed each property that we defined and it learned from the feedback! Hurray!
 


We can also compare all the different models. If we just look at the top-5 documents, the combined F-Score of our best performing model is 0.47 (vs 0.35), with a precision of 0.36 (vs 0.28) and a recall of 0.89 (vs 0.61) This is a lot better, as 90% of our relevant documents appear in the top 5 of all search queries. The precision tells us that from the top-5 documents, 36% has been selected as relevant. This is a bit skewed, though, as for some results we only have one highlighted result.

So, to fully compare, I did the calculations it for the first result. With our original model we only see 46% of our desired results pop up as the first result. With our best-performing model, we improve this score to 79%!

Obviously we still have some work to do to turn 79% up to 100%, but I would like to stress that this result was achieved without changing a single thing to the client’s content. The remaining few cases are missing keywords in the meta tags of Dutch words that somehow are not processed correctly in Solr.
 


Of course we wouldn’t be Dropsolid if we hadn’t integrated this back into Drupal! Are you looking to give back to the community and needing a hand to optimize your search? Give us the opportunity to contribute this back for you - you won’t regret it.
 

In brief

We compiled a learning dataset, we trained our model and uploaded the result to Apache Solr. Next, we used this model during our queries to re-rank the last 100 results based on the trained model. It is still important to have a good data model, which means getting all the basics on Search covered first.

Need help with optimizing your Drupal 7 or Drupal 8 site search?

Contact us

Nick Veenhof
Categories: Drupal

Composer Forced

New Drupal Modules - 26 February 2019 - 2:41am

Allows your website to have update manager enabled but totally disables ALL updates from the Drupal UI. Update manager module will only be useful for sending you email notifications about the updates of modules and core as configured from the administration settings page.

Categories: Drupal

Game Developers Conference 2019 – new & fun content! - by Simon Carless

Gamasutra.com Blogs - 26 February 2019 - 2:39am
So, with Game Developers Conference 2019 rapidly approaching (it’s March 18th to 22nd, date fans!), I thought it would be good to highlight some of the interesting new conference content we’ve rolled out this year.
Categories: Game Theory & Design

Sooper Drupal Themes: Expectations for Drupal 9 in 2020

Planet Drupal - 26 February 2019 - 1:51am
Turmoil in the Drupal community?

Considering the fact that there are around 800.000 websites currently operating on Drupal 7, there will be a huge resource drain for upgrading to the latest installment. Not only will be it be an expensive feat to achieve, but also time demanding. Quite frankly speaking, there is not enough time to be able to upgrade all of the Drupal 7 websites and also not enough Drupal developers to be able to take on the workload. So, what do you think, is it feasable for so many websites to upgrade to Drupal 9 in such a short period of time?

Drupal 9 will be released in the summer of 2020

Drupal 8 has been released on November 19, 2015, this makes it 3 years old already. Its successor, Drupal 9, is making its way towards a release. Drupal 9 is scheduled to be released on 3rd June in 2020. But what does this mean for Drupal 7 and 8?

For starters, Drupal 8 and 7 will stop receiving support in 2021. This is mainly because Symfony 3, one of the biggest dependencies of Drupal 8, will stop receiving support. Drupal 7 will not be backed up by the official community and by the Drupal association on Drupal.org. What this means is that the automated testing services for Drupal 7 will be shut down. On top of that, the Drupal Security Team will stop providing security patches. If you are not able to upgrade to Drupal 9, there will still be some organisations that will provide Drupal 7 Vendor Extended Support, which will be a paid service. However, despite this, there will be a approximately year's worth of time to be able to plan for and upgrade to the latest installment of Drupal.

Overview over the consequences for Drupal 7

What this means for your Drupal 7 sites is, as of November 2021:

  • Drupal 7 will no longer be supported by the community at large. The community at large will no longer create new projects, fix bugs in existing projects, write documentation, etc. around Drupal 7.
  • There will be no more core commits to Drupal 7.
  • The Drupal Security Team will no longer provide support or Security Advisories for Drupal 7 core or contributed modules, themes, or other projects. Reports about Drupal 7 vulnerabilities might become public creating 0 day exploits.
  • All Drupal 7 releases on all project pages will be flagged as not supported. Maintainers can change that flag if they desire to.
  • On Drupal 7 sites with the update status module, Drupal Core will show up as unsupported.
  • After November 2021, using Drupal 7 may be flagged as insecure in 3rd party scans as it no longer gets support.
  • Best practice is to not use unsupported software, it would not be advisable to continue to build new Drupal 7 sites.
  • Now is the time to start planning your migration to Drupal 8.

Source: https://www.drupal.org/psa-2019-02-25

Drupal promises a smooth upgrade to Drupal 9

Good news is that, the change from Drupal 8 to Drupal 9 will not be as abrupt as the change from Drupal 7 to 8 was. This is because Drupal 9 will be based off of Drupal 8, in fact, the first release of Drupal 9 will be similar to the last release of Drupal 8. In short, there will be some new features added, the deprecated code will be removed and the dependenciess will be updated, however, the Drupal experience will not be reinvented. Now, in order to have really smooth upgrade, the only thing necessary is to keep your Drupal 8 updated at all times. This will ensure that your upgrade will come as fluid as possible, without many inconveniences.

What is the best course of action to follow when upgrading to Drupal 9

Well, at first, you have a couple of options at your disposal:

  1. You either wait for Drupal 9 to be launched and then make the change from 7 directly to 9.
  2. You make first the change to Drupal 8 from 7, which is going to be an abrupt change anyway, and then you prepare for the change to Drupal 9.
  3. Your final option would be to find a new CMS altogether, which would be the most resource hungry option out of all. 

So, considering the choices you have at hand, the best of the bunch would be to start preparing for upgrade to Drupal 8 and then, when the time comes, to Drupal 9. By doing this, you will have enough time to plan ahead your upgrade roadmap, without having to compromise on the quality of the upgrade by rushing it. On top of that, this is an opportunity to review how good your website is at attracting leads and converting those leads to sales. Moreso, you can check if your website is still in line with your enterprise vision and mission statement, if not, then here is an opportunity to make your site reflect the beforenamed aspects of your business.

Even though change might look scary to some of you, this is an opportunity to also evaluate and improve your online digital presence. So make use of this chance at its fullest to create and provide a better online environment for your potential and current customers.

Categories: Drupal

Configuration Split Ignore

New Drupal Modules - 26 February 2019 - 1:00am

The module allows to ignore certain configurations owned by modules enabled in configuration splits during export and import.

It works the same way as Config Ignore module for the configuration which exist in configuration splits made by Configuration Split module.

Categories: Drupal

Pages

Subscribe to As If Productions aggregator