It is always the best policy to tell the truth, unless, of course, you are an exceptionally good liar.
In the last part of our blog series, we dealt with the specifications of a project. Today we discuss the issue of responsibilities and ongoing communication. Ensure that the infrastructure exists to support project communication and that everyone has access to it – and uses it! Keep up the communication in the project and make sure that there’s a central communication tool, especially if you’re working in distributed teams. Define which communications are task-related and should be persisted on the task. This is mostly about teamwork: there’s nothing more damaging to a project than the stagnation of communication after a certain length of the project time, because then everyone in the project team makes different assumptions, leading into different directions. Recurring meetings, such as dailies and weeklies, help to build a culture of ongoing communication.
In addition to the above, also note the following as important points to take into account:1) Provide a contact person Project work is teamwork. Both suppliers and customers need to meet their obligations in the project. One of the key things is the obligation to cooperate in making decisions. In particular, this includes the acceptance of partial results or the overall project. To make such decisions, a person needs the skills to take this decision and the authority to do so. If there is no clear contact person who’s responsible for all decision areas – financially, professionally and organizationally – the process may stagnate. This can delay the whole project and applies to both sides, vendors and customers alike. 2) What might happen if this contact person doesn’t exist (from the perspective of customer)? Everyone has something to say. The change requests will be considered by all project participants, but remember that only one of them gets the bill at the end of the day. Naturally, the customer may wonder about the amount, but no one ever wants to be responsible. Maybe the wishes were contradictory and mutually canceled each other out in the progression of things, but now, of course, one individual is surprised that – wouldn’t you know – exactly HIS wishes were swallowed up by the ominous “project monster”! This role is usually awarded to the responsible project manager, who in turn must prove that he’s only done what was required. 3) And what can happen from the perspective of the provider? During the project, customer requirements are discussed with various people. If these requirement changes don’t end up in a pot and if reviews are always subjective, it can lead to unwanted side effects. Example: Contact A builds a new forum feature. Person B says: "We’ll delete the forum" while Person C is under the belief that everything is ready and, hence, plans to start the acceptance process. 4) Emphasize all duties right from the beginning As a provider you and your customers should be clear on what obligations each respective party has. Ensure that the project is clear through transparent project management. Don’t drop tasks and always record decisions so that, later, you can see how they were made and how they’ve influenced the overall project. 5) Communicate problems early Of course, it can also come to additional expenses or delays in the project. Make sure that you identify these problems early and communicate them throughout the responsible team. If problems are addressed and resolved constructively in your project culture, as opposed to having long debates about who’s to blame for the problem, even large hurdles can be overcome together. 6) It’s no shame to question things, so ask often! If there are any questions in the project – and there will be – ask them! Anyone who’s too afraid or lazy to ask when there are ambiguities does harm to the project. Assumptions and certainties contradict every form of transparent communication. So provide the team with a centralized and transparent communication (a chat isn’t sufficient as it doesn’t persit the communication). Also, ensure that information and agreements actually make it to the people who need to have access to it. In the next part of our series, we’ll focus on unrealistic budgets and deadlines.
Here's an example of an assumption; The sun will rise tomorrow. An assumption is something that is accepted as true or as certain to happen, without proof. This kind of thinking, while convenient, is prone to concealing facts, and troublesome when debugging code. This article defines what an assumption is, and provides some techniques for helping to eliminate them during debugging.
This module provides a framework to detect potentially fraudulous orders and act on this.
This module provides:
I recently worked with Blue Dot Lab to build a rapidly redeployable, interactive booking system for the University-National Oceanographic Laboratory System (UNOLS) based in the US. UNOLS is a consortium of over 60 academic institutions involved in oceanographic research, and individual institutions can require their own system for organizing research expeditions and booking the necessary equipment and boats. Such new systems need to be ready to go with the minimum of fuss and at reasonably short notice.
A submodule of Entity Documentation that exports documentation of vocabularies.Requirements
No problems at the moment.
This module offers extra functionality for Events in the OpenLucius distro:
- Let users choose: ‘Attending’ or ‘Not attending’.
- Send email reminders to all users that did not make a choice yet.
- Let users provide a reason: ‘Why are you not attending?’
- Show a Google map with location of Event.
Download and install this module as usual.
So, stemming, what is stemming? Generally speaking, stemming is finding the basic form of a word. For example, in the sentence "he walks" the verb is inflicted by adding a "s" to it. In this case the stem is "walk" which, in English, also happens to be the infinitive of the verb.
We will first present a few examples of stemming in natural language, and since Dutch is my native language I will concentrate on Dutch examples.
After that we will show the results of a number of stemmers present in Solr and give a few pointers about what to do if the results of these stemmers are not good enough for your application.Plurals
On of the things you absolutely want your user to be able to, is to find results which contain the single form of a word while searching for the plural and vice versa, e.g.: finding "cat" when looking for "cats" and finding "cats" when searching for "cat".
Although in English there are well-defined rules for creating the plural form (suffix with "s", "es" or change "y" to "ie" and suffix "s"), there also are a number of irregular nouns ("woman" -> "women") and nouns for which the single and plural form are the same ("sheep", "fish").
In Dutch more or less the same situation exists, be it with different suffixes ("s", "'s", "en") and, of course, other exceptions.
Furthermore, in Dutch if the stem ends on a consonant directly preceded by a vowel, this consonant is doubled (otherwise, in the plural form, the vowel would sound like a long vowel instead of a short vowel), e.g.:kat (cat) -> katten (cats)
But, to this rule there also are exceptions, likemonnik (monk) -> monniken (monks)
in contrasts with:krik (car jack)-> krikken (car jacks) Verb conjugation
Conjugation of verbs in Dutch is, to be blunt, a bit of a mess.
In Dutch, for forming the past tense of a verb, two types of conjugation co-exist: the (pre-) medieval system, now called strong and the more recent system, called weak. When I say the systems co-exist, one should note that most (native) Dutch speakers are not aware of the fact that the strong-system is a system at all: the they consider the strong verbs to be exceptions, best learned by heart.
An example of a strong verb is "lopen" (to walk):hij loopt (he walks) -> hij liep (he walked)
While an example of a weak verb is "rennen" (to run):hij rent (he runs) -> hij rende (he ran)
These examples make clear that determening which verb is strong and which verb weak is indeed a case of learning by heart.
Furthermore the change from strong to weak verbs is a ongoing process. One exampe of a verb which is currently in transition from strong to weak is the verb "graven" (to dig) of which both the form "hij groef" (he digged) and "hij graafde" can also be found, although most language-purist would consider the last form as "wrong".
NB: if you are interested in this kind of things, a classic book about language changes is Jean Aitchisons Language change: progress or decay (1981, yes, it is a bit pre-internet...)Diminutives
In a number of languages, like Dutch, German, Polish and many more, diminutives are created by inflicting the word. In English you form the diminutive by adding an adjective like 'little', but in Dutch the general rule to form a diminutive is to add the suffix "je" to the word, e.g.:huis (house) -> huisje (little house)
This is the general rule, because the suffix can als be inflicted, like inbloem (flower) -> bloempje (little flower)
And in some words te ending consonant is changed to keep the word pronouncable:hemd (shirt) -> hempje (little shirt)
It is however also possible in Dutch to use an adjective like 'klein' (little) and even to combine both:kleine bloem (little flower) -> klein bloempje (small little flower)
A last peculiarity I should mention is that in Dutch (but also in many other languages) there are words which only have a diminutive form, like 'meisje' (girl).Homographs and homonyms
For some words it is not possible to find the correct stemming without knowing the semantics or context, e.g. kantelen which if pronounced like kantélen means "battlements" but when pronounced kántelen means "tip over". Or zij kust ("she kisses") versus kust like in de Noordzeekust ("the North sea coast").Why bother?
So maybe by now you are asking yourself: why bother? Well, you should be bothered because stemming will make it easier for the visitors of your site to find what they are looking for.
For example, you can almost be sure that when a visitor is interested in articles about houses (in Dutch 'huizen'), he will also be interested in articles which mention a house ('huis').
So when using the search term 'huizen' it would be nice if results which contain 'huis' would automatically be shown.
Of course searching a verb is much less common, and the chance that a visitor will use the dimunitive is also not very great, but still it happens and if it takes only a minimal effort to make sure the visitor finds what he is searching for, then why not?Solr
Starting form Solr version 3.1, for English (and a number of other languages) there is a standard filter "EnglishMinimalStemFilterFactory" which has the ability to stem English words. For Dutch however, such a simple filter factory is not available.
There are however a number of default languages that can be used with the SnowballPorterFilterFactory and in the default schema included in Solr 5 a number of such fields are predefined.Solr 5 default schema
In Solr 5 the default schema defines a list of language specific fieldtypes. For Dutch the fieldtype 'text_nl' is defined as follows:<dynamicField name="*_txt_nl" type="text_nl" indexed="true" stored="true"/> <fieldType name="text_nl" class="solr.TextField" positionIncrementGap="100"> <analyzer> <tokenizer class="solr.StandardTokenizerFactory"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_nl.txt" format="snowball" /> <filter class="solr.StemmerOverrideFilterFactory" dictionary="lang/stemdict_nl.txt" ignoreCase="false"/> <filter class="solr.SnowballPorterFilterFactory" language="Dutch"/> </analyzer> </fieldType>
So in short, in the SnowballPorterFilterFactory the language is set to Dutch.
There is however a alternative stemming algorithm avilable, the Kraaij-Pohlmann algorithm, see Porter’s stemming algorithm for Dutch, known in Solr as Kp
To compare both algorithms, we define a new Dutch fieldtype as follows:<dynamicField name="*_txt_nlkp" type="text_nlkp" indexed="true" stored="true"/> <fieldType name="text_nlkp" class="solr.TextField" positionIncrementGap="100"> <analyzer> <tokenizer class="solr.StandardTokenizerFactory"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_nl.txt" format="snowball" /> <filter class="solr.StemmerOverrideFilterFactory" dictionary="lang/stemdict_nl.txt" ignoreCase="false"/> <filter class="solr.SnowballPorterFilterFactory" language="Kp"/> </analyzer> </fieldType>
To complete our analysis we will also use the default English language field, defined as:<dynamicField name="*_txt_en" type="text_en" indexed="true" stored="true"/> <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100"> <analyzer type="index"> <tokenizer class="solr.StandardTokenizerFactory"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt" /> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.EnglishPossessiveFilterFactory"/> <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> <filter class="solr.PorterStemFilterFactory"/> </analyzer> <analyzer type="query"> <tokenizer class="solr.StandardTokenizerFactory"/> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt" /> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.EnglishPossessiveFilterFactory"/> <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> <filter class="solr.PorterStemFilterFactory"/> </analyzer> </fieldType> Comparison
In the data shown next we compare the three above defined fields with the correct values.Fieldtype text_en Input katten monniken meisje hempje krikken huizen huisje bloempje loopt Output katten monniken meisj hempj krikken huizen huisj bloempj loopt Input liep lopen rent rende rennen kust kussen kantelen Output liep lopen rent rend rennen kust kussen kantel Fieldtype text_nl (language = dutch) Input katten monniken meisje hempje krikken huizen huisje bloempje loopt Output kat monnik meisj hempj krik huiz huisj bloempj loopt Input liep lopen rent rende rennen kust kussen kantelen Output liep lop rent rend renn kust kuss kantel Fieldtype text_nlkp (language = kp) Input katten monniken meisje hempje krikken huizen huisje bloempje loopt Output kat monnik meis hem krik huis huis bloem loop Input liep lopen rent rende rennen kust kussen kantelen Output liep loop rent rend ren kust kus kantel Correct values Input katten monniken meisje hempje krikken huizen huisje bloempje loopt Output kat monnik meisje hemd krik huis huis bloem loop Input liep lopen rent rende rennen kust kussen kantelen Output loop loop ren ren ren kus (verb)
kust (noun) kus kantel (verb)
The most noticable conclusion of above comparison is that the output of the text_nl-field does not differ much from the text_en-field.
It seems that the 'Dutch'-language implementation of the SnowballPorterFilter has no way of stemmming dimunitives, results like "huisj" and "bloempj" are just plain wrong, while the Kraaij-Pohlmann correctly returns "huis" en "bloem".
The same holds for the plural "huizen" which is correclty stemt by Kraaij-Pohlmann to "huis".
The dimunitive "meisje" is stemt by Kraaij-Pohlmann to "meis" which in some dialects of Dutch, like the dialect spoken in De Zaanstreek, is actually correct. There is however a way to correct this, see the section about KeywordMarkerFilterFactory under "a bit disappointed?".
And "hempje" is wrongly stemt to "hem", which seems a too general application of the rule which correctly stems "bloempje" to "bloem"
None of the algorithms knows how to handle homographs like "kust" and "kantelen" but this was to expected.A bit disappointed?
Well, maybe your expectations were a bit high then... Natural language processing is notoriously hard and, for that part that requires background knowledge, as good as impossible when working on single words or short phrases.
But in general the Kraaij-Pohlmann algorithm does a rather good job stemming Dutch words. Sometimes however, like with the word "meisje" it is a bit over-enthusiastic.
But there are a number of ways to improve stemming if, for some reason, the results of Kraaij-Pohlmann algorithms are not good enough.KeywordMarkerFilterFactory
The KeywordMarkerFilter makes it possible to exclude words from a (UTF-8) text file from stemming. A word like "meisje" would be a good candidate for this. To use it, add a filter to your fieldtype like this:<filter class="solr.KeywordMarkerFilterFactory" protected="notStemmed.txt" />
The file "notStemmed.txt" should be in the same directory as the schema.xml.StemmerOverrideFilterFactory
The StemmerOverrideFilterFactory is a variation on the KeywordMarkerFilterFactory filter, but instead only saying "do not stem these words" you must provide a file which defines the stemming for given words. To use it, add a filter to the field type like this:<filter class="solr.StemmerOverrideFilterFactory" dictionary="dictionary.txt" ignoreCase="true"/>
and make sure the file "dictionary.txt" is present in the conf-directory. In this file (which, like all others has to be encoded UTF-8) you add 1 ine per word, each lne consisting of the word an the stemmed word seperated by 1 tab, like this:hempje hemd
Both KeywordMarkerFilterFactory and StemmerOverrideFilterFactory should be used as addition to the default stemming,HunspellStemFilterFactory
Hunspell is the open source spellchecker used in a number of open source projects like LibreOffice, Mozilla Thunderbird etc.
It is possible to use Hunspell if it supports your language. To do so add a filter to the fieltype like this:<filter class="solr.HunspellStemFilterFactory" dictionary="nl_NL.dic" affix="nl_NL.aff" ignoreCase="true" />
And make sure the files "nl_NL.dic" and "nl_NL.aff" are present in the conf-directory.Creating your own stemming algorithm
Of course if you are really ambitious you can start from scratch and write your own Snowball implementation, from the Snwoball website:
Snowball is a language in which stemming algorithms can be easily represented. The Snowball compiler translates a Snowball script (a .sbl file) into either a thread-safe ANSI C program or a Java program. For ANSI C, each Snowball script produces a program file and corresponding header file (with .c and .h extensions). The language has a full manual, and the various stemming scripts act as example programs.
But be warned: no natural language or natural language phenomenon is easy to fit in an algorithm and you have to be sure to have all quirks and exceptions absolutely clear before you start.Links and literature
When choosing whether or not to have commenting on your site, one of the factors that I always discuss with clients is whether or not the site has a need for logged in users. If the site has users who log in, Drupal core's commenting system is great and can be tweaked to fit pretty much every need.
If users don't need to have users log in, there are a few solutions for "outsourcing" your comment system
- Using an external commenting system has benefits like:
- better security
- better/cheaper spam management
- more full page caching opportunities.
- Some popular commenting system providers are:
- Disqus (Drupal Module)
- Facebook comments (Drupal Module)
- Livefyre (Drupal Module)
I chose to go with Disqus for this site. Disqus' social login includes all of the necessary platforms. If not, people can sign up with an account in a minute. The design is light, and these days very common. Their spam detection, and moderation/flagging rules are simple and easy to maintain.
Cruising down the Amstel River on a blissfully warm Amsterdam evening – drink in hand – the fabulous FABIAN FRANZ (Senior Performance Engineer & Technical Lead, Tag1 Consulting) opens up about DrupalCons, chance encounters, caching, salsa dancing, love, and what to listen to when you’re programming.Tags: Video DrupalCon DrupalCon Amsterdam Video:
This module extends the CiviCRM Entity module by adding a contact's address, phone, and email details to tokens. You can specify the "location type" in the token for the address, phone, and email properties of a contact.Example
Notice the use of "main" in the token below:
Almost a year ago we started putting together a site that needed to integrate with our main librarie's search engine. We used Drupal's restful services to expose our content, but ran in to a problem with getting aliased paths to link up correctly. What this meant was that while http://www.bioconnector.virginia.edu/content/introduction-allen-mouse-brain-atlas-online-tutorial-suite worked fine, http://www.bioconnector.virginia.edu/content/introduction-allen-mouse-br... didn't... this became really problematic when we were trying to create linked data, and traversing was just obnoxious... https://www.bioconnector.virginia.edu/node/36.json just doesn't roll off the digital tongue... as a workaround we used views to do some wonkiness.... it worked, but certainly was not "the drupal way."
This module implements a Drupal field type that stores JSON or JSONB data. A core patch is required to get the data type supported at the moment.
Currently, only PostgreSQL database (version 9.2 or higher) is supported, but MySQL is adding support for JSON data types in the future.
In this article, we are going to look at how we can create a Drupal module which will allow your users to like your posts. The implementation will use jQuery to make AJAX calls and save this data asynchronously.Creating your Drupal like module
Let’s start by creating the new Drupal module. To do that we should first create a folder called likepost in the sites\all\modules\custom directory of your Drupal installation as shown below:
Inside this folder, you should create a file called likepost.info with the following contents:name = likepost description = This module allows the user to like posts in Drupal. core = 7.x
This file is responsible for providing metadata about your module. This allows Drupal to detect and load its contents.
Continue reading %Drupal goes Social: Building a “Liking” Module in Drupal%
I want to give a big thank you to all of our new and renewing members who gave funds to continue the work of the Drupal Association in the first quarter of this year. We couldn't do much without your support. Shout outs to all of you!Membership Makes a Difference
We had several recap blog posts a few weeks ago, but just as a reminder, your membership is incredibly important not only to us, but to the project too! Dues from memberships go to fund intiaitves like our community cultivation grants, which help people around the world build their local Drupal communities and improve the project. For more information on how membership makes a difference, check out this infographic or see what changes are coming in 2015.Thank You, Members!!
There are 845 fantastic members on our list of first quarter donors. You can find the list here. Let's give them a big thank you all together!
Personal blog tags: Membership
DrupalCons are a very important time of the year for the Drupal community. It is a time for us to come together and continue our collaboration that we share throughout the year in a virtual space and establish goals and plans to move forward in a way that is in the community’s best interest. It is also a time to take stock of our accomplishments, and who we are as a community. One of our favorite moments in DrupalCons is the group picture: it’s always amazing to see how the community stands together and continues to grow.
This year’s DrupalCon in Los Angeles is especially important to us because this is where we will unveil the name of the new Drupal agency that consists of the companies formerly known as Blink Reaction and Propeople. We have come together to create a new digital agency, the largest one in the world with a focus on Drupal, and we are very excited about what this means.
Our combined agency is part of the Intellecta Group (listed on the NASDAQ OMX) , and consists of 400+ employees in 19 offices in 11 countries on 4 continents. This is an amazing reach for an organization that is so passionate about Drupal! We’re excited for this unique opportunity to support the Drupal project and the community in ways that would have been impossible prior to the merger.
For example, we’re eager to begin promoting Drupal as a solution for the biggest enterprises on a global scale. Locally, we can influence awareness and excitement in our 19 local communities, helping the next generation find opportunity and excitement in Drupal.
We now have the ability to affect change in a multitude of cultures, in the many diverse communities where each of our offices are located. Where there aren’t yet camps, we can lead their initiation. Where there are Cons, we can help to inspire the next generation of Drupal leaders. We are committed to building up the next generation of talent via our orchestrated public and private training efforts, and look forward to beginning that work at DrupalCon Los Angeles.
So please, stop by booth 300 to say hello and learn more about the new company, and our future within the community. We look forward to seeing all of our friends in the Drupal community, old and new, and are even more excited to discuss how we’ll work with the community for many years to come.
Blink Reaction and Propeople are joining forces to create a new digital agency built on technology, driven by data, and focused on user experience. The two companies have delivered state-of-the-art Drupal solutions for a variety of the open-source platform’s largest customers. The agencies’ collective portfolio includes brands such as Pfizer, NBC, Stanford University, the City of Copenhagen and General Electric.
Blink Reaction and Propeople are a part of the Intelleca Group. The companies in the group are Blink Reaction LLC, Bysted AB, Bysted A/S, Hilanders AB, Intellecta Corporate AB, ISBIT GAMES AB, Propeople Group ApS, Rewir AB, River Cresco AB, Unreel AB and Wow Events AB. Intellecta AB is noted on NASDAQ OMX Stockholm and employs around 550 people in Sweden, Denmark, Austria, Germany, the Netherlands, the United Kingdom, Bulgaria, Moldavia, Ukraine, Brazil, USA, Vietnam and China.