Upgrading A Drupal 7 Module to Drupal 8: Using Composer

Drupal 8 Composer File

Of the many new and shiny changes included in Drupal 8, one of the most significant additions in the spirit of "getting off the island" is using the Composer tool. Composer is a package manager that uses packagist.org to add and update your project's dependencies. 

In this post, I will go through my perspective of transitioning from the Drupal 7 way of adding external dependencies to the way things work in Drupal 8 with Composer. At the end of the post, I also go about the process of switching a site installed via cloning Drupal 8's Git repo from the project page to using Composer to manage its contrib dependencies. In setting up your site, you'll have to use "composer install" for the initial core dependencies, but you won't automatically be able to install contrib modules via Composer until you perform the final steps.

I won't go into to explaining the particulars of Composer or package management so you should probably read a primer before this post if you aren't familiar with any of those concepts.

Libraries

In previous versions of Drupal, outside dependencies other than Drupal core and contrib modules mainly lived in the libraries directory. The standard routine was to download a contrib module wrapper that told Drupal where the library was located, provided any configuration options for that library, and also potentially provided an API for other contrib modules to interact with. Some modules provided business logic on top of an external library, e.g. the MailChimp module, and you could simply use that module and library as a one-stop solution for a part of your application, e.g providing a newsletter to your subscribers.

While many modules used the library system to load only JS or CSS assets, the MailChimp module is a good example to use when talking about Composer since they include a PHP library with the module. In fact, the 7.x-4.x version of the module uses Composer to install some of its dependencies.


/**
 * Implements hook_libraries_info().
 */
function mailchimp_libraries_info() {
  $libraries['mailchimp'] = array(
    'name' => 'MailChimp API',
    'vendor url' => 'https://github.com/thinkshout/mailchimp-api-php',
    'download url' => 'https://github.com/thinkshout/mailchimp-api-php/archive/v1.0.5.zip',
    'version arguments' => array(
      'file' => 'composer.json',
      'pattern' => '/"version": "([0-9a-zA-Z.-]+)"/',
    ),
    'files' => array(
      'php' => array(
        'src/Mailchimp.php',
        'src/MailchimpAPIException',
        'src/MailchimpCampaigns.php',
        'src/MailchimpLists.php',
        'src/MailchimpReports.php',
        'src/MailchimpTemplates.php',
        'vendor/autoload.php',
      ),
    ),
  );

  return $libraries;
}

As you can see in the example above and read more about hook_libraries_info(), the important parts of the declaration are the name of the library, the location of the source code, and the specific version of the library that should be included with the module. These keys map well to the same information that Composer needs to build a dependency tree for your whole Drupal project. 

D8 Composer Status

I'm not even going to pretend like I know the skinny on how best to use Composer within Drupal 8. When I've been checking in periodically to see how I should be doing Drupal 8 development, it seems like best practices for using Composer seem to always be changing, although I think there is an end to major changes in sight.

As a warning, there is still a lot of outdated and conflicting information out there so watch out. It's probably best to stick to drupal.org articles, like "Drupal.org's Composer endpoints are out of beta", or issue queues for updates and to pay strict attention to posting dates...so, ironically I guess I'm telling you to not trust what I write here a few months down the road, but I think it is a decent warning since some articles last updated in December 2016 are already marked out of date and conflict with or duplicate information contained in the same documentation section. 

One attempt at adding Composer support to Drupal modules came with the Composer Manager module. The module worked by allowing every contrib module to be packaged with a composer.json file that the Composer Manager module aggregated into a canonical manifest file located in "sites/all/libraries". Then, all a developer would have to do is go to the libraries directory and run "composer install" or "composer update" to install the desired dependencies.

That approach works fine for Drupal 7 where no other composer.json files exist; however, it violates the "one composer file per application rule" when Drupal 8's core composer.json file is included. So, the Drupal 8 version of the Composer Manager module has been deprecated for awhile. 

In its place, Wikimedia's Composer Merge Plugin was adopted. The merge plugin allows for "easier dependency management for applications which ship a composer.json file and expect some deployments to install additional Composer managed libraries." It is quite fitting for Drupal to adopt this approach since the merge plugin was born out of figuring out how to install the Mediawiki project which has the concept of "extensions" similar to the contrib module ecosystem.  

Top-level Composer Manifest File

Since Drupal 8 comes with two composer.json files, there are a few parts of the top-level composer.json file worth mentioning. The first thing to note is the name of the package.


"name": "drupal/drupal",
"description": "Drupal is an open source content management platform powering millions of websites and applications.",
"type": "project",
"license": "GPL-2.0+",
"require": {
    "composer/installers": "^1.0.21",
    "wikimedia/composer-merge-plugin": "~1.3"
},

Those lines are the basic ones you would want to include in any project you would send up to packagist.org, although the type and license keys are optional. Once you scroll a little bit below those lines, you then see...


"replace": {
    "drupal/core": "~8.2"
},

I scratched my head at this line for a little while. Why would Drupal want to replace itself, I thought? Composer's documentation on the replace option didn't help either.

"This is also useful for packages that contain sub-packages, for example the main symfony/symfony package contains all the Symfony Components which are also available as individual packages. If you require the main package it will automatically fulfill any requirement of one of the individual components, since it replaces them."

I thought that since Drupal contains many sub-systems it might be following the Symfony example; however, I still didn't quite understand what those two sentences meant. Things started to click for me when I read the "extra" part of the file which contains the Composer Merge Plugin information. 


"extra": {
    "_readme": [
        "By default Drupal loads the autoloader from ./vendor/autoload.php.",
        "To change the autoloader you can edit ./autoload.php."
    ],
    "merge-plugin": {
        "include": [
            "core/composer.json"
        ],
        "recurse": false,
        "replace": false,
        "merge-extra": false
    }

Within "core/composer.json", you'll find the package listed as '"name": "drupal/core"'. So each point release of Drupal will update the version it replaces, e.g. "'drupal/core': '~8.3'" for the next release, and when "composer update" is run everything in the drupal core namespace will be updated to 8.3.x. There is still work to do here with splitting out Drupal's components so they could be installed separately. So for now, while "drupal/core" replaces a lot of "drupal/*" packages, you pick and choose to install the whole thing or components like with Symfony components.  

In my developmental future, I'm sure I'll add to the merge plugin part to include a profile or potentially for a custom module I'm developing. To do that you would add something like:


    "merge-plugin": {
        "include": [
            "core/composer.json",
            "profiles/my_profile/composer.json",
            "modules/custom/my_module/composer.json",
        ],

However, this syntax might change as issues regarding things like profiles and distributions are still being fleshed out. 

You'll also notice a composer.lock file in the root directory that lists all of the packages with explicit versions to install next to them. You can look at that file for the canonical source of package versions installed on your project, and it will be updated every time you run "composer update", if a package is updated. When tracking down a bug, it is best to look at the lock file as it will give you exact version numbers and way more info that a composer.json file will.

Drupal Core Composer Manifest File

The meat and potatoes of dependency management for Drupal core lies in the "core/composer.json" file. In the "require" section, you'll see all of the Symfony components that Drupal now uses as well as some other general PHP libraries.

Along with managing packages, Composer also helps with autoloading classes. You can see this in action under the "autoload" key.


"autoload": {
    "psr-4": {
        "Drupal\\Core\\": "lib/Drupal/Core",

The "use Drupal\Core\xxx" statements get loaded by searching for that class in the "/core/lib/Drupal/Core/xxx.php" file. Compared to Drupal 7, Drupal 8 has a lot more parts and pieces loaded from different places. I luckily use an IDE, PHPStorm, that allows me to jump straight to the definitions of classes and functions I need more information about, but you should be able to follow the pattern above to find the source code you're looking for related to namespaces. 

Converting Your Site to Use Composer Endpoints

Now that we understand a little more about how Drupal 8's Composer system is setup, we can complete the steps necessary to install and keep our modules up-to-date via Composer instead of downloading modules the old-fashioned way.

To keep your Drupal core dependencies up-to-date you can user composer as-is. The next time you "git pull" and changes are made to the "core/composer.json" file, you can run "composer update" to install the new versions of packages. Your terminal output will look something like the following:

Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Removing behat/mink (v1.7.1)
  - Installing behat/mink (dev-master bb28774)
    Downloading: 100%         

> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
  - Removing symfony/dom-crawler (v3.2.1)
  - Installing symfony/dom-crawler (v2.8.15)
    Downloading: 100%         

> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
Writing lock file
Generating autoload files
> Drupal\Core\Composer\Composer::preAutoloadDump
> Drupal\Core\Composer\Composer::ensureHtaccess 

The step you need to require modules as packages is actually quite simple as described in this documentation page. Simply type

$ composer config repositories.drupal composer https://packages.drupal.org/8

into your terminal while in the root directory.

To test out installing modules this way, I chose the token module ("composer require drupal/token") which downloaded token 8.x-1.x-dev to my "/modules/contrib" folder. I was a little surprised to get the dev release of the module when there was an 8.x recommended release; however, switching module versions is pretty easy to do.

As mentioned in a recently edited Composer documentation page, you can simply append the version of the module you want to download to your Composer command:

$ composer require drupal/token:^1.0.0

Since I am trying to keep as current with Drupal 8 development as possible, I'll switch back to the dev branch, but in most cases, you'll want to use a range operator between at least minor versions so you can keep your contrib modules as up-to-date as possible, including security releases, while keeping in mind backwards compatibility/breaking changes.

It is also a good idea to use "composer outdated" before blindly updating your modules. That command will give you a list of packages with the version currently installed and then the most current version. For any module you are currently developing with dependencies, it would be a good rule of thumb to check for outdated packages before a release. Otherwise, you would have no good way of identifying how packages you depend on are changing. You could also spot an outdated package for another contrib module you depend on and help that maintainer keep their module up-to-date.  

Also mentioned in the Composer post linked above is the ongoing issue of dealing with semantic versioning in Drupal 8 modules. Since a core prefix (7.x vs 8.x) is used for versioning purposes, this puts Drupal outside of the realm of a lot of other semantically versioned projects. Even though 7.x modules generally have a pattern of 7.x-1.3 instead of 7.x-1.3.2, it is easy to shim 7.x-1.3 to 7.x-1.3.0 as they are doing with the custom Composer endpoints. However, what to do with the Drupal version prefix is trickier. You can follow that discussion here: https://www.drupal.org/node/1612910

A Note About .gitignore

Drupal 8 now ships with an example.gitignore file. I find this file super useful since I like to keep all of my projects in version control. If you initialize a Git repo before you have a .gitignore file or if you have one not ignoring the vendor directory, then you can end up committing changes you made for development and pushing them up to your live site. Since I am new to using Composer and have been burned by blindly trusting tools like Homebrew before, I am wary of committing a lot of changed code I know nothing about. 

It is a best practice to NOT include the vendor directory in your Git repository as it can lead to a few issues, and for me, it just makes getting into Drupal 8 development slightly less confusing.