Planet Drupal

What is Burnout?

Submitted by rfay on

This article is one of a series in preparation for the Burnout core conversation at Drupalcon London. This article will discuss what burnout is; future posts will cover how individuals can respond to burnout, what about burnout is unique to the Drupal community, and what the Drupal community can do to deal with burnout.

What is burnout? Many of us "just know" the answer to this, as we've we've struggled with it ourselves in our professional, personal, or community work. I found a couple of high quality discussions of burnout. The first is the book The Truth about Burnout: How Organizations Cause Personal Stress and What to Do About It1, written by the leading researcher in the field, Christina Maslach. The second was the seminal article Burned Out2 from Scientific American Mind magazine.

First, a more formal definition of burnout from Preventing Burnout3:

Burnout is a state of emotional, mental, and physical exhaustion caused by excessive and prolonged stress. It occurs when you feel overwhelmed and unable to meet constant demands. As the stress continues, you begin to lose the interest or motivation that led you to take on a certain role in the first place.

Burnout reduces your productivity and saps your energy, leaving you feeling increasingly helpless, hopeless, cynical, and resentful. Eventually, you may feel like you have nothing more to give.

Most of us have days when we feel bored, overloaded, or unappreciated; when the dozen balls we keep in the air aren’t noticed, let alone rewarded; when dragging ourselves out of bed requires the determination of Hercules. If you feel like this most of the time, however, you may be flirting with burnout.

Maslach describes three dimensions of burnout:

  1. Exhaustion: Feeling overextended, both emotionally and physically
  2. Cynicism: Taking a cold, cynical attitude toward responsibilities.
  3. Ineffectiveness: When people feel ineffective, they feel a growing sense of inadequacy.

She goes on to describe these causes of the syndrome:

  1. We feel overloaded.
  2. We lack control over what we do.
  3. We are not rewarded for our work.
  4. We experience a breakdown in community.
  5. We aren't treated fairly.
  6. We have to deal with conflicting values.

The conventional wisdom is that burnout is primarily a problem of the individual: People burn out because of flaws in their characters, behavior, or productivity. In other words, *people* are the problem, and the solution is to change them or get rid of them.

But our research argues most emphatically otherwise. We believe that burnout is not a problem of the people themselves but of the *social environment* in which people work. The structure and functioning of the workplace shape how people interact with one another and how they carry out their jobs.

The Scientific American Mind article describes twelve stages of burnout.:

  1. A compulsion to prove oneself (demonstrating worth obsessively) "It tends to hit the best employees, those with enthusiasm who accept responsibility readily."
  2. Working harder and longer; an inability to switch off.
  3. Neglecting our own needs (erratic sleeping, eating disrupted, lack of social interaction)
  4. Displacement of conflicts (problems are dismissed, we may feel threatened, panicky and jittery)
  5. Revision of values (values are skewed, friends and family dismissed, hobbies seen as irrelevant, work is only focus)
  6. Denial of emerging problems (intolerance, perceiving collaborators as stupid, lazy, demanding, or undisciplined, social contacts harder. Cynicism, aggression. Problems are viewed as caused by time pressure and work, not because of life changes)
  7. Withdrawal (social life small or non-existent, need to feel relief from stress, alcohol/drugs)
  8. Odd behavior (changes in behavior obvious, friends and family concerned)
  9. Depersonalization (Seeing neither self nor others as valuable, and no longer perceive own needs.)
  10. Inner Emptiness
  11. Depression: Feeling lost and unsure, exhausted, future feels bleak and dark
  12. Burnout Syndrome: Can include total mental and physical collapse. Time for full medical attention.

This has been a quick list-oriented discussion of burnout, but the book and article I used as sources are recommended.

In the next post, we'll talk about what individuals can do about burnout.

Resources

  1. The Truth about Burnout: How Organizations Cause Personal Stress and What to Do About It, by Christina Maslach and Michael P. Leiter, Jossey-Bass Publishers, 1997 ↩︎

  2. Scientific American Mind, June/July 2006, "Burned Out", by Ulrich Kraft ↩︎

  3. Preventing Burnout: Signs, Symptoms, Causes, and Coping Strategies ↩︎

What Can Individuals Do To Prevent Burnout?

Submitted by rfay on

This post is one of a series in preparation for the Burnout core conversation at Drupalcon London. The first post discussed what burnout is; this one talks about what individuals can do about burnout, and the final two will discuss what about burnout is distinctive to the Drupal community, and what the Drupal community can do to deal with burnout. Also, please see Arianek's excellent post about burnout

There are dozens, even hundreds of books addressing individuals and burnout. I was amazed how many of them are aimed squarely at church members, pastors, and other caring professionals. Interestingly enough, the Drupal community can be a bit like many churches: plenty of needy people, lots of ideas, an amazing amount of work that "ought" to be done, and most of it done by volunteers or vastly overloaded professionals. These books have pretty uniform advice to the individual for dealing with burnout, and of course it's what you already know:

  • Don't just work all the time, regardless of the load. Enforce your work hours.
  • Prioritize. Take on reasonable obligations that you can meet.
  • Take time off.
  • Develop and love hobbies besides Drupal.
  • Socialize.

From Scientific American Mind1:
1. Budget your physical resources. Eat wholesome foods at mealtimes. Exercise regularly. Get enough sleep.
2. Workaholics must aim for equilibrium between tension and relaxation. "Find your work-life balance".
3. Cultivate close social contacts. Spending time with friends, family, or even colleagues at work protects against excessive stress.
4. So Important: The decisive step must be made first in your head: "As early as possible in your career, you must absorb the idea that physical and mental health are at least as important as anything you can do seeking success at work".

Pretty easy sounding, huh?

Next time we'll talk about things from an organizational perspective: what is distinctive about Drupal and burnout?

  1. Scientific American Mind, June/July 2006, "Burned Out", by Ulrich Kraft ↩︎

How Does the Drupal Community Burn People Out?

Submitted by rfay on

This post is one of a series in preparation for the Burnout core conversation at Drupalcon London. The first post discussed what burnout is; the second is about what individuals can do about burnout, and the final one will discuss what the Drupal community can do to prevent and deal with burnout. Also, please see Arianek's excellent post about burnout.

One of the striking things about the Drupal community is that so many of us have such loose or nonexistent boundaries between work and home, Drupal and work, hobbies and work. And, being techies, we're always connected. The result is that Drupal can dominate every aspect of our lives. Not so good.

Let's revisit the six causes of burnout from The Truth about Burnout: How Organizations Cause Personal Stress and What to Do About It1. The author of that book, Christina Maslach, a leading researcher in the field, makes the bold claim that burnout is in many ways a result of organizational choices, not just individual ones.

  1. We feel overloaded. Many, many of us are helpless suckers for love and approval, and the Drupal community has an infinite number of tasks that need to be done that we can do. There's an opportunity for approval and an existing need at every bend. It's infinite. We're not.
  2. We lack control over what we do. Many aspects of Drupal (need I name core work?) involve an enormous lack of control over the final result, many times due to our community-collaboration culture. One can participate in an issue for years before it finally lands. A huge discussion (sometimes ending in deadlock) may be required before something can get done that an individual could just take on and do.
  3. We are not rewarded for our work. While the Drupal community is quite good at acknowledging efforts, there are certainly many who do not feel adequately acknowledged (or compensated) for their superhuman efforts.
  4. We experience a breakdown in community. Although we value the vibrance of our social community, it's not all roses. Many, many newcomers feel that it's hard to break in, and others feel slighted or off-put by the difficulty of actually getting involved. Note also the breakdowns we saw during the final days leading to Drupal 7's release. Some key contributors were making hopeless, sad statements demonstrating a kind of breakdown at a critical time.
  5. We aren't treated fairly. Many of us have nearly given up after being lambasted by some more senior community member (who was probably burned out). Many of us have seen contributions languish despite the best efforts. The rather chaotic and often unwritten rules in the community can seem impossible to figure out. How do I get somebody to answer the issue I filed? How do I communicate with a project maintainer? What happens if they don't answer? How do I get somebody to acknowledge me on IRC? (Suggestion: We can make our hierarchy, organization, and rules more explicit.)
  6. We have to deal with conflicting values. In the Drupal community, the tension between work responsibilities, economic realities, community contribution desires and commitments, and quality is always at play. This is quite a lot more complicated than ordinary work tensions.

In addition, there are many examples of explicit choices we've made that cause community burnout. They're mostly procedures that could never be accomplished even with an army of paid help:

  • Module contributors automatically become module maintainers. Their contribution innately adds to their future responsibilities.
  • There is no explicit "way out" of community responsibilities even though the DCOC says "step down gracefully"
  • We have created some tasks and processes which are innate tarpits. For example, our documentation team is hopelessly overworked, with no known path to sustainability, and the Full Project Application process has never been sustainable. (There are more than 200 full project applications in "needs review", the oldest untouched for 8 weeks. There are 15 applications in RTBC, one for 7 weeks.)

I'm sure you have other examples of choices we make in our procedures that overload community members - please add them in the comments.

  1. The Truth about Burnout: How Organizations Cause Personal Stress and What to Do About It, by Christina Maslach and Michael P. Leiter, Jossey-Bass Publishers, 1997 ↩︎

What can the Drupal Community do about Burnout?

Submitted by rfay on

This post is the last in a series in preparation for the Burnout core conversation at Drupalcon London. The first post discussed what burnout is; the second is about what individuals can do about burnout; the third was about how the Drupal Community burns people out.

The key point of this series is that we need to do strategic thinking as a community about how to treasure and keep our wonderful contributors. That's the point of the core conversation. It will not be a support group for people to share their burnout stories, but a gathering to think strategically about how the community can prevent or heal burnout to protect and keep our members.

We must value, keep, and grow our contributors. Burnout is not a problem we can ignore. It costs incredible productivity, of course, when a contributor becomes contribution-disabled or disappears. It actually costs huge sums of money to all the economically invested members of the community. But most important, we as a community care about our members and don't want to see them damaged by the fact that they care and invest themselves.

To begin the conversation, here is some brainstorming (thanks for your notes @Bojhan!). Here are some free-form thoughts to prime the pump for our core conversation:

  • Explicitly and deliberately eschew unsustainable processes and projects: We have a number of projects that seem to only increase disillusion and hopelessness because they're unsustainable the way we currently look at them. We'll have to figure out how to define what "unsustainable" is, of course.
    • The docs and UX projects are potential examples. What do we have to do to change these tasks to make them sustainable?
    • The full project application process currently has applications in "Needs review" that go back many weeks, and even a significant backlog of RTBC applications that have not been promoted. What do we do to make this sustainable?
    • Creating a module or theme implicitly makes you the owner/maintainer, multiplying the commitments (or guilt level) of our most prolific contributors. Can we find a way to do this differently?
  • There are some places where we can deliberately increase the resources available and resolve the strain that way. In the Drupal.org redesign, which was mired in never-never land for a long time, money, organization, and paid work were leveraged to lead to a (very) successful conclusion. Money is an easy answer, but there are many other ways to increase resources for a project. What are they?
  • Does Drupal need a human resources department? A human resources initiative leader?
  • Develop explicit replacement strategies for people. Provide a "recycling" area or forum where people can explicitly publicize responsibilities they're trying to get rid of.
  • Promote and publicize the personal side of burnout management, as discussed in the first couple of posts here.
  • Promote balance in community responsibilities.
  • Learn from what we do right.
    • We do much right with contrib modules. Maintainers are enabled to do what they need to do with their modules. Others are enabled to get involved through the issue queue. This is all good. What are other examples of what we do right?
    • The new D8 initiatives, with lieutenants and more explicit decisionmaking authority, may be a real step forward in dealing with the "lack of control of our work" issues.
  • Recognize that enabling others requires giving up some control. Bring on co-maintainers, for example. Mentor them. But let them work. Some will stick. Module owners may need to be more liberal in granting commit access.
  • Make it clear to all that there are alternatives to withdrawing from the community. Publicize those alternatives.
  • Establish communication channels amongst community leadership, develop leadership support groups.
  • Understand that timely communication is half the work. Often people communicate too late, requiring passionate people to do damage control rather than creating something great.
  • Increase the visibility of “coordination” efforts such as docs team lead, git migration lead, etc..
  • Say "No" when necessary. If we can't be successful individually or as a community and retain our balance, let's not undertake the project.
  • List and study the areas where a lack of control by contributors leads to ineffectiveness and burnout.
  • Cultivate the resilience and health of our overall community. We've long valued civility. Let's grow that civility and community health. We need to commit to courtesy, civility, and valuing our community. Every aggressive tweet will hurt somebody's feelings. Promote civility. Even on Twitter.

A last note: Burnout as Opportunity

We should remember that people wearing out on their projects is actually an opportunity. If we can get them moved to new things they can be more effective and motivated, and if we can get new people into the burned-out areas, we can get better maintenance there. If people know they can move around, they can be refreshed. New blood can be inserted into key tasks.

Please comment with your ideas

In preparation for the core conversation, please post ideas here that can be included on Monday, whether you can be there or not.

Drupal Community Conflict Resolution (and Twitter)

Submitted by rfay on

We value our community like a family, and have put loads of energy into the Drupal world. But we have some problems, and it's probably time for some discussion about our community manners and approach to problem-solving. We have the excellent Drupal Code of Conduct (copied from Ubuntu's model) but it doesn't yet address a number of specifics (especially conflict resolution). So I'd like to throw some ideas into the wild.

First of all, Controversy is not a bad thing. It's not a problem when people disagree on issues, even if they permanently disagree. It is a bad thing, though, when people start campaigns (on Twitter or otherwise) without understanding the problem or trying to resolve it directly. Feelings can be hurt, the community can be divided, contributor energy can be sapped, and it's mostly not the most likely way to resolve the situation anyway. Why do that?

Name Calling, Personal Attacks, and Twitter Pile-ons are not the way to resolve problems in the community. They just create new problems. I recently wrote and spoke about Burnout in the community and many people responded favorably - it seemed to strike a chord with the community, because everybody knows how dispiriting it is to get worn down with responsibilities. But if we add to that hurt feelings (and worse) due to misunderstandings, rudeness, and the like, we end up with something perhaps worse than burnout.

I propose that we discuss some additions to the Drupal Code of Conduct, either formally or informally.

  • When you see something that you think is wrong (or that you otherwise disagree with), approach the party responsible directly rather than making an immediate public statement.
  • Try to understand and resolve the situation directly with the parties directly involved. A discussion may help to
    • Understand the actual issue.
    • Understand the history behind the issue.
    • Have a complete understanding of the facts involved.
    • Understand how each party understands those facts
    • Understand the emotions involved involving each party.
    • Figure out a course of action to resolve the problem.
  • If direct communication fails, pursue formal or informal community conflict resolution channels. Let's establish a conflict resolution methodology. For starters, we have a number of trusted and level-headed community members who might be willing to serve as moderators or arbitrators.

I don't think that Twitter and related fast-communication techniques are bad for general discussion or notification, but they're really lousy for actually solving (or even understanding) real problems. Let's tone down the public pile-on (and related public attacks) and find more successful ways to actually get something done. And let's all just review the DCOC, especially Be Considerate, Be Repectful, Be Collaborative, When We Disagree We Consult Others. Oh, that's almost the whole thing, isn't it :-)

I'll write soon proposing some first ideas with an actual conflict resolution technique.

Happy Notes! (About *not* burning out!)

Submitted by rfay on

As you know, I've been writing and talking and thinking about burnout lately.

One of the key causes of burnout is the situation where we lack control over what we do.

For a long time, it seemed impossible to move Drupal.org (or our workflow or methodologies) forward. It just seemed like we were stuck with what we had. We lacked control over our website and our community processes.
Recently, though we've come light-years from the old ways. Incredible improvements to longstanding problem areas like the Follow! Instead of subscribe comments, Deployment of BUEditor on d.o, API Change Notices, Issue summaries, testing contrib projects with dependencies and Images for all on drupal.org demonstrate that we do have control of our work environment and our territory. What great news! In addition, progress in the docs team toward a more sustainable strategy (with curated and uncurated docs) is a huge step forward.

The improvement of Drupal processes and d.o features can and will go a long way toward improving the health of our community and avoiding burnout. Congratulations to all who have led this sea-change in how our community works.

And if you want to participate in making Drupal.org awesome, read webchick's post. Hurry!

Drush, File Permissions, and Web Servers - Lots of things can go wrong

Submitted by rfay on

The marvelous drush makes an enormous number of things possible that would otherwise be tedious at best, and allows us to automate a shocking number of Drupal tasks.

But it has a dark side. Drush does tasks that would normally be done by the web server (cron, update.php, enabling modules), and that means that it can leave files laying around that the web server may not be able to access and control. It also does jobs that would normally be done by a site maintainer (drushpm-download, drushpm-updatecode) from the command line. These should create files that are not writeable by the web server.

The results of this dark side can cause enormous confusion to Drupal developers. If you do a drush cron as yourself and cron happens to write some files in sites/default/files, (for example, updating CSS or JS cache files), those files will never by default be deletable or possibly editable by the web server, meaning that Drupal will never be able to properly handle those files. The flip side is that if you do a command like " drush cc all" as yourself your drushcommand will not be able to delete or possibly edit files that were created (normally) by the web server.

I have debugged dozens of broken websites with subtle breakage (most of them my own) that were broken in this way.

In the future, I'd hope to see drush figure out how to categorize which of its jobs should be done as the web server user and which should be done as a non-web server user, and offer to do that for you (using sudo). But right now, here is some background and some workarounds to go with that background.

(Note: The problems described here are common to almost all Unix/Linux servers and the Mac, but usually do not affect Windows, CPanel, or Dreamhost shared hosting users. That's because traditional Unix servers run the web server as a separate user ("www-data" on Debian/Ubuntu, "apache" normally on RedHat/CentOS.)

Concepts

  • In Linux, every file and directory has an "owner" and a "group". Every user has a primary group. And every file has read, write, and execute permissions for owner, group, and all.
  • When a Linux user creates a file or directory, that file or directory by default gets the "owner" of the user that created it and the "group" of the primary group of the user that created it.
  • Only a user with permissions to write in a directory has permissions to delete or rename a file.

Workarounds

  • Add yourself and anybody who might use drush to the web server's group (www-data on Debian/Ubuntu). This will give you permissions, when using drush as a normal user, to update files created by the web server (but you still won't be able to delete files like CSS aggregation files created by the web server)
  • Please, please never do a "sudo drush" or run drush as root. This can leave scattered files laying around that neither you nor the web server can access. If you think you need to use sudo with drush something else is badly wrong. Unless, of course, you manage all your sites and files as root, which is another huge mistake from a system reliability and security perspective.
  • When issuing drush commands that do things the web server would normally do (cron, updatedb, enable, disable, cc, running simpletest tests) use sudo -u [web server_user] drush [whatever], which runs drush as the web server user. So, on Debian/Ubuntu: sudo -u www-data drush cc all
  • Make the files directory (and private files directory) and everything below them have the group of the web server and be writable by the web server user:
    sudo chgrp -R www-data sites/default/files # Make all files have group www-data
    sudo chmod -R ug+rw sites/default/files #Grant write permission to user and group recursively
    sudo chmod g+s sites/default/files # Set the SGID bit so all files created inside the files directory will have group set for the files directory
  • Use this excellent technique to make sure the web server user's umask (default file permissions) are set to allow the group user to edit files.

Some examples of failures

OK, maybe you're not taking me seriously :-)

  • Install Backup and Migrate and install it for the first time using drush. drush en -y backup_migrate and then do a manual backup drush bb. By doing those two things you have created the backup_migrate directory... but with your own permissions. Backup Migrate can never write to that directory, and all accesses by the web server will fail. And generally we want backups to succeed. :-)
  • Use drush cc css+js to to clear all js and css aggregation files in sites/default/files/css, etc. If the files there are more than 30 days old (D7), you'll get these errors and the files won't be deleted, and sadly you'll see these messages forever, every time you clear the cache or do any action that clears the cache, until you go in and delete the files or solve the permissions problem. This is no great tragedy, but it just demonstrates what we're talking about:
    unlink(/home/quickstart/websites/example.dev/sites/default/files/css/css_0HZDHWccQCzJ2oHZ0hVqMjDA4BAT77eUWYWDqQbAEM0.css):[warning]
    Permission denied in drupal_unlink() (line 2139 of
    /home/quickstart/websites/example.dev/includes/file.inc).
  • Install CSS Injector using drush (drush dl css_injector; drush en -y css_injector</code). Now when you go to try to create a CSS rule (which creates a file in the directory sites/default/files/css_injector, which the web server doesn't have write access to, you'll get
    Error message
    The directory /home/quickstart/websites/example.dev/sites/default/files/css_injector is not writable

These are just a few examples of unintended consequences of of using drush without understanding what happens with file permissions. There are many more.

If you use the workarounds described above, you can ameliorate all of these. If you don't, you can debug forever or be dogged by these problems forever.

Resources on drush:

  • drush.ws is the Drush documentation on a website.
  • This issue describes a partial approach to the permissions problem.

Armageddon

OK, I might have overstated the consequences. Not Armaggeddon, but really important to the stability of your website.

Please do leave your prescriptions for these issues in the comments!

Drupal Deployment with Git Submodules

Submitted by rfay on

[Update October 2013: Warning: While I found submodules extraordinarily easy to use, especially when contributing back changes, I failed at nearly every turn to successfully involve other people with them. They always had unnecessary technical problems. So I have reverted back to a single repo for the entire project.]

(There are screencasts illustrating these concepts at the bottom.)

I've been using git submodules for Drupal site deployment for quite awhile now, and I wanted to share my experience (for good and bad) with others who might be interested in using submodules (and to solicit comments from those who know how to do it better).

What are git submodules?

First "git submodules" have nothing to do with Drupal modules. They're a way of including one git repository inside another, in its entirety, and managing it from within the parent repository.

In a regular git repository, you have one .git directory in the root directory of the project that contains the entire history and all the revisions for that project. Every change in the project is tracked in that one .git repository.

Using git submodules you have one or more sub-repositories of the main one. They look exactly like a regular git repository, but the parent repository knows just enough about them to manipulate them.

For example, if we're using the regular Drupal checkout as our main repository and checking out Drupal modules as submodules, we might do this:

git clone git://git.drupal.org/project/drupal.git --branch 7.x
git submodule add --branch 7.x-1.x git://git.drupal.org/project/examples.git sites/all/modules/examples
git submodule add --branch 7.x-3.x git://git.drupal.org/project/admin_menu.git sites/all/modules/admin_menu

This would check out Examples and Admin Menu into sites/all/modules in the Drupal tree, but as real first class git repositories (they have their own .git directories), but the parent repository (the Drupal repo in this case) knows what they are and what to do about them. I can now use all the git tools I can think of in the Examples or Admin Menu directories, including pulling (to follow a tracking branch), git bisect to debug, git checkout (to get a particular tag/version). I can also create a patch after solving a problem with the regular git tools and I can apply a patch with git apply in those directories, all without thinking twice. There are lots of reasons this appeals to me.

Practical deployment with git submodules

Once we have a repository that contains submodules, we can push it to a remote, as with any git repository. Note that we're not pushing the Drupal module submodules though (as in most cases we don't even have commit privileges on them).

When building a site based on Drupal I usually do this:

# Clone drupal into a directory "mysite"
git clone -b 7.x git://git.drupal.org/project/drupal.git mysite
cd mysite

# Create a new branch for our site deployment work
git checkout -b site

# Add a couple of submodules to it
git submodule add --branch 7.x-1.x git://git.drupal.org/project/examples.git sites/all/modules/examples
git submodule add --branch 7.x-3.x git://git.drupal.org/project/admin_menu.git sites/all/modules/admin_menu

git commit -m "My site is ready to go"

# Now change the remote named "origin" to "drupal"
git remote rename origin drupal
# and create a new remote that points to my repository
git remote add origin git@git.myrepo.example.com:mysite.git
git push origin site  # Push the thing up there.

Now I have a local site that's all ready to go, and has been pushed to my private repository (or github, or whatever).

I can deploy this site on a server by just cloning it and then updating the submodules. The fact that you have to do two discreet steps is in fact unfortunate and it is easy to forget after cloning or pulling. But here's the process for deploying from the new private repository that we pushed to:

git clone --branch site git@git.myrepo.example.com:mysite.git
git submodule update --init

Not too hard. The git submodule update --init tells the main repository to go through all the submodules listed in its .gitmodules file and initialize each one (cloning it and checking out the correct revision).

Updating the main repository and submodules

When the time comes to update a deployment (or a dev environment; they're the same), we would use this technique.

To update the main repository based on what's already on the branch we're tracking ("site"):

git pull
git submodule update --init

To update the main repository to the latest Drupal version or to some specific version:

git fetch drupal  # Get all the latest from drupal.org
git merge 7.x    # Merge the latest Drupal from the drupal.org 7.x branch
git push origin site   # push these new changes to our remote

To update submodules we just pull (or change branches, or check out a tag, or whatever) and then add the updated submodule to the main site repo. This example would update to the latest version of Examples, since it's set up to be tracking the origin/7.x-1.x branch:

cd sites/all/modules/examples
git pull

# to update the parent repository we have to get into its scope, so go up a directory
cd ..
git status

git add examples
git commit -m "Updated examples"
git push origin site

Two major directory structures

I know of two major ways to organize Drupal with submodules.

  1. Using Drupal itself as the base repository, as I've done above. This is easy, intuitively obvious, and works beautifully with Drush.
  2. Creating a main repository (which includes sites/all) and then adding Drupal as a submodule, and other projects also as submodules, This allows Drupal to exist as a separate repository rather than as a container for submodules, and also allows adding other assets, etc.

The structure in this case might be:

  • Master repo (containing directories named sites, assets, etc.)
  • Drupal as a submodule of the master repo and located in /drupal
  • Other projects as submodules of the master repo and located in the sites directory.

The great thing about this organization is that Drupal is a submodule just like everything else, and that non-Drupal assets can be added in other directories, etc.

The two drawbacks are:

  • The Drupal submodule's "sites" directory must be replace with a symbolic link that points to the master repo's sites directory (usually "../sites")
  • Out of the box, drush dl doesn't work with this configuration, although I still intend to figure out how to make it work.

Miscellanous Submodule Tidbits

  • drush has excellent integration with git submodules, which is one reason I love submodules and the basic "use-Drupal-as-the-master-repo" technique. drush dl mymodule --package-handler=git_drupalorg --gitsubmodule will happily download "mymodule" and set it up as a submodule. Just like that. If you want drush always to work this way put these lines in your ~/.drush/drushrc.php:
    $options['package-handler'] = 'git_drupalorg';
    $options['gitsubmodule'] = TRUE;
  • If you fork a Drupal project (if you have your own patches for it) you'll have to have your own forked repository for it, either as a sandbox on Drupal.org or in a private repository. The nice thing is that when the main module catches up to those patches, you can just change the remote on that submodule and do nothing else, and you'll be back in sync with the original project.
  • git submodule foreach can be delightful. For example, if all your submodules track a branch, you can do this to do a "git pull" on each submodule:
    git submodule foreach git pull
  • Removing a submodule is rather awkward. If you can believe it, there is not yet a command to do this. You have to do three things:
    • Remove it from the .gitmodules
    • Remove it from .git/config
    • Remove it from the index: git rm --cached sites/all/modules/xxx
  • You'll probably want to use the excellent Git Deploy module, which figures out what version corresponds to the commit for each of your modules. The 2.x version seems to work much better than the previous 1.x versions.
  • This is probably obvious, but is worth saying: Everywhere you pull, you need to have network access to and permissions on all of the git repositories mentioned in the remotes of both your main repo and the submodules.

The Good, the Bad, and the Ugly

OK, yes there are tradeoffs in complexity and robustness with the git submodule deployment options

Good things

  • For a developer, the ability to make and apply patches is marvelous. If you just start working on a patch for a module, everything will work out and you can just use git to do what you need.
  • It's nicely integrated with drush dl and drush upc (pm-updatecode)
  • You can easily test with various versions of a project and do a git bisect with no trouble at all.
  • It's fantastic for rapidly changing environments (like D7 was for a very long time) as everything can be easily updated to any level.
  • It results in a completely source controlled environment, completely recreatable from original repositories, with full history, at any time.

Bad things

  • The required git submodule update --init is easy to forget when pulling, and it is annoying that it is required.
  • There isn't any explicit support for removing a submodule.
  • The fact that multiple repositories must be accessed to update a site adds complexity to the process, and some fragility.
  • Using git, the Drupal version number of modules is not in the info file, so git_deploy has to derive it. git describe --all can to tell you what version it's on, but that's a big awkward.

Submodule Resources

Screencasts

Basics of deploying with the "Drupal as master repository" repo organization technique:

Additional topics in part 2, including drush integration, different repository organizations, and pros and cons

Do you have comments from your own experience? Feel free to post them here. This is sometimes a controversial topic.

Using Scrapbook Plus to capture the Drupal issue queue for a plane ride

Submitted by rfay on

Webchick tweeted about favorite ways to capture the issue queue for long plane rides. I use the Scrapbook Plus Firefox addon.

Here's the quick screencast, followed by the recipe.

Using Scrapbook Plus to Capture the Drupal Issue Queue for Plane Rides from Randy Fay on Vimeo.

  1. Visit the issue queue and select what you want (needs review, etc.)
  2. Make the capture title descriptive.
  3. Choose to grab all links you want (especially .patch!)
  4. Use a depth of one (find all pages linked by this page)
  5. Filter the results using the regex /node/[0-9]+ to limit the pages captured to issue pages.

Refreshing a Stale Patch with Git

Submitted by rfay on

Git is so smart about merging that it's often possible to referesh a stale patch in the queue using just git, without even looking at the contents of the patch. (I'm not saying you shouldn't look at any patch you post, just that git will often do all the refeesh work for you.)

  1. Update to the latest version of the project you're working with.
  2. Use git log to find the hash of a commit near or just before the time the patch was posted.
  3. Create a branch from that commit: git checkout -b <date_based_branch_name> <commit_hash>. (I often use the date for the branch name: git checkout -b aug10 <hash>
  4. Apply the patch. It should apply cleanly unless there was already something wrong with it when it was created.
  5. Use git add and git statusto make sure that all your changes are staged, then commit the results.
  6. Create a new branch for the updated patch. I'll use git checkout -b bundles_removed_1245332_04 origin/8.x, which branches off of 8.x just for this feature branch.
  7. Merge the date-based branch: git merge <date_branch>
  8. If all goes well, you have a clean merge, and you can create a patch. In this case, git diff origin/8.x >drupal.bundle_warning_1245332_04.patch
  9. If you still have a merge conflict, you probably only have the really relevant part of the conflict (changes actually made in your patch which conflict with changes which have been committed since the issue was last active.) In that case, you can use git mergetool to reconcile the differences, but that's for another session.

For just updating a patch that was obsoleted by the D8 core files move patch in November, see xjm's excellent, focused article.

Here's the screencast:

Pages

Subscribe to Planet Drupal