The most recent issue of Drupal Watchdog includes an article on software freedom and social change in Drupal. While this article raises a number of thoughtful questions about the social implications of the Drupal community's evolution, it includes some misinformation (both because it misrepresents the data that are easily available and because it lacks data that are not easily available). In the first part of this post, I look at the specific information presented in the article and provide some more depth, including some first-hand information about Acquia, since I work in Acquia's Office of the CTO. In the second part, I explore how we can mitigate some of the concerns the article raises.

Who am I and why do I care?

I'm xjm. I started contributing to Drupal 8 core in 2011 when I was working as a part-time hourly at a university. Core contribution was a life-changing experience for me, and I've been high in the list for Drupal 8 patch commit mentions since late that year. I also led what later became the Core Contribution Mentoring initiative. I wanted to "pay it forward" and help make core contribution more accessible to other "outsiders" like myself. I guessed that I was not the only one who had an interest in participating in open source in general and Drupal in particular, but had not done so because of underconfidence, time limitations, lack of information, bad previous experiences, or that that same "outsider" feeling.

Image

In 2012, merlinofchaos invited me to be on the team for the Views in Drupal Core initiative. I cut my hours back at the university and started drawing a salary from the community chipin for that initiative. My budget was tight, but the opportunity to be paid for contributing to that project was an amazing privilege. Views was merged into Drupal 8 core in October of 2012.

In 2013, Dries offered me a position in Acquia's Office of the CTO (OCTO) to work on Drupal 8 full time. Last week was my two-year anniversary at Acquia. I spend almost all of my paid time doing release management for Drupal 8, and part of my job is gathering metrics to help the community make informed decisions about the codebase and the upcoming release. (For example, see the critical issue charts at Help Get Drupal 8 released!).

The Watchdog article's misleading chart (and corrections to it)

Under a header titled "Structured for Profit?", the Watchdog article tries to illustrate which individuals are involved in formal decision-making roles in Drupal, with the goal of asking whether current decision-making in Drupal is "enterprise-oriented" and with a specific focus on Acquia. The article describes the following "influential" roles:

• Core committers These are the individuals who have final say over which particular changes are made to Drupal core. There is a single permanent core committer, Buytaert. For each major version of Drupal, Buytaert appoints one or more “branch maintainers” who commit changes to that version of the software. [reference]
• Official Drupal 8 initiative leads Drupal 8 development was organized around a series of official initiatives, approved by Buytaert, who appointed an individual to lead each initiative. [reference] Official initiatives shaped the core directions and solutions adopted in Drupal 8.
• The Drupal Association The Drupal Association is a nonprofit that, according to its mission, “fosters and supports the Drupal software project, the community and its growth.” [reference]
• Drupal Working Groups Three Working Groups – Community, Documentation, and Technical – oversee key areas of the Drupal project. Individuals are appointed to these groups by Buytaert or his designate(s). [reference]

Beyond formal decision-making roles, the individuals whose code contributions are accepted also exert a significant influence on the software and, by extension, the Drupal project.

(For a background on decision-making in Drupal, I recommend reading Randy Fay's article from three years ago: Drupal Governance. There are two primary changes since Randy's post was written: Drupal 8 now has multiple branch maintainers who collaborate to provide quick feedback and strategic focus in core issues, and the Drupal Working Groups were added precisely to help address some of the concerns that Randy eloquently lays out in that post.)

The Watchdog article's summaries of these roles (each accurate on its own) are followed by a chart, which I at first found confusing, then found misrepresentative, and eventually (after close examination) decided was misleading:

Image

Several of the misrepresentations in this chart are due simply to the fact that complete data are not easily available (especially historically), and I attempt to correct this by providing better data with this post. However, there is one particularly misleading distortion in the chart that obscures everything else, even with the data that are available, and I address that first below.

A contributor with two responsibilities is not two people

The first thing that made me squint at the illustration in the post was a claim that there were somehow 15 people from Acquia that were "Drupal decision makers and top contributors", but I could not account for anywhere near that many people who worked for Acquia in the categories described. Then I squinted more and noticed that there were two people counted for Chapter Three -- a Drupal core branch maintainer and, separately, a top 10 Drupal 8 contributor. Huh? There is only one such individual at Chapter Three (alexpott).

Image

It wasn't until my third read of the article that I actually absorbed this bit in the text:

Particular individuals may fill multiple roles and therefore be counted more than once.

Aha. So it's not stated outright, but perhaps this was meant to illustrate the "size" of an individual's influence based on different ways they might be involved. In particular, Dries is undeniably more influential than any other person in the project, so perhaps counting him three times (once as the project founder, a second time as a Drupal Association board member, and a third time as a branch maintainer) is an attempt to represent that. Unfortunately, I don't think this is clear or effective, and it actually seems rather misleading. The axis on the left side of the chart is labeled "Number of individuals", and even though the note about multiple counting is right there in the text, I (and others) missed it and took the chart at face value.

Image

Three of the roles represented in the chart (core branch maintainers, initiative leads, and "top 10" patch contributors) reflect prominent participation in Drupal core development, and there is actually significant natural overlap between those roles. Our core development process also includes many checks to individual influence for all three roles. All patches must undergo peer review, and we have a culture of collaboration and consensus-building rather than overruling others' decisions. Even branch maintainers cannot commit patches that they contribute to in a nontrivial way (either as patch authors or by providing the "RTBC" or peer review signoff). Finally, when a patch contributor disagrees with a branch maintainer about a change, the branch maintainer consults others (either other patch contributors, or other branch maintainers).

For all these reasons, double-counting people for different aspects of their involvement in core development is particularly flawed. The Drupal 8 initiative leads contributed to many patches both directly and indirectly; the specific number of commits that mention them does not reflect or substantially add to their impact on Drupal 8's development. And all the branch maintainers (excluding Dries) are equal to each other, regardless of whether or not they also author many patches.

It's less clear to compare the influence of core contributors with the Drupal Association board or the working groups, because the responsibilities are so different, but double-counting still doesn't reflect reality. For example, webchick is arguably more influential than other members of the Community Working Group because she is a branch maintainer, but not more influential than other branch maintainers because she is a member of the Community Working Group. In summary, not all individuals on the chart have equal influence, but their influence is a function of their most significant role, not of the number of roles they hold.

Contributors change jobs

Drupal 8 has been under development for four years. Drupal Association board member terms are two and three years. Many people change jobs more frequently than that, so it's imbalanced to only represent contributors' current employers. For example, four of the seven Drupal 8 initiative leads have changed jobs since Drupal 8's feature completion deadline in February 2013. Their current employers weren't their employers when they were doing the work.

In the Drupal core commit log, branch maintainers mention all the individuals who contributed directly to a patch for an issue in its commit message. Generally, the more work an issue takes, the more people end up in the commit message (though this isn't always the case). The Watchdog article includes "top 10" patch contributors based on an open source script that parses and analyzes these "commit mentions". tim.plunkett and I both work at Acquia now, and are both in the top 10 patch contributors to Drupal 8 by commit mention, in part because of our work on the Views in Drupal Core initiative. Except... Views was in core before either of us started at Acquia, and most of our other contributions were made before then as well. To illustrate this, we can compare the commit log references to tim.plunkett before his first full day at Acquia, versus after it:   

git log --before=2014-06-01 --after=2011-03-09 | grep "tim\.plunkett" | wc -l   
985   
git log --before=2015-03-01 --after=2014-06-01 | grep "tim\.plunkett" | wc -l   
182

So not only is the chart overstating Acquia's influence by crediting them with all our previous contributions, it's also under-crediting Tim's previous employers, Zivtech and Stanford, as well as the hundreds of community contributors who gave money to the chipin that paid my salary in 2012 as I worked on Views in Core.

Image

Another interesting aspect of this is that patch contributions are not immediate. In fact, 20% of Drupal core issues take over 6 months to complete. For most of the past two years, I've rarely contributed to patches directly. And yet, between March 2013 and now, I got many of my Drupal 8 commit mentions:   

git log --before=2015-03-01 --pretty=oneline | grep "xjm" | wc -l   
451   
git log --before=2015-03-01 --after=2013-03-04 --pretty=oneline | grep "xjm" | wc -l   
199

This disparity comes from the fact that lots of my work from 2011 and 2012 wasn't committed until later. Even if I left Acquia today, never visited Drupal.org again, and threw my laptop in Lake Monona, I would continue to get commit mentions for work I did in the past.

Contributors are usually volunteers

Acquia currently has six employees who are paid to be core contributors, all in the Office of the CTO: Gábor Hojtsy, Wim Leers, tim.plunkett, webchick, effulgentsia, and myself. During recent months, Wim, Tim, and Gábor are mostly full-time Drupal core engineers; their main role is to work on patches and do code review for issues that block Drupal 8's release. effulgentsia and I do a mix of code review, occasional patch work, issue management, and release management. We work on problems like:

• How can we help get Drupal 8 done sooner and give people accurate information about it?
• How do we make it easier for people to contribute to Drupal 8 criticals?
• How do we unblock this nasty bug that no one is working on or that other contributors can't agree on how to solve?

webchick also currently works primarily on these sorts of problems, plus does code review and patch committing in her role as a branch maintainer. Dries (who is not full-time on core) sets the goals for what we work on in our paid time, though in practice, we have a lot of autonomy and we set our own objectives with Dries' feedback and approval.

On the other hand, pwolanin works for Acquia, but is not paid to contribute. He works on Drupal 8 in his free time. His efforts to lead the Princeton critical issue sprint were all his own volunteer contribution. He used paid vacation time to participate during the sprint. Does Acquia deserve the credit for his work?

Image

Branch maintainer catch works for Tag1. Tag1 does not typically pay their employees to contribute to core. Instead, according to catch, they have good employment practices that make it possible for him to spend more unpaid time contributing. Is Tag1 exerting influence over Drupal 8's direction by having good employment practices? Should Tag1 get the credit for work catch contributes as a volunteer?

Image

Organizations can affect the contributions of the people affiliated with them in a variety of ways:

• In rare cases (for the six of us in OCTO, for branch maintainer alexpott at Chapter Three, and for a number of others), the organization might employ the contributor to do open source contribution full time.
• More commonly, companies give their employees some paid time where they can work on their own projects, outside their normal work responsibilities. Friday afternoons might be contribution time at the office, for example.
• Most frequently, people run into a problem in the course of their day jobs, and contribute to a fix for it because this contribution also helps them solve their problem. This can be done entirely within working hours, though often the contributor might get intrigued by the issue and work on things related to it in their free time. For example, MD Systems is building and maintaining Drupal 8 sites and modules, so Berdir contributes to issues he encounters on those projects in addition to his extensive voluntary contributions.
• Finally, organizations affect the individual indirectly. A contributor's day job affects what the contributor thinks about Drupal, what parts of it they know well, what seems important or good or bad. And, as described above, the organization's employment practices might make it easier or harder for the individual to contribute.

To examine the role of influence in contribution, and to acknowledge contribution with proper credit, we should avoid lumping together these different aspects. There is a set of Drupal.org features under development that will allow contributors to indicate when their contribution is sponsored by a particular organization, and when they are contributing on their own as volunteers. Until those data are available, it's up to us to consider the specifics, and to remember that contributors are usually volunteers.

Finally, none of the forms of influence above cause contributors to lose their own particular passions and interests, their own unique view and skills. People are individuals, not automatons operating within a corporate machine. Even the six of us in OCTO contribute outside of paid time, to things that are not part of our jobs but that we care about. Gábor leads the multilingual initiative because he wants to make Drupal support all languages; Acquia did not prioritize it. Tim voluntarily works on core problems that bug him based on experience building sites and contributed modules, in addition to critical issues Acquia pays him to help fix. And Acquia doesn't own what I think or what I do with my spare time. So while we should recognize the influences organizations may exert over contribution, we should give individuals the credit for their own work.

Counting the "top 10" patch contributors makes no sense

The Watchdog article's chart represents the "top 10" patch contributors by commit mention, but no others. If we treat one commit mention as a unit of work in Drupal 8 core (an imperfect, but not unreasonable metric), this means it provides no representation for over 75% of the work that has been done on Drupal 8. (source data)

Field API maintainers swentel, amateescu, and yched come in 13th, 14th, and 15th from the "top" respectively. They are each mentioned in around 300 commits. The have built and re-built what is perhaps the most fundamental Drupal API -- the one that makes Drupal, "Drupal" -- making fields compatible with new systems in Drupal 8 like the Configuration Management system and the APIs that support Web Services, doing the equivalent of an initiative's worth of work. Why aren't they counted in the graph? Are they really less important to Drupal 8 than me, such that I get a little blue rectangle and none of them do? I'd say the opposite is true--each one of these brilliant developers has done more for the codebase than I have. And all three of them are currently independent.

Drupal 8 core has over 2700 contributors (nearly three times as many as Drupal 7). This contribution follows a stark long tail distribution of commit mentions to individual:

 
The long tail of Drupal 8 contribution, March 1 2015

Inspecting the data and chart closely, there is a "head" on the distribution of a small group of individuals that account for a significant portion of the work done. I'd say it's about the top 50 people, those with 100 Drupal 8 commit mentions or more. Then there is a middle tier of regular contributors, the next 100 or so, who have between 20 and 100 commit mentions. And finally there's the "long tail", the 2500-some contributors with fewer than 20 commit mentions over the four years of Drupal 8's development. (Among them, 1200 contributors have worked on only one patch.)

By this metric, the "long tail" has done more than a quarter the work in Drupal 8. These are people from all over the world helping to improve the open source software they collectively own. By excluding the long tail, the Watchdog article is not crediting a big chunk of the participatory, collective, "grassroots" audience that it seeks to encourage.

Finally, while "top" contributors indeed have disproportionate influence in a pure "do-ocracy", Drupal's top contributors cannot commit their own patches and do not have any particular decision-making authority. So counting patch contributors identically with people who actually have formal decision-making roles does not make sense.

Many kinds of contribution are not represented

A strength of the Watchdog article is that it includes two forms of non-code contribution (Drupal Association board membership and Drupal Working Group membership). However, it's important to keep in mind that there are many other essential influences in our community. As mentioned above, peer review is a required part of our core development process, but typically the Drupal core commit log only credits patch authors, not reviewers. (See the proposal to change this policy.) Furthermore, training, documentation, and mentoring are all essential Drupal contributions, as are the sponsorships and crowd-funding that make sprints and large initiatives feasible. This omission is understandable since these contributions aren't surfaced very well on Drupal.org, but it's important to remember that all are influential.

Finally, there's also no representation of contributed module development. This is reasonable for Drupal 8 since core does not yet have a stable release, but many 8.x branches of contributed modules are already under development using Drupal 8 betas, and these projects will play a bigger role following the release of 8.0.0. It's also worth noting that contributed module development also has its own formal decision-making role, since we have a contributed project application process where ultimately one of our Git administrators must approve each application.

A different picture

The following chart has several differences from the Watchdog article's:

1. I'm representing the actual number of individuals rather than counting someone once per role, because influence is not a function of the number of roles.
2. I list the organization the individual was actually in during the majority of their term in their role, because contributors change jobs. In cases where an individual has been at more than one organization during their active term, I provide an explanation for the organization used in the source data. (Sometimes it's close, but I think counting half-individuals would also be a bit confusing. If you see an error, please comment on the cell in the spreadsheet so I can correct it.)
3. I'm excluding the top 10 patch contributors, because these individuals do not have any particular authority. (I illustrate patch contribution separately further down.)
4. I'm not indicating the nature of each individual's role(s) on the graph, but you can see these in the source data.
5. I'm using current data (the Watchdog article is from Sept. 2014).

 
Drupal contributors in formal decision-making roles by organization

Not all individuals on the chart have equal influence, as described earlier, but I think this illustration is less confusing than representing numerous individuals multiple times, and a better starting point for discussion. While it's still clear that Acquia has the most individuals involved in formal Drupal decision-making roles for a single company, it's a reasonable difference rather than an enormous imbalance.

Next I'll look at the Drupal 8 commit mentions for individuals while they were affiliated with a particular organization. This reduces the distortions related to my second point above (that contributors change jobs). Keep in mind, however:

• Contributors are usually volunteers, and we can't yet distinguish between volunteer and paid contributions on Drupal.org.
• Contributors' work while they are affiliated with one organization might be committed after they move to another organization.
• Individual contributors still matter more than the companies they work for.
• Many forms of contribution are not represented.

To make this chart, I first crowdsourced information on which contributors worked where when. This information is from a variety of sources, like Twitter, Linkedin, personal knowledge, etc. (If you see an error in it, please leave a comment on the cell in the spreadsheet so I can correct it.) To keep this mini-project manageable, I limited this to contributors with 50 commit mentions or more (who collectively account for about 2/3 the commit mentions in Drupal 8). For less prolific contributors, I merged in "current organization" data from Drupal.org, gathered using a database query run on Drupal.org's staging servers. Finally, I adapted the drupalcores script to also parse the date that each commit was made, and then matched that date to the timeframes listed for each patch contributor.

 
Commit mentions and contributor affiliations, March 1 2015

Note that I've only represented the top 15 organizational affiliations and put the rest under "other". In reality, there are many other organizations that have people contributing prolifically, and I encourage examining the full data.

The first interesting thing here is that by far the most important group is that of independent contributors: people who are freelancers, self-employed, etc., or who list no affiliation on Drupal.org. For the long tail, some of these people might be mis-categorized under "other", since there's no easy way for my query to distinguish between the name of someone's one-person freelance business and the name of a multinational corporation.

The second interesting thing is that we confirm that Acquia does indeed employ the most prolific contributors for a single company, even when the data are updated so employees' previous contributions are listed with their previous employers. However, other organizations also make a significant mark, and the individual contributors are still the most influential factor (compare the full organization data with the individual commit mentions). As tim.plunkett put it:

Two dawehners and an andypost == one Acquia.

So do we still need to worry about organizational influence?

Yes, we always should. In practice, Acquia does not direct my work for specific company interests, nor Chapter Three alexpott's. But theoretically they could. We are none of us immune to the organizational influences described above, nor are we above conflict of interest. And while independents and volunteers remain the lifeblood of the Drupal core community, professional contribution has been a very important part of the development of both Drupal 7 and 8, and numerous prominent volunteer contributors are choosing to work for certain organizations (like Acquia, Tag1, Lullabot, and PreviousNext) more than others.

The availability of these volunteer and professional resources is still a good thing--it means individuals and organizations are giving back and helping the project thrive, to the benefit of everyone who uses the software. Still, in order to benefit from these resources while protecting the community against any one group having too much influence, we need a diversity of perspectives. Professional developers building large-scale sites have valuable perspective and experience. So do freelancers who build small sites for local organizations. So do people who need a website and don't care what the code looks like inside, and just want something that works. What can we do to prove Drupal 8 is for everyone and to make sure that anyone can still contribute?

First, I recommend reading Ashe Dryden's thoughtful and well-researched article on The Ethics of Unpaid Labor and the OSS Community for perspective on why potential contributors have unequal resources and face unequal barriers to contribution. Then, here are specific steps we (as a community) and you (as someone in the community who cares) can take:

Lower barriers to volunteer contribution

As mentioned above, Drupal 8 core has had over 2700 patch contributors. The goal of the Core Contribution Mentoring initiative is to inspire, enable, and encourage new core contributors, regardless of background or affiliation, and to work on the core tools, process, and community to make it easier for new contributors to get involved.

Image

Over the past three years, core mentors have learned a lot about what makes contribution easier or harder for volunteer contributors, and we have lots of recommendations for improvements to Drupal.org that need work. You can also help by mentoring at DrupalCon Los Angeles, in IRC, or in your local community, or by joining the conversation at one of the proposed DrupalCon sessions (see Drupal.org changes to support first time contributors and mentors, How do we encourage repeat contributions to Drupal Core?, and Pain points of learning and contributing in the Drupal community).

Continue to improve Drupal's usability and accessibility

If Drupal itself is a barrier, then that's a barrier to contribution. Help with usability and accessibility issues on Drupal.org, or do usability and accessibility testing and report the results.

Credit and support all forms of contribution

The proposal to credit reviewers and other non-coders as first-class contributors will help highlight the impact of contributors in these roles, and the proposal for a new design for user profiles also highlights non-patch contributions more prominently. The work to provide credit to organizations is also essential to recognize how contributions are funded.

Image

Make it easier to sponsor contribution, and fund more contribution more diversely

Sustainable funding for Drupal core has been a topic of discussion throughout Drupal 8 development. Dries' DrupalCon Amsterdam keynote explores social theory and research related to how we sustain open source projects, and the core conversation on funding Drupal core from the same event discusses specifics for funding core in the near term.

In December, the Drupal Association announced the Drupal 8 Accelerate program to help fund critical work on Drupal 8. The fund makes it possible for independent contributors to receive grants for work that brings Drupal 8 closer to release. You can contribute to the fund financially by becoming a Drupal Association member or partner, or contact the Drupal Association about donating directly at accelerategrants@association.drupal.org. Or, if you have an idea on how funding could help speed Drupal 8's release, read the program details and apply for a grant.

Image

 For information on how organizations can invest in contribution through their employees, read YesCT's thorough article on strategies for businesses to invest in Drupal 8. If you are a business owner, you might even consider whether you could join with Acquia, Chapter Three, Pantheon, BlackMesh, and other companies in employing full-time open source contributors.

Finally, the Drupal Core Gratipay Team exists to help fund regular contributors who are not paid for core work.

Adopt employment practices to make contribution both fair and feasible

Ashe's article (referenced already above) has lots of great recommendations on fair employment practices that enable contribution.

Be transparent about governance and decision-making

Image

When the Drupal community was smaller, we didn't have the same need for formal structures for decision-making, because most core developers knew each other. That is no longer feasible; contributors shouldn't need to "know someone" to get something done. As our community grows, transparency is essential.

I was not involved during the development of Drupal 7, so I don't have firsthand experience of how the community responded to these challenges before 2011. Nonetheless, I think we've made a lot of progress in Drupal 8. For example, the introduction of the Working Groups improved the transparency of community decision-making. We've added the core gates, issue summaries, Drupal Core Updates, and beta evaluation criteria all to make core development and patch acceptance more transparent. Branch maintainers now also collaborate to give quicker feedback on issues, including regular triage of criticals so the community has accurate information on what is blocking Drupal 8's release.

None of these changes have completely resolved transparency problems around core development, and some of them have been rocky, especially at first. It's a struggle to scale core development without adding unneeded overhead. Our next challenge is definiting the roles of core component, topic, and branch maintainers more clearly.

Be transparent about volunteer and paid contribution

Finally, for all the reasons in this post, transparency about both volunteer and paid contribution is essential. The most important step is the work to provide credit on Drupal.org for organizations sponsoring development. This change will simultaneously provide fairer credit to all organizations--not just those employing full-time developers--and clarify what contributions are funded versus contributed by volunteers.

I'm glad Acquia invests directly in Drupal 8 core contribution the way that it does. After all, the fact that I'm getting paid to contribute full time is awesome and the entire reason I work for Acquia in the first place. I also think we need to talk more about all the other organizations that contribute in significant ways, and most especially the individuals who do. I still recognize that we are all prone to conflict of interest, and it's crucial that we maintain and expand Drupal's diversity to provide balance. Accurately representing the current diversity of contribution is the first step.

Thanks to YesCT, effulgentsia, catch, and alexpott for contributing to this post. All data are from March 1, 2015.