As mentioned in an earlier post, DataFlow is an Oxford-based project in the JISC UMF programme building a data management infrastructure to help researchers manage their research data.

OSS Watch, in collaboration with Open Directive, are providing licensing, development, community and sustainability support to the project, which is now getting very close to a new release.

Developers have frozen the code and are preparing beta versions of DataStage and DataBank, which will be available for testing as virtual machines. Please keep an eye on the project website and twitter channel for updates on when and where you will be able to access them.

We will hold a launch workshop in Oxford on 2 March with colleagues from the VIDaaS project, who are building an exciting cloud-deployable Database as a Service system.

Attendance is free but places are filling quickly, so book early to avoid disappointment.

Google has been in the news repeatedly over the last six months for closing down some of its many, many side projects. In general these are being mothballed in perpetuity, but in some cases, there is a transition plan. This is the case for Google Sky Map, and for our community it’s an interesting variation on the more traditional ‘open source it and hope it takes off’ approach that industry players like Nokia have tried in the past, and HP seem about to try again.

Sky Map is an application for Android mobile devices that was born out of the so-called 20% time that Google grants its engineers for the pursuing of personal projects. Back in 2009 when Sky Map was launched, one of the few hardware advantages that the few Android phone then on the market had over the iPhone was a hardware sensor compass. The ‘wow’ moment of the first public display of Google’s Android hardware (the G1) was a demonstration of how the Streetview service could be used in combination with the device’s compass to display information relevant to the direction you are facing. To build on this brief window of competitive advantage (the iPhone acquired a compass in its next hardware iteration) Google’s Pittsburgh office (then based on the campus at Carnegie Mellon University) developed and released Sky Map. The functionality – showing information about the night sky in the direction you were looking – was useful and educative and spawned tens of imitators over the next few years.

Given that the software no longer promoted a competitive advantage of Android hardware, and that the competition for apps of this kind has been getting tougher and tougher, it’s not entirely surprising that in Larry Page’s seemingly endless round of belt-tightening the project has been let go. Sky Map is returning to its roots on the Carnegie Mellon Campus:

“Today, we are delighted to announce that we are going to share Sky Map in a different way: we are donating Sky Map to the community. We are collaborating with Carnegie Mellon University in an exciting partnership that will see further development of Sky Map as a series of student projects. Sky Map’s development will now be driven by the students, with Google engineers remaining closely involved as advisors. Additionally, we have open-sourced the app so that other astronomy enthusiasts can take the code and augment it as they wish.”

This is an interesting approach. Although it’s not clear yet exactly what kind of student projects will be invited (computer science? astronomy? both? neither?) the idea of taking end-of-life, production code and open sourcing it to facilitate learning and teaching is a model I would like to see more generally adopted, for a few reasons.

Firstly, it is likely to teach open development methodologies to student software authors, something which is still notable by its absence in too many academic primary and secondary software development programmes. Secondly, it provides a compelling means of community engagement for the academic institution, opening a window into their teaching for the outside world and inviting collaboration. Thirdly, it advertises the skills of the students with a directness and accessibility that mere CV distribution cannot really match.

Although I’m not thrilled by Google’s year of the long knives (goodbye Google Sets *sniff*) solutions like the one proposed for Sky Map are genuinely exciting, and I’ll be watching for the resulting academic projects with interest.

Last week, Education Secretary Michael Gove announced that the current ICT curriculum, characterised as ‘demotivating and dull’, is to be replaced by a computer science programme. Gove said that the Programme of Study will be withdrawn and teachers are given freedom over what and how to teach.

Later that week, the Royal Society published the study on Computing in Schools, named ‘Shut down or restart?’. The main recommendation of the study is to restructure ICT education and make a clear distinction between these three main components of ICT education:

1. Digital literacy – How to use the most important IT applications.
2. Computer Science – Learn how the computer works.
3. Information technology – How to use ICT to solve important problems in business and industry.

A major problem, highlighted by the Royal Society, is the small proportion of ICT teachers who have qualifications in the area. Only 1 in 3 ICT teachers have the relevant qualifications, whereas on average this is 75%. It remains to be seen if giving teachers the freedom over what and how to teach, when most of them don’t have the qualifications in the field, is the best way forward.

In any case, there are some open initiatives that may help develop better ICT education. First of all, there are developments towards sharing teaching materials as Open Educational Resources, that can benefit ICT education. Repositories such as Jorum and OER Commons make it easier for teachers to share and learn from their peers.

There are also exciting tools available, such as ToonTalk and Scratch that will help children learn to programme in a playful manner. Although Scratch itself is not open source, it is available for free and it is built upon the open source tool Squeak, a Smalltalk language and environment. The One Laptop per Child project distributes the open source tool Etoys, built on Squeak, to help children get familiarised with programming. Websites dedicated to open source in schools, such as Schoolforge.net and Open Source Schools are also useful in sharing the relevant software and expertise.

Finally, there are exciting initiatives to help get more computers into the classroom. The Raspberry Pi project is creating a computer the size of a deck of cards, available for $25-$35, that is still powerful enough for all common tasks that you would need in the classroom. Although it sounded implausible when I first heard about it, they are now actually started manufacture! It reminds me of the old BBC Micro, a project where the chair of the Royal Society’s study mentioned above, Professor Steve Furber, was actually the principal designer of. If we could supply computers to our kids in the same way that we supply pencils, we may be a step closer to ensuring our next generation is ready for the future.

Following a two-year revision process based on feedback from Mozilla and the broader open source legal community, version 2.0 of the Mozilla Public License (MPL) was recently released. The licence encourage contributors to share modifications they make to MPL-licensed code, while still allowing them to create projects that combine MPL-licensed code with code under other licenses (either open or proprietary).

Like its previous version, MPL2 has been acknowledged as a free software license by the Free Software Foundation and as an Open Source license by the Open Software Initiative.

The most important changes from the previous version include:

• MPL2 is simpler and shorter;
• addresses the recent changes in copyright law and incorporates feedback from lawyers outside the United States;
• offer patent protections aligned with other open source licenses, which allow communities to protect contributors if these are sued;
• provides compatibility with the Apache and GPL licenses, making code reuse and redistribution easier.

For years now the Apache HTTP Server has been by far the most widely used web server on the internet. Netcraft publish statistics on web server usage monthly, using a variety of metrics, and this month’s stats show an interesting change. While Apache HTTP Server is still miles in the lead, second place in the ‘active sites’ metric (meaning sites which are not just mothballed domain names) has transferred from closed source Microsoft web server IIS (Internet Information Server) to open source upstart Nginx (pronounced ‘Engine X’), released under the two-clause BSD license. Nginx has developed a reputation for speed and low resource requirements that has made it popular in a relatively short time.

So the fact that the top two slots in one of Netcraft’s surveys are now filled by open source web servers is interesting in itself, but there’s something more to this. Unlike Apache HTTP Server which is developed under the supervision of a US not-for-profit foundation, Nginx has recently become a commercial company offering paid support and successfully raising $3m in series A venture capital funding. As well as paid support, Nginx has announced that the intend to implement an Open Core model for their business going forward.

Now the Open Core model is what we used to call ‘proprietary extensions’, meaning that the open source code is supplemented with closed source paid add-ons for those that want them. In a way it is similar to the shareware model that did so much for PC gaming during the 1990s, bringing games like Doom to offices everywhere. One often cited problem with the Open Core model is, however, that users of the open source ‘core’ are at liberty to build competing open source versions of your proprietary extensions. Indeed you can find that ideological opponents of the partial freedom that Open Core embodies may be motivated to compete simply because of that ideological opposition, essentially enforcedly ‘opening’ the parts of the project functionality that you wish to keep closed. The only really effective defence against this risk is to be the best-resourced and most skilled team working on the code, thereby ensuring that your extensions cannot be easily replicated by competitors. So Open Core is an interesting strategy, in that it has drawbacks from both the purely ‘open’ point of view and the more traditional closed source approaches to software exploitation. In the past it has been accused of attempting to benefit from the ‘Halo Effect’ of open source while in fact leveraging closed methodologies for value realisation, but the fact that Nginx has managed to achieve so much in such a short period makes it a technology and a company to watch.

Collaborating in an open source project might take time, but is worth the investment. Especially when bootstrapping a project and with a small community, the overhead projects need to put into the collaboration can be significant. But the rewards can make this worth the effort, even when having as few as 3 collaborators. At TransferSummit, I presented on this topic and the primary slide from that presentation was this one:

It shows that even when as much of 40% of your time is spent on collaboration, you can get more back than you invest, even in smaller communities. The loss of productive time is compensated by the contributions you get back as a result of the investment in the collaboration. Not everything others are contributing to your project is of value to you, but there is always an overlapping need, and in the example on the slide we’re assuming that’s about half of their contribution. When working with a small number of project partners, this can already be worth while.

An example of how you can get more back by working together occurred recently with a series of projects that OSS Watch has been involved with. It’s an example of how by working in an open development context you can innovate faster and get results quicker, with everyone benefiting.

This example begins with the University of Bolton, who created the Wookie Widget environment as a result of an EU-funded project. This is now a project in the Apache Incubator.

The Wookie Widget environment formed the basis of a project funded by the IPO. That project built a widget making use of the Wookie server, to display a walk-through to guide users through the different issues concerned with open licensing. Most of the content that was used in this project was previously created by OSS Watch, and we therefore profited ourselves from this project.

Separately, the Rave in Context project built a templating system usable in the Wookie environment. By working with the Wookie community the Rave in Context project received more value through validation and testing of the template system and some additional functionality, such as the
option to change the order in which overridden content is given preference in template-produced widgets. On the other hand, the Wookie community profited from getting a templating system that makes it much easier to create widgets that are usable and accessible out-of-the-box, thereby increasing chances for uptake of the project by new potential collaborators.

Now OpenDirective are about to embark on creating a self-learning widget to help users assess the openness of open source projects. This new widget will be used by OpenDirective as a tool for supporting their clients and has value for OSS Watch in supporting our projects as well. This new ‘openness widget’ will be based on an improvement of the original IPO widget and on the Rave in Context templating system. This feeds features of the templating system back to the IPO widget. For example, it now works well on small screens, which was out of scope on first version. It benefits OSS Watch, because it enables us to use our content on assessing the openness of projects in a new and easy-to-use tool. It is beneficial to the Rave in Context templating system and therefore to the Wookie community, because of the extra validation of the templating system and additional functionality that the approach with the IPO widget offers.

On a very small scale, all of these projects provide value to each other by collaborating with the other communities and reusing what has already been produced. It demonstrates that you don’t need a huge project with big budgets to reap the fruits of open developments!

I was thoroughly drawn in by the Amazon ‘Black Friday’ event last week, buying both a phone and a camera, against my better judgement and to the disgust of my bank manager. While trying to suppress my buyer’s remorse by searching the internet for all the marvellous capabilities of my soon-to-arrive devices, I noticed that the camera, a Canon Powershot SX220 HS, was one of the models capable of running a piece of open source software called CHDK released under the GNU GPLv2. This program leverages the fact that the camera will execute anything that look even remotely like a firmware update that is located on its SD card without requiring a digital signature, allowing an adjunct to the device’s firmware to be executed every time it starts up. You can even place the program on the SD card and select whether it is booted or not by changing the ‘write protect’ switch on the card.

Once the software is booted the user has access to an almost ridiculously long list of tweaks and features, including saving pictures as ‘RAW’ (meaning that the data from the camera’s sensor is saved to the card unaltered, rather than being crushed down into a smaller JPEG file), greater control over exposure times and the ability to construct more complicated ‘bracketing’, meaning that a series of shots can be taken with differing focal lengths or levels of white balance, allowing creation of HDR images and focus-stacked images. Even more geektastic is the ability to script the functionality of the camera using UBASIC or LUA, allowing a user to build functionality like time lapse photography and the taking of pictures only when motion is detected.

One question that remained with me, even as I contemplated spending more unwise pounds, was what Canon’s attitude was to this project. After all, some of the functionality that CHDK contains can be obtained from Canon in its more expensive models. They could close the technical loophole that allows the additional software to be run fairly easily, so one must assume that they do not see the existence of open source expansions of their equipment as a threat to their business model. Might they even see it as a selling point? Certainly it seems that running CHDK is likely to void your warranty, so perhaps the existence of a group of customers who opt out of expensive warranty provision is seen as a bonus.

Discovering this went some way towards alleviating the guilt of my spending. After all, I had got a large amount of functionality at essentially half price. But… if you want to run long scripts then you really need the attachment that lets the camera run off mains power. That’s not such a bargain. I could also probably do with a better tripod… Oh dear…

OSS Watch is currently leading a small development project called Rave in Context. This project is developing templates to easily write W3C widgets that encompass best-practices in ussability and accessibility out-of-the-box. It is not feasible or useful for small projects like this one to try and develop a completely new community around the software. Instead, we looked for an existing project that would be a good home for the software and found that in Apache Wookie (Incubating). However, the project is also closely related to Apache Rave (also Incubating), because this project also supports the W3C widget standard and a few people involved with that project have shown an interest in what we’re doing.

Because of the close connection with these Apache projects and for the benefit of wider uptake of our project code, we thought it would be good to present about the Rave in Context project at ApacheCon 2011 in Vancouver. I was happy to be allocated a session in the ‘Fast Feather’ track, a track that focuses on latest developments and upcoming technologies.

The widgets that are being developed by the Rave in Context project will conform to the W3C widget standard. This allows for deployment in a W3C widget container such as Wookie. However, there are many other standards in the widgets and gadgets space. Another popular one is the OpenSocial gadget specification, which defines not only social concepts but also the packaging and specification of the gadgets themselves. Because of the big overlap between W3C widgets and OpenSocial gadgets, it has been relatively easy for the Rave project, that started of as an OpenSocial container, to also support W3C widgets. This makes the Rave in Context widget templates useful for that community as well.

During the presentation and some of the other talks and meetups at ApacheCon, it became clear that there is a lot of interest in the gadget/widget portal functionality that is being developed by the Rave project. Also the gadgets and widgets that are being developed may themselves be interesting to the wider community. We talked to a few people, for example, that have worked with portals based on the Java portlet specification, and are finding that this is not a specification that is suiting their needs anymore. They are looking to use a different technology for their front-end portals and the Rave portal is an option they’re considering. Also a project like Sakai OAE (f.k.a. Sakai 3) are working with gadgets and gadget repositories on their front-end. They are currently not working with a gadget standard, but moving to OpenSocial and using Rave as their front-end is something they are interested in.

The code of all the gadgets and widgets are currently scattered around the different projects, and this is not usually the best place for this kind of code. Hence, there is a clear need for a separate project to host the gadgets and widgets created, and this can also include the widgets created as part of the Rave in Context project. We will set one up at the Apache Extras site, which is a code hosting site specifically for Apache-related projects. This will enable anyone to reuse widgets and/or gadgets generated in any of the projects, thereby delivering on the promise of open innovation in software.

Just a quick one on the subject of open source and patents. John Carmack is well known in gaming circles as the lead programmer behind such classic PC and console games as Castle Wolfenstein, Doom (and sequels) and Quake (and most of its sequels). Carmack and his company id software are the originators of the ‘First Person Shooter’ genre of game which has in turn spawned such gigantic franchises as Call of Duty and Halo. As well as being technical pioneers, id has an interesting policy of releasing their old engine technology (the software which renders the game’s video and audio) as open source under the GNU GPL v2. This allows students of gaming software development to look at how real commercial games software is written, and also allows the games to be ported to new hardware platforms by volunteers. As the art and sound assets are not included with the code, this also generates a small market for licences to old id games – games which may well not run on more modern operating systems – in order to get the game data for use with the aforementioned ports.

Rage, the id game which uses version five of the id rendering engine (id tech 5, as it is known) has just been released. This is the point at which the source to the previous engine would usually be released as open source. However in this case there is a problem. Back when id tech 4 was being written for the game Doom 3 over the period 2000-2004, many games developers were looking into what were then cutting edge graphical technologies for inclusion in their games. One such technology was stencil shadowing, which accurately projects shadows from moving objects onto surrounding surfaces based upon the light sources which are illuminating them. This is hard work even for the specialist graphics hardware in PCs and consoles, and so any algorithmic optimisations that are possible are highly valued by the industry. So, various developers hit upon the same optimisation around the year 2000. This optimisation has come to be known as Carmack’s Reverse, even though it was first presented by William Bilodeau and Michael Songy of Creative Labs back in 1998. John Carmack discovered it independently some time later, and was perturbed to discover that Creative had already patented the process.

Faced with a choice between licensing the patent from Creative or making his code sub-optimal, Carmack decided to strike a deal with Creative that allowed him to use the technology at no cost. Perhaps coincidentally, id also agreed to use sound rendering technology by Creative in their game.

So the game was shipped and everyone was happy, until half a decade later when it came time to ship the code as open source. Obviously whatever deal was agreed between Creative and id did not involve making code available that embodied Creative’s patent under an open source licence for the whole world to use. In a tweet on the subject Carmack explained:

Lawyers are still skittish about the patent issue around “Carmack’s reverse”, so I am going to write some new code for the doom3 release.

It will be interesting to see how Carmack replaces the code, but the issue is also of interest because it illustrates the importance of keeping good records of the inbound IP in a software project. id’s lawyers caught what could have been an expensive potential infringement of Creative’s rights when they ’skittishly’ requested that Carmack rewrite the code.

(Apologies to E J Thribb)

Adobe’s announcement that it will be dropping Flash Player for mobile devices from its future plans has been widely interpreted as a victory for Apple, and in particular their late Chief Executive Steve Jobs. Perhaps because of his essay Thoughts on Flash, the absence of Flash technology from Apple mobile devices has seemed to be a personal decision of Jobs. In that essay Jobs made a group of points about exactly why he saw Flash as a detrimental technology, certainly for Apple mobile devices, and to some extent for Apple computers in general (“We also know first hand that Flash is the number one reason Macs crash.”) Competing phone and tablet makers pointed to their devices’ ability to run Flash, although not always that well. Apple’s refusal to engage with Flash on mobile led Adobe to declare that Apple mobile devices could not access the ‘full web’, particularly the video content that at that time was most frequently packaged as a Flash object.

In fact, Flash became so synonymous with web video packaging that it is easy to forget that it started life (as FutureSplash Animator back in the dark days of dial-up) as primarily a vector graphics animation package. It provided the possibility of small files containing large images and enhanced interactivity (this was before Javascript was well supported or unified). Gradually Javascript and broadband made these selling points more moot, and Flash then made the leap into bundling video codecs and providing a unified way for browsers to display video. Now, partly as a result of Apple’s stand against it, native video decoding in the browser combined with HTML 5’s <video> tag have once again made Flash less relevant. Now, it could be argued, the chief use of Flash is in rapid prototyping and dissemination of simple web games, many of which go on to spawn native titles on the bigger gaming platforms. Even that function is likely to be somewhat supplanted by HTML 5 apps in the future.

So has Flash as a whole been doomed by Apple’s tactics. It’s worth looking at that Steve Jobs essay again to find out. Among the many legitimate criticisms he levels (instability, insecurity, poor efficiency) Jobs also attacked the fact that Flash provided a path to developing mobile apps (both in the browser and compiled to native code) that was out of the control of the platform owner:

We know from painful experience that letting a third party layer of software come between the platform and the developer ultimately results in sub-standard apps and hinders the enhancement and progress of the platform. If developers grow dependent on third party development libraries and tools, they can only take advantage of platform enhancements if and when the third party chooses to adopt the new features. We cannot be at the mercy of a third party deciding if and when they will make our enhancements available to our developers.

This becomes even worse if the third party is supplying a cross platform development tool. The third party may not adopt enhancements from one platform unless they are available on all of their supported platforms. Hence developers only have access to the lowest common denominator set of features. Again, we cannot accept an outcome where developers are blocked from using our innovations and enhancements because they are not available on our competitor’s platforms.

Flash is a cross platform development tool. It is not Adobe’s goal to help developers write the best iPhone, iPod and iPad apps. It is their goal to help developers write cross platform apps. And Adobe has been painfully slow to adopt enhancements to Apple’s platforms. For example, although Mac OS X has been shipping for almost 10 years now, Adobe just adopted it fully (Cocoa) two weeks ago when they shipped CS5. Adobe was the last major third party developer to fully adopt Mac OS X.
Our motivation is simple – we want to provide the most advanced and innovative platform to our developers, and we want them to stand directly on the shoulders of this platform and create the best apps the world has ever seen. We want to continually enhance the platform so developers can create even more amazing, powerful, fun and useful applications. Everyone wins – we sell more devices because we have the best apps, developers reach a wider and wider audience and customer base, and users are continually delighted by the best and broadest selection of apps on any platform.

In other words: Apple did not want development tools for their mobile devices to exist which were not under their control. Jobs cited this as ‘the most important reason’ that he rejected Adobe’s technology. Yet only months after that essay Apple backtracked on the restriction and permitted Flash tools to compile Adobe Flash Actionscript applications for submission to the App Store. Indeed, if you don’t own a Mac, using Flash to generate iOS apps is one of the very few alternatives available to you. In fact, Flash Builder, the tool which does the actual compiling of Actionscript programs into iOS applications, is really the open source development environment Eclipse distributed with a proprietary Adobe plugin. Potentially the concession won by Adobe could lead to entirely open source tool chains for the development of iOS apps.

So while Jobs’ tauntings over the possibility of a robust and useful Flash Player Mobile (“We have routinely asked Adobe to show us Flash performing well on a mobile device, any mobile device, for a few years now. We have never seen it”) have proved prescient, in fact Adobe won a crucial concession from Apple almost a year ago. That concession widened the potential role of open source tools in developing iOS apps (it’s always been possible for Android). Adobe have also announced that they intend to ‘aggressively contribute’ to HTML5, perhaps indicating that they will be extending their development tools to allow Actionscript programs to be emitted as HTML5 web apps.

What really emerges from the struggles over mobile Flash is a strong sense of the entropy of the mobile device space at the moment. Rhetoric is deployed and attitudes struck, only for the originators to back away in a matter of months. App Store policies change monthly and with them the possibility of using open source code on the devices they serve. Competitive head-butting between closed source behemoths (like Adobe and Apple) can result in the opening up of data standards (as the pressure from iOS users has resulted in more HTML5 compliant video on the web – although let’s not get into the patents around those). For open source authors and proponents, the mobile space remains a changing and challenging environment. What the skirmishes around Flash demonstrate, though, is that the struggle of closed source vendors for competitive advantage can provide opportunities for free and open source code.

The amount of data that is being generated is still rapidly increasing and both the commercial and the academic sector are working to tackle new challenges that arise from it. These are exciting times for open source projects like Apache Hadoop, a framework that allows for the distributed processing of large data sets across clusters of computers. Many big IT players like Microsoft, IBM, Oracle and Amazon use Hadoop in their offerings.

Academic researchers also continue to generate bigger and bigger data sets. This provides not only challenges for processing the data (something Hadoop can help with), but these data sets need to be managed as well. This involved aspects like version management and longer term curation of the data, to make sure they are and will remain available, just as the scientific publications that were created based on the data.

One exciting project that OSS Watch is currently involved with is DataFlow. This is a project that is tackling the issue of research data management in two stages.

Firstly, there is a software tool called DataStage. In a way this tool works similar to the popular tool Dropbox: researchers can save files to a dedicated location on a network drive, which means it will be stored on a departmental server and the file will be version-managed automatically. As a result, a new version will be created whenever a file is changed and saved onto the drive, which means that the researcher can always go back to a previous version of the file if necessary.

The second stage of data curation is when a file or a set of files is finally used for a publication and the researcher wants the data set to be available for other researchers, or wants to include a DOI reference to the data set. The researcher can then copy the file over to DataBank, an institution-level research data repository.

Both DataStage and DataBank are open source software projects so we welcome potential users and developers to try it out. The projects carry the permissive open source licence MIT. This makes it possible for commercial companies to include the software in a proprietary offering.

Many universities are looking for a solution for Research Data Management and we believe that the software DataFlow is developing are very useful tools that fulfil that requirement. Join us on our mailing list and find out more about this project!

“Build a better mousetrap, and the world will beat a path to your door” as Wikipedia informs me Ralph Waldo Emerson never quite said. The point – that real innovation sells itself – remains true today. Indeed it could be argued that the average consumer is more engaged with the heartbeat of technological innovation now than ever before, with software releases making headlines among the more traditional stories of war and celebrity.

Emerson’s non-quote does raise a question, however. How do we identify technology which is better? With mouse-traps there are some fairly obvious metrics relating to mouse mortality and cheese preservation, but not all inventions are as easy to benchmark. The last few weeks have seen anouncements of upgrades to the world’s two most commonly used smartphone operating systems: Apple’s iOS (version 5) and Google’s Android (version 4). Each brings a raft of new features, although in both cases it has to be said that these new features are no longer as core to the operation of the device as innovations in earlier versions. Voice-operated search and facial recognition are nice, but hardly essential elements of a mobile computer, at least for now. Perhaps lost in the combative comparisons deployed by proponents of each OS is the fact that a genuinely key ability – web browsing – is implemented on both platforms using essentially the same code: the Web Kit open source project. While newer functionality is added by Google and Apple to differentiate the competing products, it pays them to cooperate on key, unavoidable elements of their offerings. Given this, it’s fair to repeat the question – how do we identify real innovation? The newer differentiating features appear to be the cutting edge of endeavour, but their very newness is a demonstration that – up to now at least – they have not been essential elements of the technology in question. Some of them will die away despite their novelty, having never truly improved the invention that they embellish. Like a cheese grater on your mouse trap, it’s possibly a nice idea and undoubtedly novel, but how useful is it really? Only time will tell, and in the meantime better springs, and better browsers, are being developed.

So perhaps the question needs to be: “looking back at innovations that have proved to be key, how do they tend to develop?” Using the answer to this, we might be able to form some techniques for looking at our cutting-edge-but-possibly-pointless innovations and making guesses about their eventual utility. We might even be able to identify over-arching strategies for conducting and rewarding innovation…

Here we get into an argument that flared up earlier this month, when a video of Francis Gurry, the Director General of the UN’s World Intellectual Property Organization (WIPO) back in June was discovered by the internet commentating community. Gurry was speaking to sum up his views on a debate which had just taken place on ‘Accelerating Growth and Development’ in relation to invention and intellectual property. Gurry’s argument was seemingly  summed up by the headline on the BoingBoing article which drew it to the internet’s attention: “WIPO boss: the Web would have been better if it was patented and its users had to pay license fees”. Reading the article, though, even the quote that BoingBoing had pulled failed to use that emotive word ‘better’:

Intellectual property is a very flexible instrument. So, for example, had the world wide web been able to be patented, and I think that is a question in itself, perhaps the amount of investment that has gone into or would be able to go into basic science would be different. If you had found a very flexible licensing model, in which the burden for the innovation of the world wide web had been shared across the whole user community in a very fair and reasonable manner, with a modest contribution for everyone for this wonderful innovation, it would have enabled enormous investment in turn in further basic research. And that is the sort of flexibility that is built into the intellectual property system. It is not a rigid system.

Reaction to the video from proponents of open content and open source across the internet was voluble and aggravated. Gurry was accused of being ideologically indoctrinated and blinkered, tied to anachronistic models of IP registration and exploitation even in the face of the incredible growth and success of the web largely without the intervention of these models. In fact though, the most that Gurry says is that the web would have been ‘different’. Taken in the context of the statements which preceded it (and which you can hear by downloading the video), in which the value of the traditional IP systems had been questioned repeatedly, Gurry’s statements do not really support the distillation they were given, and which caused so much anger. He is trying to argue that the web could have grown within more traditional licensing structures. Whether he is right about this or not, he is not claiming here that it would have been ‘better’ under those circumstances.

The anger and confusion here are natural, though. The battle lines between proponents of the traditional and the more ‘open’ approaches to innovation (and here we should note that the buzz phrase ‘open innovation’ often itself refers to deeply traditional IP exploitation patterns) have long been drawn, and the forces on both sides are keen to tackle and destroy the arguments of their opponents wherever they see them. The web is often perceived  - with much justification – as a triumph of innovation outside the traditional IP exploitation framework. To hear someone perceived as being part of the old-guard even discussing it can seem presumptuous to some ears. Yet in reality the implied dichotomy here is simplistic. The open licensing movements themselves are underpinned by the arcane operations of traditional licensing and exploitation. While they may give these operations an innovative twist, they could not be enforced or defended without them. Conversely, Gurry’s example of why  the patent regime is beneficial fails to address the criticisms of openness proponents. He points to the publication framework implicit in the current patent system, and makes the comparison between the saxophone – which has fully documented design documents available thanks to its having been patented – and the violin – where many secrets of producing the greatest instruments have been lost through secrecy and the passage of time. This critique – while interesting – is almost wholly inappropriate as a defence of the current system in opposition to more open models. In the modern case, both models involve complete publication – the distinction lies in how benefits are reaped from exploitation and by whom.

Given the frequent failures of either side in this debate to engage with what the other is actually saying – illustrated by this sad tale –  it’s not surprising that telling which innovations are better remains hard. While ideology is important, it can often obscure our view of what actually matters to most people: how many mice are killed (or indeed captured).

At OSS Watch we work with projects that develop software within and for academia. When these projects use the open development approach, they have the opportunity to engage third parties with the project and receive contributions from people with an interest in the project. In order to be able to do that, the project needs a governance model that makes it clear for external parties how to engage with the project. An example of how such a governance model would work in practice is the Cocoon project at the Apache Software Foundation (ASF), which is a classic example of how a meritocratic governance model works in practice. In this post, I will highlight bits of a case study that Andrew Savory has written about the project. It will demonstrate how the project governance at the ASF is organised and can help you when considering a governance model for your own project.

The Apache Software Foundation and ‘the Apache Way’

The Apache Software Foundation (ASF)is a non-profit 501c(3) organisation incorporated in the USA, set up to support open collaborative software development by supplying the hardware, communication and business infrastructure. The ASF is an independent legal entity to which companies and individuals can donate resources and be assured that those resources will be used for the benefit of the public. The Apache Software Foundation provides the structure and governance to support the Apache community of open source software projects. That community is made up of nearly 100 top level projects that cover a wide range of technologies.

All Apache projects follow what has become known as ‘the Apache way’ in that they follow the guiding principles behind the Apache Foundation and its community of developers and users:

• collaborative software development
• commercially-friendly standard licence
• consistently high quality software
• respectful, honest, technical-based interaction
• faithful implementation of standards
• security as a mandatory feature

The illustration below shows the layout of the Apache Software Foundation and the projects within it. The Board of Directors is responsible for maintaining the business affairs and legal oversight of all parts of the Foundation subject to a set of byelaws. Reporting to the Board is a Project Management Committee (PMC) for each of the projects within the foundation. The PMCs are established by the board to be responsible for the management of one or more communities. They provide oversight, ensure legal issues are addressed and procedures are followed, and are also responsible for the long-term health and development of the community as a whole. The PMC’s responsibilities do not include coding, although they do play a crucial role in approving all software releases. The PMC has a chair, but this is for legal and bureaucratic reasons only—the emphasis is always on a community of peers. Each PMC has the freedom to set up their own byelaws to arrange matters such as the election of the chair.

Within the ASF the emphasis is very much on individual contribution rather than the contribution of companies (or people working on behalf of companies). No developer is directly paid by the ASF for the work they do on projects although developers may be, and often are, paid by their own employers to work on the project. The roles that an individual may play within a project and within the ASF as a whole are very clearly defined, and this helps individuals to understand how, when and where they can make a contribution.

Each project community revolves around a code base, and operates as a meritocracy—literally, governed by merit. When an individual working within the community is felt to have earned their place in the community, they are granted direct access to the code repository, thus increasing the group and increasing the ability of the group to develop the code. The roles defined within the meritocracy are shown below and are as follows:

• User: someone that makes use of ASF software. They may be a passive user (simply downloading and working with the software) or an active user (providing feedback via suggestions or bug reports), and they can participate in the community by helping other users via the mailing lists.
• Developer: a user who contributes to the project in the form of code or documentation. They actively contribute on the developer mailing list, provide fixes, suggestions and criticism. Developers are also known as contributors.
• Committer: a committer is a developer who has been recognised by the community and given write access (the ability to make updates to a project’s code) to the code repository, and who is also able to make short-term decisions for the project.
• PMC member: a developer or committer elected due to their contribution to the evolution of the project and their demonstrated commitment. They have the right to vote on community-related issues and to propose other developers for committership.
• ASF member: similar to a PMC member, the ASF member is nominated for commitment and contributions to the ASF as a whole and membership is by invitation only. They participate across several projects, and have the right to elect the ASF board.

There are two ways for decisions to be made within the community. Because the committers have already been recognised as responsible developers by their peers, mutual trust and peer pressure are in place to ensure committers act responsibly and know when it is appropriate to make decisions themselves and when to call a vote.

Minor decisions are usually made on the assumption that ‘code talks’, i.e. developing and committing the code to prove a concept or provide a solution is often more effective than endless discussions. For changes that are a bit more drastic, many ASF projects use the tag [PROPOSAL] in an email to suggest a certain direction. If nobody objects in this email thread, the person proposing the change is entitled to go ahead in the proposed direction. This is known as lazy consensus and it has proven to be a powerful mechanism to keep things moving within the community.

For more important decisions, such as architectural changes or major new features, votes are required. Voting is carried out by voting, that is, a discussion is started on the project mailing list titled [VOTE], and is left for at least 48 or 72 hours (depending on the project’s byelaws) before the initiator sums up. A lack of response is assumed to be tacit agreement. The period is designed to take into account the distributed nature of development, and to allow developers in different time zones to play equal roles.

Votes are cast by either providing a +1 (in favour of the proposal), a 0 (no opinion) or a -1 (against the proposal). Any negative votes must be accompanied with full explanations, to reduce friction within the community and to stimulate discussion of better solutions.

Anyone can vote, though only committer votes count. Non-committer votes are considered ‘non-binding’, i.e. recommendations or expressions of preference that help the committers decide what is best for the community.

Sustainability

The key to the Apache Software Foundation’s sustainability model, and to the projects underneath the ASF umbrella, is the emphasis on reducing ‘conservative’ resources (e.g. money, energy, time) whilst emphasising non-conservative resources (e.g. fun, respect, friendship, visibility).

Sustainability is achieved by removing some of the traditional constraints on the lifetime of a project (the conservative resources), and focusing on attributes that have proved to be good for long-term project viability. The good health of the community is guaranteed by providing a neutral environment for developers to interact as peers. This has been seen to work in many ASF projects.

When academic software development projects consider their sustainability, joining a foundation such as the ASF can be a viable option. If there is sufficient interest in the software, the ASF has a separate project called the Incubator which deals with these new projects. The Wookie project is an example of software developed in an academic setting that has joined the Incubator. If you are considering the sustainability of your project, feel free to contact us.