On Saturday September 15, I attended a free workshop about continuous delivery by Robert Chatley, in Amsterdam. This workshop was organized by Ugly Duckling.

Continuous delivery is most about getting feedback much faster than with traditional (i.e. conservative) delivery strategies. Traditionally releasing is error-prone and time consuming, and often quite stressful. Teams and project managers want to release as few times as possible because it is so hard. The agile mindset teaches that if something is this hard you should it more often (practice it) so that you get better at it. Continuous delivery adopts this mindset and is about removing the bottlenecks from the delivery process, so that you can deliver in minutes rather than hours (or sometimes even days), at any time. Read more…

A few weeks ago a customer needed two web services. One of them needed to return an image, the other some data in XML document form. The contract of these services didn’t really matter. And of course, haste was required. They had other companies who wanted to start their integration with these services. Read more…

As soon as you start using third party libraries in your project, you also introduce all the bugs in those libraries into your project.

There are various ways to handle these kinds of situations:

• Sometimes you won’t even notice the bugs being present, when you don’t use that section of the library.
• When you do notice a bug, you can upgrade to newer version of the library in which the bug was already fixed.
• You might create a work-around in your code.
• Or you could patch the third-party library.

Sometimes the patch already is present in the trunk of the project, but no newer release is made yet. Encouraging a release-early, release-ofter strategy sometimes work to get a stable release without the bug being present, but it’s good to have alternatives. Most likely you aren’t a committer on the project of the library, so you become dependend on other developers, which could create problems for your deadlines.

This procedure in this post will describe a way to create a stable patched released version of the third party library in your own software repository.

In this post I will use the recaptcha4j library as the example of the software to patch. The only available released version of that library is net.tanesha.recaptcha4j:recaptcha4j:0.0.7. The code in this library still references the server URL’s on https://api-secure.recaptcha.net instead of the newer https://www.google.com/recaptcha/api. Since the names in the SSL certificates won’t match various browsers will show warnings or won’t work at all on the old URL’s. So the task at hand is to create a patched version of the library, which will use the new URL’s.

One way of doing this is copying the entire project into your own source code management tool, make the changes and release software. I prefer to keep the original software as seperate as possible from my own changes. So my way of doing this is:

1. Create a maven project for the patched version.
2. Extracting the original classes out of the third party library during a maven build cycle.
3. Replace the classes, which need to be patched by custom versions.
4. Create a new jar-file containing the combination of original and patched classes.

Note: Of course I use some naming convention to prevent version conflict between the original software and the patched version.

Create a maven project for the patched version.

Of course you need a project location in your own source code management tool and create the default maven project structure with a pom.xml.

To identify the patched version of the library I use the groupId and artifactId of the original library, so everybody will still recognize dependency on the third party library. Within the version of I add an extra version-digit plus a describtion of the patched.
In our example it would become:

  <groupId>net.tanesha.recaptcha4j</groupId>
  <artifactId>recaptcha4j</artifactId>
  <version>0.0.7.1-iprofs-https-patched-SNAPSHOT</version>

Within this pom I add a dependency to the original ‘broken’ version of the third party library. This will add all classes and dependencies to the classpath of my project enabling my IDE to use those classes during the build.

  <dependency>
    <groupId>net.tanesha.recaptcha4j</groupId>
    <artifactId>recaptcha4j</artifactId>
    <version>${original.recaptcha4j.version}</version>
    <type>jar</type>
  </dependency>

Extracting the original classes

During the build I use the maven-dependency-plugin to extract the classes from the original version to the target build location. This will ensure all classes will become part of the new patched jar-file. You need to exclude all broken classes from extraction, because your patched versions will not become part of the jar-file if the originals are already present in the target location.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <executions>
    <execution>
      <id>unpack</id>
      <phase>generate-sources</phase>
	<goals>
	  <goal>unpack</goal>
	</goals>
	<configuration>
	  <artifactItems>
	    <artifactItem>
		<groupId>net.tanesha.recaptcha4j</groupId>
		<artifactId>recaptcha4j</artifactId>
		<version>${original.recaptcha4j.version}</version>
		<type>jar</type>
		<overWrite>true</overWrite>
		<outputDirectory>${project.build.directory}/classes</outputDirectory>
		<excludes>
		  **/ReCaptchaImpl.class,**/ReCaptchaFactory.class
		</excludes>
	    </artifactItem>
	  </artifactItems>
	</configuration>
     </execution>
   </executions>
 </plugin>

Replace the classes, which need to be patched by custom versions.

Within my project I create the same package structure and classes for all classes, which need to be patched.
Extracting them from a source jar is a very good way to get started or copy them from the libraries version control system.
In my example it would be net.tanesha.recaptcha.ReCaptchaImpl.

Create the patched jar-file.

For this step you don’t need to do anything special. If you properly excluded the patched classes the maven build process will extract and compile all classes to the proper locations and the maven-jar-plugin will build your patched jar-file. Using your normal maven release process you are now able to create a patched stable release version and deploy it into your own software repository.

Tricky situations.

After using this procedure to create a patched version of the recaptha library and redeploying the software I still noticed that references to the old server URL’s were being used. The cause of this weird behaviour was a compiler optimalization called ‘Constant folding‘. ’Constant folding’ is the proces of replacing all references to or calculations of constants values by the final constant value in the byte code of the compiled classes.

Within the recaptcha library the ReCaptchaFactory references the constants defined in the ReCaptchaImpl. In the original class file for the ReCapthcaFactory the values still were the server URL’s which we needed to patch in the first place. Therefor the exclusions in the dependency extraction does contain that Factory as well.

Recommendations.

Contribute back to the original project.

If you went through all this trouble to fix a bug in an open source library, it is recommended that you create a patch of your changes and submit those to the open source project. Many project welcome such contributions and you get to make the world a better place for everybody.

Monitor releases for the original project.

Subscribe to the release mailing list of the original project and monitor their future releases. As soon as the provided a release, which incorporated your patch, switch to using the normal orginal library again.

Many developers already use a lot of open-source software in their development process. They mostly use libraries offered as open-source by developer groups, but some companies even offer their complete product as open-source. This all looks as one can use it for free and with any form of integration. However, most of those software package come with a software license similar as almost commercial software does. Only open-source software is by many considered ‘free’, but what is a free software license? A free software license grants recipients extensive rights to modify and redistribute, which would otherwise be prohibited by copyright law.

HISTORY

In the mid-1980s, the GNU project created for each of hist software packages a free software license. The first free license in history, the GCC General Public License, was applied to the GNU Compiler Collection and was initially published in 1987. The first BSD license dates to 1989 and was amongst the first free software licenses. In 1989, the first version of the GNU General Public License (GPL) was published, but within 2 years an updated Version 2 of the GPL (GPLv2) was released. GPLv2 went on to become the most widely used free software license. With the rise of the Internet to the public in the mid-90s and until the mid-00s, a new trend started where companies and projects wrote their own licenses, or adapting others’ licenses to insert their own name. From that point on also the complexity of licenses increased significantly due to incompatibilities. For instance, the GNU GPL version 2 license, has been brought to court, first in Germany and later in the USA. In the German case the judge did not explicitly discuss the validity of the GPL’s clauses but accepted that the GPL had to be adhered to “If the GPL were not agreed upon by the parties, defendant would notwithstanding lack the necessary rights to copy, distribute, and make the software ‘netfilter/iptables’ publicly available”, because the defendant did not comply with the GPL, it had to stop using the GPL-ed software. The US case (MySQL vs Progress) was settled before a verdict was arrived at, but at an initial hearing the judge “saw no reason” that the GPL would not be enforceable. After this, different groups promoting and defending specific software licenses started to exist.

SOME MAINSTREAM LICENSES

The following section provides a summary of well-known licenses and how they could be used effectively in commercial software.

NOTE: The following statements are guidelines, but for usage of software with a specific license is advised to search legal council as, for instance, this might also change per country’s law.

GPL [1]

The GNU General Public License (GNU GPL or simply GPL) is the most widely used free software license, originally written by Richard Stallman for the GNU Project. GPL is based on the copyleft license that derived works van only be distributed under the same license terms. Anyone using a software library with a GPL license must effectively assign the same GPL license to its final software product.
With the concept ‘copyleft’ an author surrenders some but not all rights under copyright law. Instead of allowing a work to fall completely into the public domain (where no ownership of copyright is claimed), copyleft allows an author to impose some restrictions on those who want to engage in activities that would more usually be reserved by the copyright holder. Under copyleft, derived works may be produced provided they are released under the compatible copyleft scheme.

BSD license [2]

The original BSD license was used for the Berkeley Software Distribution (BSD), a Unix-like operating system after which it is named. The license was initially written by the Regents of the University of California, because BSD was first written at the University of California, Berkeley.
The BSD license are a family of permissive free software licenses and software libraries with the BSD license allows proprietary use. Such a software library can be incorporated without enforcing the newly created software to be release under the same BSD license as GPL would have done. Therefore, works based on the software may be released under a proprietary license or as closed source software.

MIT License [3]

The MIT License is a permissive free software license originating at the Massachusetts Institute of Technology (MIT). The license allows reuse within proprietary software provided all copies of the licensed software include a copy of the MIT License terms. The resulting software that incorporates a library with a MIT license  retains its proprietary nature even though it incorporates software under the MIT License. In short, all derived works require that the copyrights of the incorporated software is attributed, but may have a different license.
A well-know derived license is the XFree86 project that was the X-Windows system for Unix like operating systems.

Lesser GPL [4]

The GNU Lesser General Public License (LGPL) is created by the Free Software Foundation as a compromise between the strong-copyleft GNU General Public License (GPL) and permissive licenses such as the BSD licenses and the MIT License. The main difference between GPL and LPGL is that the latter allows software linked against LGPL software to be any other type of software license, but any derivative work of the LPGL-ed software falls under the LGPL license. Essentially, if you use the LGPL software you have the freedom to choose the license, but if you are modifying the software itself those changes must be published under LGPL.

Apache License [5]

The apache License is a free software license created by the Apache Software Foundation. This type of license allows users of the software the freedom to use the software for any purpose, to distribute it, to modify it, and to distribute modified versions of the software, under the terms of the license. But the Apache License does not require modified versions of the software to be distributed using the same license (in contrast to copyleft licenses), but in every licensed file, any original copyright, patent, trademark, and attribution notices in redistributed code must be preserved. In case a licensed file has been changed also a statement must be added that it is changed compared to the original licensed file.

Multi-License

This approach to licensing is used by commercial companies that like to have their product offered as open source. That way and most commonly done, commercial companies offer under an open source license the software to be used for free as long it is not for commercial usage. As soon the user would like to use the software commercially a different commercial license is applicable.

SUMMARY

When choosing open source software as components for your application it is important to realize the eventual use of the application. Is it commercially used or distributed are the main questions that impact the choice. Almost any software that is used commercially is better not to implement with GPL licensed software as you application is that also GPL and should be distributed as open source. Various other licenses, such as BSD, MIT, Apache or the LGPL license, have little impact on your choice of license of the new application and allow commercial usage.

References:

[1] http://en.wikipedia.org/wiki/GNU_General_Public_License
[2] http://en.wikipedia.org/wiki/BSD_licenses
[3] http://en.wikipedia.org/wiki/MIT_License
[4] http://en.wikipedia.org/wiki/LGPL_license
[5] http://en.wikipedia.org/wiki/Apache_license