vkuzel.com

Gradle multi-project using cross dependent modules

2016-01-23

I have a framework which consists of several modules. On top of this framework I'd like to build various projects where each project can use one or more modules of this framework. In real world these projects can represent framework's implementations for customers. Framework's modules can depend on each other. For this task I decided to use Gradle build system.

Part of this article is an example project on GitHub.

Setting up the build

I've created three modules that are shared by two root projects. Project1 and project2. Second project doesn't use one of the modules as it shown by following diagram. Both root projects has very similar build scripts so I will focus on project1 for the rest of this article.

:project1
+--- :module1
|    \--- :core-module
\--- :module2
     \--- :core-module

:project2
+--- :module1
     \--- :core-module

Root project's build script contains buildscript block which introduces plugin dependency and other common configuration used by Gradle when compiling/building project. The build script applies spring-boot-multi-project on the root project. This plugin is wrapper of the Gradle Spring Boot plugin and enhances it's functionality. The plugin also adds new task that generates serialized version of dependency graph. We will get to the dependency graph later.

// If your build script has some dependencies on plugins you will have to
// declare these dependencies in root project's buildscript block.
//
// @see https://docs.gradle.org/current/userguide/organizing_build_logic.html#sec:external_dependencies
//
// Unfortunately buildscript block can't be externalized into a script plugin
// so following code is going to be duplicated in build scripts of all root
// projects.
//
// @see https://discuss.gradle.org/t/how-do-i-include-buildscript-block-from-external-gradle-script/7016/2
buildscript {
    repositories {
        mavenCentral()
        maven { url "https://jitpack.io" }
    }
    dependencies {
        // This project uses Gradle Spring Boot Multi Project Plugin that wraps
        // the Gradle Spring Boot plugin and adds some more functionality.
        classpath "com.github.vkuzel:Gradle-Spring-Boot-Multi-Project-Plugin:2.4.0"
    }
}

// Spring Boot Multi Project Plugin adds Maven Central and JitPack repositories
// to all projects of this multi-project build. So it's not necessary to
// specify them separately.
//
// @see https://github.com/vkuzel/Gradle-Spring-Boot-Multi-Project-Plugin

ext {
    // You need to specify which sub-project contains SpringBootApplication
    // annotation so the plugin can apply findMainClass task properly.
    springBootProject = 'core-module'
}

apply plugin: 'spring-boot-multi-project'

dependencies {
    compile project('module1')
    compile project('module2')
    // If project A has compile dependency on B and B has compile dependency on
    // some external project then A will have this external project dependency
    // on it's classpath too. On the other hand testCompile dependencies does
    // not work this way. TestCompile dependencies are used only for testing
    // sub-project where is declared. So to be able to use a certain dependency
    // you need to declare it in every sub-project where needed.
    //
    // @see http://stackoverflow.com/questions/6023188/my-gradle-configuration-does-not-use-the-correct-classpath-during-build/10633623#10633623
    testCompile project(path: ':core-module', configuration: 'testFixturesUsageCompile')
    testCompile('org.springframework.boot:spring-boot-starter-test')
}

jar {
    baseName = 'project1'
    version = '0.0.1-SNAPSHOT'
}

There are three modules (sub-projects). A core module which is Spring Boot project and two modules (module1 and module2) which depends on core module and extends its functionality. These modules forms the framework. The framework is just a collection of libraries so it can't work as a standalone application.

Core module depends on Spring Boot starter project so to be compiled it needs to have compile dependency configuration set to spring-boot-starter. To be able to test this module it also need to have declared testCompile dependency on spring-boot-starter-test.

Note that module's build scripts does not contain repository configuration or does not apply any plugin. Repository configuration is included in spring-boot-multi-project plugin which is applied in root project's build script. This plugin adds Maven Central Repository and JitPack repository to every module including root project.

ext {
    // Spring Boot Multi Project Plugin adds discoverProjectDependencies task
    // that stores each project's dependencies into a file. Following property
    // holds file's path relative to resources directory of each project of
    // multi-project build.
    serializedProjectDependenciesPath = "dependencies.ser"
}

dependencies {
    compile('org.springframework.boot:spring-boot-starter')
    testCompile('org.springframework.boot:spring-boot-starter-test')
}

Module1 depends on core module. And because core module contains Spring Boot project it's not necessary to declare Spring Boot compile dependency in module1. Compile dependencies are transitive so all dependencies from core module are going to be placed on classpath of module1 too.

This rule doesn't apply to testCompile dependencies. When a module (subproject in Gradle's terminology) is tested it's tested separately. This means module is first compiled with all it's dependencies (not root project's dependencies) and it's own testCompile dependencies (not a root project's or subproject's dependencies) and then tests are executed. This means three things.

  1. You need to add necessary testCompile dependencies to each subproject's build script.

  2. If a module depends on a resource which is not added in compile dependencies you need to add this resource as a testCompile dependency to build script of the module. For example the resource can be a configuration file located in root project. To add directory as a testCompile dependency you can use file command testCompile files("path/to/resource").

  3. If you want to use some shared class in your tests you will need to place it to main source set or a new source set and declare a dependency to it. In this example a source set called testFixtures is created in core module. Support for testFixtures source set is added in spring-boot-multi-project plugin.

dependencies {
    compile project(':core-module')
    testCompile project(path: ':core-module', configuration: 'testFixturesUsageCompile')
    testCompile('org.springframework.boot:spring-boot-starter-test')
}

Of course Gradle's multi-project build has to contain settings.gradle file which tells Gradle where all modules (in Gradle's terminology sub projects) are located.

include 'core-module'
include 'module1'
include 'module2'

// Because a project modules are located in the framework directory we have to
// modify modules directories.
//
// @see https://github.com/gradle/gradle/blob/master/settings.gradle
rootProject.children.each { project ->
    String projectDirName = sprintf("../framework/%s", project.name)
    project.projectDir = new File(settingsDir, projectDirName)
    assert project.projectDir.isDirectory()
    assert project.buildFile.isFile()
}

Project dependencies in application

Sometimes it is useful to know module dependencies in the application. For example if you want to execute data model updating scripts you will need to know which script should run first (core project's) and which last (root project's which depends on everything else).

To do this you can use project dependencies file generated by discoverProjectDependencies task. With this graph you can easily sort resources on classpath in certain order as it is shown in ProjectDependencyManager service.

Running the project

To start projects or to build it you can experiment with tasks provided by Gradle in project1 or project2 directories. For example:

gradle discoverProjectDependencies - to serialize project dependencies into a file
gradle bootRun - to run the application
gradle build - to build jar file
gradle test - to run all tests

If you'd like to open project in IntelliJ's IDEA use the import Gradle project functionality instead of simple open project. Import project will open all framework modules as a project modules so you'll have all required source codes available via project panel. You can also use IDEA's Gradle plugin to easily modify build scripts.

When running project in IDEA please make sure that your run configuration does have root project selected in use classpath of module option.

Be aware of if you don't add any source code into Gradle project then Gradle won't generate empty build directories for this project. Unfortunately it adds these non-existing build directories to a classpath when bootRun or a simmilar task is executed. Invalid paths breaks Java's class loader and it causes problems with loading libraries which usually ends up with java.io.FileNotFoundException: class path resource [] cannot be resolved to URL because it does not exist error message. This can happen in early stages of a project when you create an empty module with some static content but without any Java code.