How to Shade Dependencies in Java with Bazel
Shading Dependencies
What: Shading is the process of making two versions of a dependency available at compile time to Java code.
Why: Two versions of a dependency otherwise will “shadow” or conflict with each other on the classpath if they share the same FQN (Fully Qualified Name). This can cause runtime errors if your code loads the incorrect version of the dependency to satisfy direct or indirect (transitive) dependencies.
When: Shading is recommended only when upgrading, replacing, or removing the dependency conflict is not an available option. This is most likely to occur when a dependency is updated, but some of your code needs to continue to depend on an outdated dependency version due to errors from the upgraded version.
How: A dependency (generally the outdated version) will be put in a fat Jar containing itself and all of its dependencies. Then that dependency will be renamed to have a new FQN that no longer conflicts with the other dependency (generally the up to date version). We can do this via bazel_jar_jar.
Required Bazel Configuration
Creating an alternative Maven repository label
A maven repository label lets you define a set of dependencies and the resulting resolved transitive dependency chain. In our example, the default maven repository label is @maven and is defined in a MODULE.bazel file or a file like maven_artifacts.MODULE.bazel included in the main file.
We need to create an alternative repository label to have a version of dependency that is different from the one included in the @maven repository label.
Below is an opinionated annotated alternate repository label simialr to the example in rules_jvm_external. Annotations are marked with NOTE: to contrast them with comments we suggest you include.
# NOTE: Include a quick explanation of why the alternative label is required. Make sure to include references for determining if and when the alternative label can be removed in the future. For most alternatives, schedule this for a specific time. Keeping alternative labels short lived ensures that dependencies stay up to date.
# shaded_guava label is required because of the guava version bug referenced in examplecomponent issue link dealing with the guava dependency after version 26. It should be re-evaluated after DATE. If still required, DATE should be updated to a new re-evaluation date.
# NOTE: Add a copyable terminal command to repin the dependencies in your label
# Run `REPIN=1 bazel run @shaded_guava_maven//:pin` after updating.
maven.install(
# NOTE: The suggested naming convention is to give your repository label name in the form of `usecase_maven`
name = "shaded_guava_maven",
# NOTE: Here is the list of dependencies you require to vary from the default `@maven` label. It should be as short as possible, but no shorter.
artifacts = [
"com.google.guava:guava:26.0-jre", # shaded version here for bug referenced above, use normal version instead elsewhere
],
# NOTE: We should always throw an error if we create duplicate versions in our dependencies.
duplicate_version_warning = "error",
# NOTE: We should always throw an error if we haven't kept our lock file up to date.
fail_if_repin_required = True,
# NOTE: Fetching sources helps when we want to debug into our dependencies
fetch_sources = True,
# NOTE: `ignore_empty_files` removes a failure case not applicable to most usages.
ignore_empty_files = True,
# NOTE: You'll want multiple lock files for your different dependency trees. For more info, see https://github.com/bazel-contrib/rules_jvm_external?tab=readme-ov-file#multiple-maven_installjson-files
lock_file = "shaded_guava_maven_install.json",
# NOTE: If you have an Artifactory instance for caching performance and security constraints, use it here. Deviations from this convention should have additional documentation with justification.
repositories = [
"https://artifactory.myorg.net/artifactory/repo",
],
# NOTE: We should always use `strict_visibility` to ensure we don't use undeclared direct dependencies via the transitive dependencies of our other direct dependencies.
strict_visibility = True,
# NOTE: We should always use the "pinned" `version_conflict_policy` to ensure the resolved transitive dependencies do not change unexpectedly
version_conflict_policy = "pinned",
)
# NOTE: load the new maven repository we defined previously
use_repo(maven, "shaded_guava_maven")
Packaging a Fat Jar
An alternative maven label only gives you a way to reference two different sets of direct dependencies and their resulting resolved transitive dependency chain in Bazel, it doesn’t handle how to package the actual referenced versions in your jar files in a way that won’t cause conflicts.
A fat jar lets you put a set of direct dependencies and the entire set of resolved transitive dependencies for them into a single jar separate from your other jar files.
You define this fat jar in a BUILD.bazel file located based on its usage. For example, a fat jar only used by a single component would go into the BUILD file located in the path like //path/to/component-name.
You can see below an example of how to define the fat jar utilizing the same alternate maven repository label we defined in the previous example.
java_binary(
name = "examplecomponent",
main_class = "none",
runtime_deps = [
"@shaded_guava//:com_google_guava_guava",
],
)
Renaming the Dependency
Now that you have a fat jar file with the alternative dependency versions, you also need to be able to reference them separately on the Java classpath. We do this by renaming the packages in the fat jar.
For the new name, we use a convention of com.myorg.mynamespace.mypackage.shaded.*. mynamespace.mypackage should be scoped down based on usage. For usage only in a single package, the corresponding Java package name is probably appropriate. This convention will help ensure that 1) you don’t have a conflict on the new name and 2) someone reading your code will be able to clearly see these are shaded dependencies.
The shading of the fat jar dependencies will be defined in the same BUILD file as the fat jar itself. We do this using the bazel_jar_jar library. A simple example is given below using the same fat jar we defined above.
jar_jar(
# NOTE: We recommend a naming convention of shaded-<usage> to ensure future developers will know we are utilizing a shaded dependency
name = "shaded-examplecomponentjar",
# NOTE: See the bazel_jar_jar advanced documentation for more renaming dependencies
inline_rules = ["rule com.google.common.collect.** com.mynamespace.mypackage.shaded.com.google.common.collect.@1"],
# NOTE: There is an implicit `_deploy.jar` target based on the name of fat jar target
input_jar = ":examplecomponent_deploy.jar",
)
Changes to Consuming Java Code
So you have shaded dependencies in a fat jar now defined in a Bazel target. You reference that target in your components’ dependencies just like you would any other target.
DEPS = [
# NOTE: There's not a path here before the `:` in this example because we defined it in the same BUILD file as the shading target.
":shaded-examplecomponentjar",
# NOTE: You refer to other dependencies in the default `@maven` label the same as usual
"@maven//:com_google_code_findbugs_jsr305",
]
Now that Bazel knows where these shaded dependencies, you now need to modify your Java code to refer to them by their new name.
import com.mynamespace.mypackage.shaded.com.google.common.collect.ImmutableList;
Exploring Build Results
The output of Bazel is available in the bazel-out directory. Under that directory on my Mac laptop, the output of a specific target is under darwin_arm64-fastbuild/bin/path/to/target. In the subdirectories of that folder, I can find the Jar files that my target created.
In the examples above that includes the shaded dependency jar path/to/mypackage/shaded-examplecomponentjar.jar.
We can view the contents of these Jar files with the jar CLI command. The two options you should be familiar with are:
jar tf <jar-file>- List contents
jar xf <jar-file> <archived-file(s)>- Extract contents
In your shaded Jar should be one or more pom.xml files letting you confirm the exact dependency versions packaged inside them.
Further Reading
Context on Shading in Java Illustrated with Gradle Examples