After developing parallel-zip-gradle-plugin, which dramatically speeds up the standard ‘zip’ task in Gradle, I was wondering what could be my next project. First I had to ask myself why I want to work on these projects at all. It’s not like I need some cash on the side. It is also not like I have nothing to do after a whole day in the office - quite the contrary, hobbies, exercise, groceries, cleaning, proper eating and supplementation already compete for the ‘critical section’ between 6 pm and 10 pm. For the next project to make sense it needs to be impactful in one way or another. It needs to give me some satisfaction when thinking about it while going to sleep.

From the long backlog of ideas, in the end I decided to tackle the problem of slow building of fat and shaded JARs.

Why?

The reasoning went more or less like this:

Countless companies around the world are building their software in Java. The majority of them build JARs many times per day. Just at the company I work for we have around ~100 builds per day on GitHub CI. Add to that another 100 local builds. And that’s just for one repository. Multiplying this by the number of repositories easily gives a number in the ballpark of tens of thousands of builds per day. Each of them finishes by building not just one but often even four shaded JARs. Multiply this by the number of Java developers or companies in the world… you get the idea.

Each of these shaded JARs is often on the critical path in Claude’s or (if you still have humans doing any ‘real’ work) a software developer’s feedback loop. Each of these JARs is uploaded and downloaded to/from cache on CI. Each of these shaded JARs consumes CPU cycles to build. It occupies space on some real, physical SSD (no matter how abstract they have become through cloud services - in the end they end up on some SSD).

All of this affects bandwidth, disk and CPU usage around the world. It not only has an environmental impact but, probably more importantly to managers and CEOs, a cost impact. Obviously it affects the speed at which a team can ship features or fix bugs. Any improvement in this area, however small, would have an incredible impact on the world (no matter how ‘invisible’ it could feel - the math doesn’t lie).

Is anyone going to use it?

Obviously it is hard to say with confidence. Especially knowing my approach and view on marketing - I hate self-promoting posts on LinkedIn, Reddit or YouTube. I truly would like to see an idealistic world in which all work is judged by its merit and not by how ‘visible’ it is (this is what matters the most in the office, sad truth).

Nevertheless, some kind of demand might be there. Some time ago (ok, a long time ago) HubSpot wrote a post (https://product.hubspot.com/blog/the-fault-in-our-jars-why-we-stopped-building-fat-jars) about how their annoyance with fat JARs sent them on a journey to pursue other solutions. Some Reddit conversations can be found on this topic too (i.e. https://www.reddit.com/r/java/comments/1945ba1/the_evils_of_the_fatjar/), and the groupthink seems biased towards keeping good old fat JARs over solutions that ditch them and ship dependencies in some other way.

Overall… I was expecting to find much more on this topic, but it seems not many Java developers are worried about performance (which matches my experience in the corporate world over the last two decades).

Nevertheless, I am curious how bad our current default fat and shaded JAR solutions are and what can be done.

Landscape

Before we answer that question, a quick reminder of what we are dealing with:

• A fat JAR is just a zip file with all the compiled .class files along with all dependencies
• A shaded JAR is the same as a fat JAR with one difference - classes are sometimes renamed to avoid conflicts

The cool thing about them is that you can very easily run them using java -jar my-fat.jar. Now - how do most people build these JARs? Unless you are using Spring Boot, the odds are you use the abandoned com.github.johnrengelman.shadow or its re-grouped version com.gradleup.shadow. They are very reliable and fairly well maintained, however they still feel slow on larger projects. This is what we will be competing against.

What can be optimized?

This is what we will discover. The Shadow plugin’s primary goal was not extreme performance but rather reliability. Therefore, as with many other projects, the speed of execution was not at the forefront of each commit or pull request merged over there. One thing that for sure could be designed better is its support (or rather lack thereof) for parallelization. But that’s an easy target - one could argue that it is not even fair to pick on it.

The honest answer is - I don’t know yet. Based on my experience I have a strong conviction that it is far from being optimized. Let’s find out.