How to share single Maven local repo among multiple builds with Takari
Recently we started using Kubernetes plugin in our Jenkins for spinning up agents on demand.
Every time a build starts it creates a new pod and once it the build finishes the pod is destroyed. The agents are not reusable.
We use maven as a build tool and we have some jobs that download 7-8 Gb of dependecies. Sometimes the builds took 1 hour - 1 hour and a half to download the dependencies and most of the time the builds failed due to timeout. We created a volume that was mapped with the maven local repo of the agents, but maven local repository is not safe for concurrent access and we had to find something else.
Luckily we found a maven plugin called Takari. Takari enables safe concurrent use of the local repository. In order to share local repo among multiple builds you have to add the followin code in your pipelines bevore running maven tasks:
After we added this plugin it takes minutes for our builds to execute. We run them in multibranch pipelines and they can share the local repo.
Every time a build starts it creates a new pod and once it the build finishes the pod is destroyed. The agents are not reusable.
We use maven as a build tool and we have some jobs that download 7-8 Gb of dependecies. Sometimes the builds took 1 hour - 1 hour and a half to download the dependencies and most of the time the builds failed due to timeout. We created a volume that was mapped with the maven local repo of the agents, but maven local repository is not safe for concurrent access and we had to find something else.
Luckily we found a maven plugin called Takari. Takari enables safe concurrent use of the local repository. In order to share local repo among multiple builds you have to add the followin code in your pipelines bevore running maven tasks:
sh """
curl -O http://repo1.maven.org/maven2/io/takari/aether/takari-local-repository/0.10.4/takari-local-repository-0.10.4.jar
mv takari-local-repository-0.10.4.jar ${M2_HOME}/lib/ext
curl -O http://repo1.maven.org/maven2/io/takari/takari-filemanager/0.8.2/takari-filemanager-0.8.2.jar
mv takari-filemanager-0.8.2.jar $M2_HOME/lib/ext
"""
Here you can see an example of the pod template:
apiVersion: v1
kind: Pod
metadata:
name: maven-cache
labels:
jenkins: maven-cache-node
spec:
securityContext:
runAsUser: 1000
fsGroup: 1000
containers:
- name: jnlp
image: jenkins/jnlp-slave:3.27-1-alpine:latest
imagePullPolicy: Always
args: ['$(JENKINS_SECRET)', '$(JENKINS_NAME)']
workingDir: /home/jenkins
securityContext:
privileged: false
runAsUser: 1000
fsGroup: 1000
env:
- name: JENKINS_URL
value: https://your-jenkins.com
- name: HOME
value: /home/jenkins
volumeMounts:
- mountPath: "/home/jenkins/.m2"
name: maven-cache
volumes:
- name: maven-cache
persistentVolumeClaim:
claimName: maven-cache
You can find the full example here.

Comments
Post a Comment