How to share single Maven local repo among multiple builds with Takari
Recently we started using Kubernetes plugin in our Jenkins for spinning up agents on demand.
Every time a build starts it creates a new pod and once it the build finishes the pod is destroyed. The agents are not reusable.
We use maven as a build tool and we have some jobs that download 7-8 Gb of dependecies. Sometimes the builds took 1 hour - 1 hour and a half to download the dependencies and most of the time the builds failed due to timeout. We created a volume that was mapped with the maven local repo of the agents, but maven local repository is not safe for concurrent access and we had to find something else.
Luckily we found a maven plugin called Takari. Takari enables safe concurrent use of the local repository. In order to share local repo among multiple builds you have to add the followin code in your pipelines bevore running maven tasks:
After we added this plugin it takes minutes for our builds to execute. We run them in multibranch pipelines and they can share the local repo.
Every time a build starts it creates a new pod and once it the build finishes the pod is destroyed. The agents are not reusable.
We use maven as a build tool and we have some jobs that download 7-8 Gb of dependecies. Sometimes the builds took 1 hour - 1 hour and a half to download the dependencies and most of the time the builds failed due to timeout. We created a volume that was mapped with the maven local repo of the agents, but maven local repository is not safe for concurrent access and we had to find something else.
Luckily we found a maven plugin called Takari. Takari enables safe concurrent use of the local repository. In order to share local repo among multiple builds you have to add the followin code in your pipelines bevore running maven tasks:
sh """ curl -O http://repo1.maven.org/maven2/io/takari/aether/takari-local-repository/0.10.4/takari-local-repository-0.10.4.jar mv takari-local-repository-0.10.4.jar ${M2_HOME}/lib/ext curl -O http://repo1.maven.org/maven2/io/takari/takari-filemanager/0.8.2/takari-filemanager-0.8.2.jar mv takari-filemanager-0.8.2.jar $M2_HOME/lib/ext """
Here you can see an example of the pod template:
apiVersion: v1 kind: Pod metadata: name: maven-cache labels: jenkins: maven-cache-node spec: securityContext: runAsUser: 1000 fsGroup: 1000 containers: - name: jnlp image: jenkins/jnlp-slave:3.27-1-alpine:latest imagePullPolicy: Always args: ['$(JENKINS_SECRET)', '$(JENKINS_NAME)'] workingDir: /home/jenkins securityContext: privileged: false runAsUser: 1000 fsGroup: 1000 env: - name: JENKINS_URL value: https://your-jenkins.com - name: HOME value: /home/jenkins volumeMounts: - mountPath: "/home/jenkins/.m2" name: maven-cache volumes: - name: maven-cache persistentVolumeClaim: claimName: maven-cache
You can find the full example here.
Comments
Post a Comment