corda/.ci/dev/smoke/Jenkinsfile

114 lines
4.0 KiB
Plaintext
Raw Normal View History

PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
@Library('existing-build-control')
import static com.r3.build.BuildControl.killAllExistingBuildsForJob
killAllExistingBuildsForJob(env.JOB_NAME, env.BUILD_NUMBER.toInteger())
pipeline {
agent { label 'k8s' }
options { timestamps() }
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
triggers {
issueCommentTrigger('.*smoke tests.*')
}
environment {
DOCKER_TAG_TO_USE = "${env.GIT_COMMIT.subSequence(0, 8)}"
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
EXECUTOR_NUMBER = "${env.EXECUTOR_NUMBER}"
BUILD_ID = "${env.BUILD_ID}-${env.JOB_NAME}"
ARTIFACTORY_CREDENTIALS = credentials('artifactory-credentials')
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
}
stages {
stage('Corda Smoke Tests - Generate Build Image') {
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
steps {
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
script {
pullRequest.createStatus(status: 'pending',
context: 'continuous-integration/jenkins/pr-merge/smokeTest',
description: 'Smoke Tests Building',
targetUrl: "${env.JOB_URL}")
}
withCredentials([string(credentialsId: 'container_reg_passwd', variable: 'DOCKER_PUSH_PWD')]) {
sh "./gradlew " +
"-Dkubenetize=true " +
"-Ddocker.push.password=\"\${DOCKER_PUSH_PWD}\" " +
"-Ddocker.work.dir=\"/tmp/\${EXECUTOR_NUMBER}\" " +
"-Ddocker.build.tag=\"\${DOCKER_TAG_TO_USE}\"" +
" clean pushBuildImage --stacktrace"
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
}
sh "kubectl auth can-i get pods"
}
}
stage('Smoke Tests') {
steps {
script {
pullRequest.createStatus(status: 'pending',
context: 'continuous-integration/jenkins/pr-merge/smokeTest',
description: 'Smoke Tests Running',
targetUrl: "${env.JOB_URL}")
}
sh "./gradlew " +
"-DbuildId=\"\${BUILD_ID}\" " +
"-Dkubenetize=true " +
"-Ddocker.run.tag=\"\${DOCKER_TAG_TO_USE}\" " +
"-Dartifactory.username=\"\${ARTIFACTORY_CREDENTIALS_USR}\" " +
"-Dartifactory.password=\"\${ARTIFACTORY_CREDENTIALS_PSW}\" " +
"-Dgit.branch=\"\${GIT_BRANCH}\" " +
"-Dgit.target.branch=\"\${GIT_BRANCH}\" " +
" allParallelSmokeTest --stacktrace"
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
}
}
}
post {
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
always {
archiveArtifacts artifacts: '**/pod-logs/**/*.log', fingerprint: false
junit '**/build/test-results-xml/**/*.xml'
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
}
PreAllocate pod resources during image build phase (#5587) * use zulu for jdk add some parallel groups * port kubesTest to Java remove asterix from tests listed by ListTests, instead add after allocation * attempt to setup unit test builds with correct github integrations # Conflicts: # .ci/dev/unit/Jenkinsfile * fix issue with github context * add credentials block * start pre-allocating pods for builds * test * add blocks for reporting build stages * add logic to preallocate pods during image building * tidy up Jenkinsfile for unit tests * add magic command line flag to enable preallocation of pods * make docker tag deterministic * fix issue concatenating docker tag inputs * add build type specific Jenkinsfile * try new preallocation approach * make pre-allocation prefix group specific * force deAllocator to wait for pods to be actually deleted * revert jenkinsfiles in .ci * use smarter waiting logic to address review comments * add --stacktrace to builds to help debugging * fix issue with closed stream * add some logging around preallocation * tidy up by refactoring (de)allocate task generation into method * change default from 20 pods to 5 pods * fix issue where docker tag was unstable between building and running tests * more documentation * add some infrastructure around setting the log level for a given build * change preallocation pod duration to 5min * see if fast enough if using combined unit and integration tests * disable unit tests * print out test summaries * try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible * add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
success {
script {
pullRequest.createStatus(status: 'success',
context: 'continuous-integration/jenkins/pr-merge/smokeTest',
description: 'Smoke Tests Passed',
targetUrl: "${env.JOB_URL}testResults")
}
}
failure {
script {
pullRequest.createStatus(status: 'failure',
context: 'continuous-integration/jenkins/pr-merge/smokeTest',
description: 'Smoke Tests Failed',
targetUrl: "${env.JOB_URL}testResults")
}
}
cleanup {
deleteDir() /* clean up our workspace */
}
}
}
//@NonCPS
//static def currentBuildTriggeredByComment(){
// def triggerCause = currentBuild.rawBuild.getCause(org.jenkinsci.plugins.pipeline.github.trigger.IssueCommentCause)
// if (triggerCause) {
// echo("Build was started by ${triggerCause.userLogin}, who wrote: " +
// "\"${triggerCause.comment}\", which matches the " +
// "\"${triggerCause.triggerPattern}\" trigger pattern.")
// } else {
// echo('Build was not started by a trigger')
// }
//
// return triggerCause == null
//}