Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update GitLab Pipeline Type Detection to use Head Pipeline Property #3887

Merged
merged 6 commits into from
Nov 4, 2023

Conversation

X-Guardian
Copy link
Contributor

@X-Guardian X-Guardian commented Oct 21, 2023

what

The GitLab pipeline detection code is currently using a legacy property, pipeline of the Get Single MR API to get details of the pipeline for an MR. This PR changes the property used to HeadPipeline. It also uses the refTarget from the HeadPipeline object rather than manually calculating the ref, and adds a debug line.

The TestGitlabClient_UpdateStatus test has also been updated to add a logger to the GitLab client and fix the expected ref to match the ref in the pipelineSuccess mock data.

why

GitLab recommend using the HeadPipeline property of rather than Pipeline, ref: GitLab REST API - Get single MR - Response and in tests I have done, when creating an MR and straight away calling the 'Get Single MR' API, the Pipeline property returns null in the first 3 seconds, whereas HeadPipeline only returns null for the first second. A null value would make Atlantis use the branch reference for the pipeline rather than the pipeline reference, causing multiple pipelines to display in the UI, as raised in #3373.

I did look at the returned object to see if there was any other property that would indicate whether the HeadPipeline field was ready for reading or not, but I couldn't see anything. The problem is that if the repo has no other pipelines defined for it, the Pipeline/HeadPipeline properties will always be null. I've attached the test script and results from a repo using a GitLab CI pipeline below for reference.

I was also considering adding a single retry with say a 1 second delay to the Get Single MR API call if the HeadPipeline property was null to try and reduce the race condition window further. Interested in any opinions.

tests

GitLab REST API Test Script

GITLAB_TOKEN='glpat-xxxxxxxxxxxx'
GITLAB_HOST='gitlab.lan'
PROJECT_PATH='sheather%2ftest'
SOURCE_BRANCH='head-pipeline-test'
TARGET_BRANCH='main'
MR_TITLE='HeadPipelineTest'

# Create MR
MERGE_REQUEST_ID=$(curl --no-progress-meter --request POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests?source_branch=$SOURCE_BRANCH&target_branch=$TARGET_BRANCH&title=$MR_TITLE" | jq '.iid')

# Get MR
curl --no-progress-meter --request GET --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests/$MERGE_REQUEST_ID" | jq
sleep 0.5
curl --no-progress-meter --request GET --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests/$MERGE_REQUEST_ID" | jq
sleep 0.5
curl --no-progress-meter --request GET --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests/$MERGE_REQUEST_ID" | jq
sleep 0.5
curl --no-progress-meter --request GET --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests/$MERGE_REQUEST_ID" | jq
sleep 0.5
curl --no-progress-meter --request GET --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests/$MERGE_REQUEST_ID" | jq
sleep 0.5
curl --no-progress-meter --request GET --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests/$MERGE_REQUEST_ID" | jq
sleep 0.5
curl --no-progress-meter --request GET --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://$GITLAB_HOST/api/v4/projects/$PROJECT_PATH/merge_requests/$MERGE_REQUEST_ID" | jq
Results
{
    "id": 30,
    "iid": 29,
    "project_id": 2,
    "title": "HeadPipelineTest3",
    "description": null,
    "state": "opened",
    "created_at": "2023-10-21T10:46:13.779Z",
    "updated_at": "2023-10-21T10:46:13.779Z",
    "merged_by": null,
    "merge_user": null,
    "merged_at": null,
    "closed_by": null,
    "closed_at": null,
    "target_branch": "main",
    "source_branch": "head-pipeline-test5",
    "user_notes_count": 0,
    "upvotes": 0,
    "downvotes": 0,
    "author": {
      "id": 2,
      "username": "sheather",
      "name": "Simon Heather",
      "state": "active",
      "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
      "web_url": "https://gitlab.lan/sheather"
    },
    "assignees": [],
    "assignee": null,
    "reviewers": [],
    "source_project_id": 2,
    "target_project_id": 2,
    "labels": [],
    "draft": false,
    "work_in_progress": false,
    "milestone": null,
    "merge_when_pipeline_succeeds": false,
    "merge_status": "checking",
    "detailed_merge_status": "preparing",
    "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
    "merge_commit_sha": null,
    "squash_commit_sha": null,
    "discussion_locked": null,
    "should_remove_source_branch": null,
    "force_remove_source_branch": null,
    "prepared_at": null,
    "reference": "!29",
    "references": {
      "short": "!29",
      "relative": "!29",
      "full": "sheather/test!29"
    },
    "web_url": "https://gitlab.lan/sheather/test/-/merge_requests/29",
    "time_stats": {
      "time_estimate": 0,
      "total_time_spent": 0,
      "human_time_estimate": null,
      "human_total_time_spent": null
    },
    "squash": false,
    "squash_on_merge": true,
    "task_completion_status": {
      "count": 0,
      "completed_count": 0
    },
    "has_conflicts": false,
    "blocking_discussions_resolved": true,
    "subscribed": true,
    "changes_count": null,
    "latest_build_started_at": null,
    "latest_build_finished_at": null,
    "first_deployed_to_production_at": null,
    "pipeline": null,
    "head_pipeline": null,
    "diff_refs": null,
    "merge_error": null,
    "first_contribution": false,
    "user": {
      "can_merge": true
    }
  }
  {
    "id": 30,
    "iid": 29,
    "project_id": 2,
    "title": "HeadPipelineTest3",
    "description": null,
    "state": "opened",
    "created_at": "2023-10-21T10:46:13.779Z",
    "updated_at": "2023-10-21T10:46:13.779Z",
    "merged_by": null,
    "merge_user": null,
    "merged_at": null,
    "closed_by": null,
    "closed_at": null,
    "target_branch": "main",
    "source_branch": "head-pipeline-test5",
    "user_notes_count": 0,
    "upvotes": 0,
    "downvotes": 0,
    "author": {
      "id": 2,
      "username": "sheather",
      "name": "Simon Heather",
      "state": "active",
      "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
      "web_url": "https://gitlab.lan/sheather"
    },
    "assignees": [],
    "assignee": null,
    "reviewers": [],
    "source_project_id": 2,
    "target_project_id": 2,
    "labels": [],
    "draft": false,
    "work_in_progress": false,
    "milestone": null,
    "merge_when_pipeline_succeeds": false,
    "merge_status": "checking",
    "detailed_merge_status": "ci_must_pass",
    "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
    "merge_commit_sha": null,
    "squash_commit_sha": null,
    "discussion_locked": null,
    "should_remove_source_branch": null,
    "force_remove_source_branch": null,
    "prepared_at": null,
    "reference": "!29",
    "references": {
      "short": "!29",
      "relative": "!29",
      "full": "sheather/test!29"
    },
    "web_url": "https://gitlab.lan/sheather/test/-/merge_requests/29",
    "time_stats": {
      "time_estimate": 0,
      "total_time_spent": 0,
      "human_time_estimate": null,
      "human_total_time_spent": null
    },
    "squash": false,
    "squash_on_merge": true,
    "task_completion_status": {
      "count": 0,
      "completed_count": 0
    },
    "has_conflicts": false,
    "blocking_discussions_resolved": true,
    "subscribed": true,
    "changes_count": "1",
    "latest_build_started_at": null,
    "latest_build_finished_at": null,
    "first_deployed_to_production_at": null,
    "pipeline": null,
    "head_pipeline": null,
    "diff_refs": {
      "base_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946",
      "head_sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "start_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946"
    },
    "merge_error": null,
    "first_contribution": false,
    "user": {
      "can_merge": true
    }
  }
  {
    "id": 30,
    "iid": 29,
    "project_id": 2,
    "title": "HeadPipelineTest3",
    "description": null,
    "state": "opened",
    "created_at": "2023-10-21T10:46:13.779Z",
    "updated_at": "2023-10-21T10:46:13.779Z",
    "merged_by": null,
    "merge_user": null,
    "merged_at": null,
    "closed_by": null,
    "closed_at": null,
    "target_branch": "main",
    "source_branch": "head-pipeline-test5",
    "user_notes_count": 0,
    "upvotes": 0,
    "downvotes": 0,
    "author": {
      "id": 2,
      "username": "sheather",
      "name": "Simon Heather",
      "state": "active",
      "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
      "web_url": "https://gitlab.lan/sheather"
    },
    "assignees": [],
    "assignee": null,
    "reviewers": [],
    "source_project_id": 2,
    "target_project_id": 2,
    "labels": [],
    "draft": false,
    "work_in_progress": false,
    "milestone": null,
    "merge_when_pipeline_succeeds": false,
    "merge_status": "checking",
    "detailed_merge_status": "checking",
    "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
    "merge_commit_sha": null,
    "squash_commit_sha": null,
    "discussion_locked": null,
    "should_remove_source_branch": null,
    "force_remove_source_branch": null,
    "prepared_at": null,
    "reference": "!29",
    "references": {
      "short": "!29",
      "relative": "!29",
      "full": "sheather/test!29"
    },
    "web_url": "https://gitlab.lan/sheather/test/-/merge_requests/29",
    "time_stats": {
      "time_estimate": 0,
      "total_time_spent": 0,
      "human_time_estimate": null,
      "human_total_time_spent": null
    },
    "squash": false,
    "squash_on_merge": true,
    "task_completion_status": {
      "count": 0,
      "completed_count": 0
    },
    "has_conflicts": false,
    "blocking_discussions_resolved": true,
    "subscribed": true,
    "changes_count": "1",
    "latest_build_started_at": null,
    "latest_build_finished_at": null,
    "first_deployed_to_production_at": null,
    "pipeline": null,
    "head_pipeline": {
      "id": 65,
      "iid": 65,
      "project_id": 2,
      "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "ref": "refs/merge-requests/29/head",
      "status": "created",
      "source": "merge_request_event",
      "created_at": "2023-10-21T10:46:15.822Z",
      "updated_at": "2023-10-21T10:46:15.822Z",
      "web_url": "https://gitlab.lan/sheather/test/-/pipelines/65",
      "before_sha": "0000000000000000000000000000000000000000",
      "tag": false,
      "yaml_errors": null,
      "user": {
        "id": 2,
        "username": "sheather",
        "name": "Simon Heather",
        "state": "active",
        "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
        "web_url": "https://gitlab.lan/sheather"
      },
      "started_at": null,
      "finished_at": null,
      "committed_at": null,
      "duration": null,
      "queued_duration": null,
      "coverage": null,
      "detailed_status": {
        "icon": "status_created",
        "text": "created",
        "label": "created",
        "group": "created",
        "tooltip": "created",
        "has_details": true,
        "details_path": "/sheather/test/-/pipelines/65",
        "illustration": null,
        "favicon": "/assets/ci_favicons/favicon_status_created-4b975aa976d24e5a3ea7cd9a5713e6ce2cd9afd08b910415e96675de35f64955.png"
      }
    },
    "diff_refs": {
      "base_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946",
      "head_sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "start_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946"
    },
    "merge_error": null,
    "first_contribution": false,
    "user": {
      "can_merge": true
    }
  }
  {
    "id": 30,
    "iid": 29,
    "project_id": 2,
    "title": "HeadPipelineTest3",
    "description": null,
    "state": "opened",
    "created_at": "2023-10-21T10:46:13.779Z",
    "updated_at": "2023-10-21T10:46:17.132Z",
    "merged_by": null,
    "merge_user": null,
    "merged_at": null,
    "closed_by": null,
    "closed_at": null,
    "target_branch": "main",
    "source_branch": "head-pipeline-test5",
    "user_notes_count": 0,
    "upvotes": 0,
    "downvotes": 0,
    "author": {
      "id": 2,
      "username": "sheather",
      "name": "Simon Heather",
      "state": "active",
      "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
      "web_url": "https://gitlab.lan/sheather"
    },
    "assignees": [],
    "assignee": null,
    "reviewers": [],
    "source_project_id": 2,
    "target_project_id": 2,
    "labels": [],
    "draft": false,
    "work_in_progress": false,
    "milestone": null,
    "merge_when_pipeline_succeeds": false,
    "merge_status": "checking",
    "detailed_merge_status": "checking",
    "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
    "merge_commit_sha": null,
    "squash_commit_sha": null,
    "discussion_locked": null,
    "should_remove_source_branch": null,
    "force_remove_source_branch": null,
    "prepared_at": "2023-10-21T10:46:17.091Z",
    "reference": "!29",
    "references": {
      "short": "!29",
      "relative": "!29",
      "full": "sheather/test!29"
    },
    "web_url": "https://gitlab.lan/sheather/test/-/merge_requests/29",
    "time_stats": {
      "time_estimate": 0,
      "total_time_spent": 0,
      "human_time_estimate": null,
      "human_total_time_spent": null
    },
    "squash": false,
    "squash_on_merge": true,
    "task_completion_status": {
      "count": 0,
      "completed_count": 0
    },
    "has_conflicts": false,
    "blocking_discussions_resolved": true,
    "subscribed": true,
    "changes_count": "1",
    "latest_build_started_at": null,
    "latest_build_finished_at": null,
    "first_deployed_to_production_at": null,
    "pipeline": null,
    "head_pipeline": {
      "id": 65,
      "iid": 65,
      "project_id": 2,
      "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "ref": "refs/merge-requests/29/head",
      "status": "pending",
      "source": "merge_request_event",
      "created_at": "2023-10-21T10:46:15.822Z",
      "updated_at": "2023-10-21T10:46:17.383Z",
      "web_url": "https://gitlab.lan/sheather/test/-/pipelines/65",
      "before_sha": "0000000000000000000000000000000000000000",
      "tag": false,
      "yaml_errors": null,
      "user": {
        "id": 2,
        "username": "sheather",
        "name": "Simon Heather",
        "state": "active",
        "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
        "web_url": "https://gitlab.lan/sheather"
      },
      "started_at": null,
      "finished_at": null,
      "committed_at": null,
      "duration": null,
      "queued_duration": null,
      "coverage": null,
      "detailed_status": {
        "icon": "status_pending",
        "text": "pending",
        "label": "pending",
        "group": "pending",
        "tooltip": "pending",
        "has_details": true,
        "details_path": "/sheather/test/-/pipelines/65",
        "illustration": null,
        "favicon": "/assets/ci_favicons/favicon_status_pending-5bdf338420e5221ca24353b6bff1c9367189588750632e9a871b7af09ff6a2ae.png"
      }
    },
    "diff_refs": {
      "base_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946",
      "head_sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "start_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946"
    },
    "merge_error": null,
    "first_contribution": false,
    "user": {
      "can_merge": true
    }
  }
  {
    "id": 30,
    "iid": 29,
    "project_id": 2,
    "title": "HeadPipelineTest3",
    "description": null,
    "state": "opened",
    "created_at": "2023-10-21T10:46:13.779Z",
    "updated_at": "2023-10-21T10:46:17.132Z",
    "merged_by": null,
    "merge_user": null,
    "merged_at": null,
    "closed_by": null,
    "closed_at": null,
    "target_branch": "main",
    "source_branch": "head-pipeline-test5",
    "user_notes_count": 0,
    "upvotes": 0,
    "downvotes": 0,
    "author": {
      "id": 2,
      "username": "sheather",
      "name": "Simon Heather",
      "state": "active",
      "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
      "web_url": "https://gitlab.lan/sheather"
    },
    "assignees": [],
    "assignee": null,
    "reviewers": [],
    "source_project_id": 2,
    "target_project_id": 2,
    "labels": [],
    "draft": false,
    "work_in_progress": false,
    "milestone": null,
    "merge_when_pipeline_succeeds": false,
    "merge_status": "can_be_merged",
    "detailed_merge_status": "ci_must_pass",
    "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
    "merge_commit_sha": null,
    "squash_commit_sha": null,
    "discussion_locked": null,
    "should_remove_source_branch": null,
    "force_remove_source_branch": null,
    "prepared_at": "2023-10-21T10:46:17.091Z",
    "reference": "!29",
    "references": {
      "short": "!29",
      "relative": "!29",
      "full": "sheather/test!29"
    },
    "web_url": "https://gitlab.lan/sheather/test/-/merge_requests/29",
    "time_stats": {
      "time_estimate": 0,
      "total_time_spent": 0,
      "human_time_estimate": null,
      "human_total_time_spent": null
    },
    "squash": false,
    "squash_on_merge": true,
    "task_completion_status": {
      "count": 0,
      "completed_count": 0
    },
    "has_conflicts": false,
    "blocking_discussions_resolved": true,
    "subscribed": true,
    "changes_count": "1",
    "latest_build_started_at": null,
    "latest_build_finished_at": null,
    "first_deployed_to_production_at": null,
    "pipeline": null,
    "head_pipeline": {
      "id": 65,
      "iid": 65,
      "project_id": 2,
      "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "ref": "refs/merge-requests/29/head",
      "status": "pending",
      "source": "merge_request_event",
      "created_at": "2023-10-21T10:46:15.822Z",
      "updated_at": "2023-10-21T10:46:17.383Z",
      "web_url": "https://gitlab.lan/sheather/test/-/pipelines/65",
      "before_sha": "0000000000000000000000000000000000000000",
      "tag": false,
      "yaml_errors": null,
      "user": {
        "id": 2,
        "username": "sheather",
        "name": "Simon Heather",
        "state": "active",
        "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
        "web_url": "https://gitlab.lan/sheather"
      },
      "started_at": null,
      "finished_at": null,
      "committed_at": null,
      "duration": null,
      "queued_duration": null,
      "coverage": null,
      "detailed_status": {
        "icon": "status_pending",
        "text": "pending",
        "label": "pending",
        "group": "pending",
        "tooltip": "pending",
        "has_details": true,
        "details_path": "/sheather/test/-/pipelines/65",
        "illustration": null,
        "favicon": "/assets/ci_favicons/favicon_status_pending-5bdf338420e5221ca24353b6bff1c9367189588750632e9a871b7af09ff6a2ae.png"
      }
    },
    "diff_refs": {
      "base_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946",
      "head_sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "start_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946"
    },
    "merge_error": null,
    "first_contribution": false,
    "user": {
      "can_merge": true
    }
  }
  {
    "id": 30,
    "iid": 29,
    "project_id": 2,
    "title": "HeadPipelineTest3",
    "description": null,
    "state": "opened",
    "created_at": "2023-10-21T10:46:13.779Z",
    "updated_at": "2023-10-21T10:46:17.132Z",
    "merged_by": null,
    "merge_user": null,
    "merged_at": null,
    "closed_by": null,
    "closed_at": null,
    "target_branch": "main",
    "source_branch": "head-pipeline-test5",
    "user_notes_count": 0,
    "upvotes": 0,
    "downvotes": 0,
    "author": {
      "id": 2,
      "username": "sheather",
      "name": "Simon Heather",
      "state": "active",
      "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
      "web_url": "https://gitlab.lan/sheather"
    },
    "assignees": [],
    "assignee": null,
    "reviewers": [],
    "source_project_id": 2,
    "target_project_id": 2,
    "labels": [],
    "draft": false,
    "work_in_progress": false,
    "milestone": null,
    "merge_when_pipeline_succeeds": false,
    "merge_status": "can_be_merged",
    "detailed_merge_status": "ci_still_running",
    "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
    "merge_commit_sha": null,
    "squash_commit_sha": null,
    "discussion_locked": null,
    "should_remove_source_branch": null,
    "force_remove_source_branch": null,
    "prepared_at": "2023-10-21T10:46:17.091Z",
    "reference": "!29",
    "references": {
      "short": "!29",
      "relative": "!29",
      "full": "sheather/test!29"
    },
    "web_url": "https://gitlab.lan/sheather/test/-/merge_requests/29",
    "time_stats": {
      "time_estimate": 0,
      "total_time_spent": 0,
      "human_time_estimate": null,
      "human_total_time_spent": null
    },
    "squash": false,
    "squash_on_merge": true,
    "task_completion_status": {
      "count": 0,
      "completed_count": 0
    },
    "has_conflicts": false,
    "blocking_discussions_resolved": true,
    "subscribed": true,
    "changes_count": "1",
    "latest_build_started_at": "2023-10-21T10:46:19.484Z",
    "latest_build_finished_at": null,
    "first_deployed_to_production_at": null,
    "pipeline": {
      "id": 65,
      "iid": 65,
      "project_id": 2,
      "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "ref": "refs/merge-requests/29/head",
      "status": "running",
      "source": "merge_request_event",
      "created_at": "2023-10-21T10:46:15.822Z",
      "updated_at": "2023-10-21T10:46:19.593Z",
      "web_url": "https://gitlab.lan/sheather/test/-/pipelines/65"
    },
    "head_pipeline": {
      "id": 65,
      "iid": 65,
      "project_id": 2,
      "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "ref": "refs/merge-requests/29/head",
      "status": "running",
      "source": "merge_request_event",
      "created_at": "2023-10-21T10:46:15.822Z",
      "updated_at": "2023-10-21T10:46:19.593Z",
      "web_url": "https://gitlab.lan/sheather/test/-/pipelines/65",
      "before_sha": "0000000000000000000000000000000000000000",
      "tag": false,
      "yaml_errors": null,
      "user": {
        "id": 2,
        "username": "sheather",
        "name": "Simon Heather",
        "state": "active",
        "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
        "web_url": "https://gitlab.lan/sheather"
      },
      "started_at": "2023-10-21T10:46:19.484Z",
      "finished_at": null,
      "committed_at": null,
      "duration": null,
      "queued_duration": 3,
      "coverage": null,
      "detailed_status": {
        "icon": "status_running",
        "text": "running",
        "label": "running",
        "group": "running",
        "tooltip": "running",
        "has_details": true,
        "details_path": "/sheather/test/-/pipelines/65",
        "illustration": null,
        "favicon": "/assets/ci_favicons/favicon_status_running-9c635b2419a8e1ec991c993061b89cc5aefc0743bb238ecd0c381e7741a70e8c.png"
      }
    },
    "diff_refs": {
      "base_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946",
      "head_sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "start_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946"
    },
    "merge_error": null,
    "first_contribution": false,
    "user": {
      "can_merge": true
    }
  }
  {
    "id": 30,
    "iid": 29,
    "project_id": 2,
    "title": "HeadPipelineTest3",
    "description": null,
    "state": "opened",
    "created_at": "2023-10-21T10:46:13.779Z",
    "updated_at": "2023-10-21T10:46:17.132Z",
    "merged_by": null,
    "merge_user": null,
    "merged_at": null,
    "closed_by": null,
    "closed_at": null,
    "target_branch": "main",
    "source_branch": "head-pipeline-test5",
    "user_notes_count": 0,
    "upvotes": 0,
    "downvotes": 0,
    "author": {
      "id": 2,
      "username": "sheather",
      "name": "Simon Heather",
      "state": "active",
      "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
      "web_url": "https://gitlab.lan/sheather"
    },
    "assignees": [],
    "assignee": null,
    "reviewers": [],
    "source_project_id": 2,
    "target_project_id": 2,
    "labels": [],
    "draft": false,
    "work_in_progress": false,
    "milestone": null,
    "merge_when_pipeline_succeeds": false,
    "merge_status": "can_be_merged",
    "detailed_merge_status": "ci_still_running",
    "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
    "merge_commit_sha": null,
    "squash_commit_sha": null,
    "discussion_locked": null,
    "should_remove_source_branch": null,
    "force_remove_source_branch": null,
    "prepared_at": "2023-10-21T10:46:17.091Z",
    "reference": "!29",
    "references": {
      "short": "!29",
      "relative": "!29",
      "full": "sheather/test!29"
    },
    "web_url": "https://gitlab.lan/sheather/test/-/merge_requests/29",
    "time_stats": {
      "time_estimate": 0,
      "total_time_spent": 0,
      "human_time_estimate": null,
      "human_total_time_spent": null
    },
    "squash": false,
    "squash_on_merge": true,
    "task_completion_status": {
      "count": 0,
      "completed_count": 0
    },
    "has_conflicts": false,
    "blocking_discussions_resolved": true,
    "subscribed": true,
    "changes_count": "1",
    "latest_build_started_at": "2023-10-21T10:46:19.484Z",
    "latest_build_finished_at": null,
    "first_deployed_to_production_at": null,
    "pipeline": {
      "id": 65,
      "iid": 65,
      "project_id": 2,
      "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "ref": "refs/merge-requests/29/head",
      "status": "running",
      "source": "merge_request_event",
      "created_at": "2023-10-21T10:46:15.822Z",
      "updated_at": "2023-10-21T10:46:19.593Z",
      "web_url": "https://gitlab.lan/sheather/test/-/pipelines/65"
    },
    "head_pipeline": {
      "id": 65,
      "iid": 65,
      "project_id": 2,
      "sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "ref": "refs/merge-requests/29/head",
      "status": "running",
      "source": "merge_request_event",
      "created_at": "2023-10-21T10:46:15.822Z",
      "updated_at": "2023-10-21T10:46:19.593Z",
      "web_url": "https://gitlab.lan/sheather/test/-/pipelines/65",
      "before_sha": "0000000000000000000000000000000000000000",
      "tag": false,
      "yaml_errors": null,
      "user": {
        "id": 2,
        "username": "sheather",
        "name": "Simon Heather",
        "state": "active",
        "avatar_url": "https://secure.gravatar.com/avatar/000b3c2c6b410f2f327d6450faab4d88?s=80&d=identicon",
        "web_url": "https://gitlab.lan/sheather"
      },
      "started_at": "2023-10-21T10:46:19.484Z",
      "finished_at": null,
      "committed_at": null,
      "duration": null,
      "queued_duration": 3,
      "coverage": null,
      "detailed_status": {
        "icon": "status_running",
        "text": "running",
        "label": "running",
        "group": "running",
        "tooltip": "running",
        "has_details": true,
        "details_path": "/sheather/test/-/pipelines/65",
        "illustration": null,
        "favicon": "/assets/ci_favicons/favicon_status_running-9c635b2419a8e1ec991c993061b89cc5aefc0743bb238ecd0c381e7741a70e8c.png"
      }
    },
    "diff_refs": {
      "base_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946",
      "head_sha": "92962eec019bbbdc055fb98b376f311eafeacb3d",
      "start_sha": "e5a9e077a9306ef94da3b5eb8536357798ba9946"
    },
    "merge_error": null,
    "first_contribution": false,
    "user": {
      "can_merge": true
    }
  }

@X-Guardian X-Guardian requested a review from a team as a code owner October 21, 2023 11:17
@github-actions github-actions bot added go Pull requests that update Go code provider/gitlab labels Oct 21, 2023
@michelmzs
Copy link
Contributor

I was also considering adding a single retry with say a 1 second delay to the Get Single MR API call if the HeadPipeline property was null to try and reduce the race condition window further. Interested in any opinions.

I like the retry option, on self-hosted instances I experience even longer delays, so I suggest at least a simple retry/backoff.

@DwaineSaunderson
Copy link

DwaineSaunderson commented Oct 24, 2023

I was also considering adding a single retry with say a 1 second delay to the Get Single MR API call if the HeadPipeline property was null to try and reduce the race condition window further. Interested in any opinions.

I like the retry option, on self-hosted instances I experience even longer delays, so I suggest at least a simple retry/backoff.

Similar issue here with the delays on self-hosted instances. In my case we exclusively use (merge result) MR pipelines to do some static checks on TF and provider versions, among other things. We need the merge-result pipelines to enforce changes to these checks without forcing users to rebase their branches.

In the event that:

  • GitLab does not spawn the MR pipeline in time
  • Atlantis spawns the plan job in a branch pipeline instead
  • The static checks fail before the plan finishes
  • The plan job is successful

GitLab seems to consider the "last" pipeline to be successful and Atlantis considers the MR mergable. The user is now able to apply even if the linters in the MR pipeline failed if a reviewer doesn't catch the failing MR pipeline (by navigating to the pipelines tab, as the GitLab UI only shows the last pipeline as green).

As such, I'd like to suggest having a configuration option to never fall-through to spawning the branch-pipeline job if mr.HeadPipeline is nil, even if it means that auto-plan doesn't work consistently (or relying on a retry mechanism).

@jamengual jamengual added the waiting-on-review Waiting for a review from a maintainer label Oct 24, 2023
@jamengual jamengual added the feature New functionality/enhancement label Oct 24, 2023
@X-Guardian
Copy link
Contributor Author

I've added one retry to get the MR head pipeline details with a 2 second delay. In my tests with a locally deployed GitLab instance, it never retries (but the GitLab instance is not busy).

@X-Guardian
Copy link
Contributor Author

As such, I'd like to suggest having a configuration option to never fall-through to spawning the branch-pipeline job if mr.HeadPipeline is nil, even if it means that auto-plan doesn't work consistently (or relying on a retry mechanism).

This is outside the scope of this PR, and from my testing, if mr.HeadPipeline is empty and you set the refTarget to refs/merge-requests/<MrId>/head then the pipeline does not get associated with the MR.

@GenPage GenPage merged commit 3056701 into runatlantis:main Nov 4, 2023
13 checks passed
@GenPage
Copy link
Member

GenPage commented Nov 4, 2023

/cherry-pick release-0.26

Copy link

Cherry-pick failed with Merge error 3056701c6bf8f59fa504469bac9505d679727cff into temp-cherry-pick-8131a1-release-0.26

@jamengual
Copy link
Contributor

/cherry-pick release-0.26

GenPage pushed a commit that referenced this pull request Nov 4, 2023
…rty (#3887)

* Fix GitLab Mulitple Pipelines

* Add logger to tests and fix test ref

* Add retry to GetMergeRequest

* Update retries

---------

Co-authored-by: PePe Amengual <[email protected]>
@rgs1
Copy link

rgs1 commented Nov 21, 2023

@jamengual after this change we are seeing:

runtime error: invalid memory address or nil pointer dereference
/usr/local/go/src/runtime/panic.go:261 (0x4506f7)
/usr/local/go/src/runtime/signal_unix.go:861 (0x4506c5)
/atlantis/build-atlantis-binary/git_clone/server/events/vcs/gitlab_client.go:391 (0xdf282a)
/atlantis/build-atlantis-binary/git_clone/server/events/vcs/proxy.go:84 (0xdf8aac)
/atlantis/build-atlantis-binary/git_clone/server/events/commit_status_updater.go:135 (0x100ca77)
/atlantis/build-atlantis-binary/git_clone/server/events/commit_status_updater.go:111 (0x100c65a)
/atlantis/build-atlantis-binary/git_clone/server/events/pre_workflow_hooks_command_runner.go:146 (0x1022595)
/atlantis/build-atlantis-binary/git_clone/server/events/pre_workflow_hooks_command_runner.go:90 (0x1021bbd)
/atlantis/build-atlantis-binary/git_clone/server/events/command_runner.go:193 (0x1001596)
/usr/local/go/src/runtime/asm_amd64.s:1650 (0x46cac0)

Given that mr.HeadPipeline will always be nil here:

https://github.com/runatlantis/atlantis/blob/main/server/events/vcs/gitlab_client.go#L391

cc: @X-Guardian

@jamengual
Copy link
Contributor

there is a PR inflight to fix this I believe #3996

ijames-gc pushed a commit to gocardless/atlantis that referenced this pull request Feb 13, 2024
…rty (runatlantis#3887)

* Fix GitLab Mulitple Pipelines

* Add logger to tests and fix test ref

* Add retry to GetMergeRequest

* Update retries

---------

Co-authored-by: PePe Amengual <[email protected]>
ijames-gc pushed a commit to gocardless/atlantis that referenced this pull request Feb 13, 2024
…rty (runatlantis#3887)

* Fix GitLab Mulitple Pipelines

* Add logger to tests and fix test ref

* Add retry to GetMergeRequest

* Update retries

---------

Co-authored-by: PePe Amengual <[email protected]>
@X-Guardian X-Guardian deleted the gitlab-multiple-pipelines-fix branch April 3, 2024 14:56
@chenrui333 chenrui333 changed the title fix: Update GitLab Pipeline Type Detection to use Head Pipeline Property feat: Update GitLab Pipeline Type Detection to use Head Pipeline Property May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality/enhancement go Pull requests that update Go code provider/gitlab waiting-on-review Waiting for a review from a maintainer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants