Hi Jeff - if I'm understanding your description correctly, I believe that the build variables for stacks are already optional. If you define the variable under the build variables tab (e.g. https://portal.cycle.io/stacks/<<stack-id>>/variables) then the build will default to the configured value unless you override that value when you trigger the build. Is that what you were trying to accomplish?

The issues is the pipelines, if you define a pipeline variable, you NEED to declare it. Whereas, sometimes we want to send the variable to the pipeline, and sometimes we don't. If you manage with deployments, you need pipelines. And the optional logic of stacks doesn't equally apply to the pipelines.

I believe a suggestion was made to have a pipeline for each optional variable, but that's an N factorial problem. Is this one a bit clearer now Casey (holler if it's not)

Yeah - I know Jake was looking into something similar for us. I have a deployment pipeline for handling rainbow deployments that I use for a number of different stacks - but the stacks all take different build variables so there was no practical way to have the pipeline trigger the stack build without creating a separate pipeline for every stack individually (which I did not want to do).

For now (unless and until Jake figures out a way to simplify) I have my CI/CD process push the stack build via the API (so that each project could specify the appropriate variables for its build) and then using a single pipeline that takes the build id from the API call and triggers off the rest of the release process. Deployment script looks like this:

#!/bin/bash

HEADERS=(
  -H "X-Hub-Id: $CYCLE_HUB_ID" \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $CYCLE_API_TOKEN" \
)

# Create the new build and capture the response
response=$(curl --location "https://api.cycle.io/v1/stacks/$CYCLE_STACK_ID/builds" \
  -s "${HEADERS[@]}" \
  -d '{
  "about": {
    "version": "'"$SERVICE_NAME-$IMAGE_TAG"'",
    "description": "'"$SERVICE_NAME deployment of image $IMAGE_TAG"'"
  },
  "instructions": {
    "git": {
      "type": "branch",
      "value": "'"$BITBUCKET_BRANCH"'"
    },
    "variables": {
        "source": "'"$CYCLE_SOURCE_ID"'",
        "target": "busify-'"$SERVICE_NAME:$IMAGE_TAG"'",
        "db-user": "'"$DB_USER"'",
        "db-pass": "'"$DB_PASS"'",
        "new-relic-license-key": "'"$NEW_RELIC_LICENSE_KEY"'",
        "amqp-user": "'"$AMQP_USER"'",
        "amqp-pass": "'"$AMQP_PASS"'"
   }
  }
}')

# Extract the build ID from the response using jq
build_id=$(echo "$response" | jq -r '.data.id')

# Check if build_id is empty or null
if [ -z "$build_id" ] || [ "$build_id" == "null" ]; then
    echo "Failed to extract build ID from the response"
    exit 1
fi

echo "Extracted build ID: $build_id"

# Trigger the pipeline with the extracted build ID
curl -v -k "https://api.cycle.io/v1/pipelines/$CYCLE_RAINBOW_PIPELINE_ID/tasks" \
  -s "${HEADERS[@]}" \
  -d '{
  "action": "trigger",
  "contents": {
    "variables": {
      "build": "'"$build_id"'",
      "environment": "'"$CYCLE_ENVIRONMENT_ID"'",
      "service": "'"$SERVICE_NAME"'",
      "tag": "'"$IMAGE_TAG"'"
    }
  }
}'

LoL this is pretty much what I'm envisioning is going to need to happen; I agree it's a workaround, but stinks when you have a build with different inbound variables and no way to debug them. When we want repeatabilit, the last thing we want is for dev to find out which dev builder goes with which customer deployment. If you're runnign 2 custom vars like

ADVANCED_DEBUG=on DEBUG_OBJECTS=['x','y']

then the next person has to know that, versus just having them blank and adding them to a specific pipelines deployment, and storing that deployment. I'd rather the variability on pipeline, and simple primitives on build. I should only need to modify builds when introducing new capabilities, not based on my pipeline variance (IMHO)

I think what Jake was looking into was a way to pass arbitrary variables to the pipeline and then have the pipeline just forward all of the supplied variables to the build (vs having to explicitly define the mapping of variables from pipeline to build). I'd still need to specify the particular variables that a specific app needs when invoking the pipeline, but it would eliminate the need for the API call such that my CI/CD process would only need to trigger the pipeline.

For me, when I want to repeat a build BitBucket has an option to re-trigger the deployment step of any CI/CD pipeline that has previously run, so I just need to hit that button and it pushes the build again.

We've introduced a new type called 'OptionalString' that allows variables to not be required in a few places where defaults would otherwise provide the value:

Within pipelines:

• Stack Build Variables
• Image Build Args
• Image Target Overrides

It's important to note that the variable must not be sent -- sending a variable with an empty string (or zero value) will still be treated as a populated variable. Also, this only works if the value for a field is: {{variable-name}} -- if you were to use my-{{variable-name}} the my- would prevent the default value from being utilized.

Very solid! Deployed or to be deployed?

Pending deployment (next week)