Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend 8.3 with otel. #107

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
f5e9755
Extend 8.3 with otel.
eduard-melnytskyi Dec 9, 2024
4e55264
Merge branch 'master' into otel-instrumentation-8.3-alpine
eduard-melnytskyi Dec 9, 2024
59df613
Tmp image with Otel
alexanderM91 Dec 9, 2024
7319381
Build performance improvements.
eduard-melnytskyi Dec 13, 2024
aad550a
Build performance improvements.
eduard-melnytskyi Dec 14, 2024
c56c355
Test Otel
alexanderM91 Dec 17, 2024
ef6f0e7
Adjust actions version.
eduard-melnytskyi Dec 17, 2024
768c746
Merge branch 'otel-instrumentation-8.3-alpine' into otel-instrumentat…
eduard-melnytskyi Dec 18, 2024
7fedb0d
Add intermediate layer for extension image build.
eduard-melnytskyi Dec 18, 2024
55c71b9
added manual trigger to the new workflow
eduard-melnytskyi Dec 18, 2024
cf9a13b
added test branch for workflow
eduard-melnytskyi Dec 18, 2024
3007b14
Fix branch name definition
eduard-melnytskyi Dec 18, 2024
6f69130
Fix branch name definition
eduard-melnytskyi Dec 18, 2024
7c63e08
Adjust alpine version
eduard-melnytskyi Dec 18, 2024
ffe0c8c
Extend matrix
eduard-melnytskyi Dec 18, 2024
3775aba
Adjust matrix configuration.
eduard-melnytskyi Dec 18, 2024
3615b85
Adjust matrix configuration.
eduard-melnytskyi Dec 18, 2024
a0c0983
Increase parallel execution
eduard-melnytskyi Dec 18, 2024
d9eb3ac
Adjust php 8.3 alpine 3.20 docker file and pipeline logic.
eduard-melnytskyi Dec 19, 2024
e1a1de9
Adjust CI workflow
eduard-melnytskyi Dec 19, 2024
b3030ba
Adjust CI workflow
eduard-melnytskyi Dec 19, 2024
999d5ba
Improve condition for extension image check
eduard-melnytskyi Dec 19, 2024
d841d48
Add dependencies to CI workflow
eduard-melnytskyi Dec 19, 2024
9f56008
Change amount of parallel processes
eduard-melnytskyi Dec 19, 2024
9601885
Remove debug info
eduard-melnytskyi Dec 19, 2024
c0bffb2
Merge branch 'otel-instrumentation-8.3-alpine-build' into otel-instru…
eduard-melnytskyi Dec 19, 2024
6d7b291
Adjust extension workflow configuration.
eduard-melnytskyi Dec 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 44 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
name: CI/CD

on: push
on:
workflow_run:
workflows: ["Build GRPC-Protobuf Base Images"]
types:
- completed
push:
branches-ignore: # Exclude on master because it executed via workflow
- master

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand All @@ -10,18 +17,23 @@ jobs:
build:
runs-on: ubuntu-latest
strategy:
max-parallel: 13
fail-fast: false
matrix:
include:
## Alpine
# Alpine
### Alpine 3.18
- image: "alpine/3.18/8.1/Dockerfile"
tags: [ "spryker/php:8.1-alpine3.18" ]
platforms: [ "linux/amd64", "linux/arm64" ]
- image: "alpine/3.18/8.2/Dockerfile"
php_version: '8.2'
distro: 'alpine3.18'
tags: [ "spryker/php:8.2-alpine3.18" ]
platforms: [ "linux/amd64", "linux/arm64" ]
- image: "alpine/3.18/8.3/Dockerfile"
php_version: '8.3'
distro: 'alpine3.18'
tags: [ "spryker/php:8.3-alpine3.18" ]
platforms: [ "linux/amd64", "linux/arm64" ]

Expand All @@ -30,20 +42,28 @@ jobs:
tags: [ "spryker/php:8.1-alpine3.19" ]
platforms: [ "linux/amd64", "linux/arm64" ]
- image: "alpine/3.19/8.2/Dockerfile"
php_version: '8.2'
distro: 'alpine3.19'
tags: [ "spryker/php:8.2-alpine3.19" ]
platforms: [ "linux/amd64", "linux/arm64" ]
- image: "alpine/3.19/8.3/Dockerfile"
php_version: '8.3'
distro: 'alpine3.19'
tags: [ "spryker/php:8.3-alpine3.19" ]
platforms: [ "linux/amd64", "linux/arm64" ]

### Alpine 3.20
## Alpine 3.20
- image: "alpine/3.20/8.1/Dockerfile"
tags: [ "spryker/php:8.1", "spryker/php:8.1-alpine3.20" ]
platforms: [ "linux/amd64", "linux/arm64" ]
- image: "alpine/3.20/8.2/Dockerfile"
php_version: '8.2'
distro: 'alpine3.20'
tags: [ "spryker/php:latest", "spryker/php:8.2", "spryker/php:8.2-alpine3.20" ]
platforms: [ "linux/amd64", "linux/arm64" ]
- image: "alpine/3.20/8.3/Dockerfile"
php_version: '8.3'
distro: 'alpine3.20'
tags: [ "spryker/php:8.3", "spryker/php:8.3-alpine3.20" ]
platforms: [ "linux/amd64", "linux/arm64" ]

Expand All @@ -68,6 +88,26 @@ jobs:
with:
fetch-depth: 0

- name: Check if image exists in Docker Hub
id: check-image
run: |
if [[ "${{ matrix.distro }}" =~ alpine ]]; then
IMAGE_TAG="spryker/grpc-protobuf:${{ matrix.php_version }}-${{ matrix.distro }}"
TOKEN=$(curl -s -u "${{ secrets.DOCKER_USERNAME }}:${{ secrets.DOCKER_PASSWORD }}" \
"https://auth.docker.io/token?service=registry.docker.io&scope=repository:spryker/grpc-protobuf:pull" | jq -r .token)
EXISTS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $TOKEN" \
"https://registry-1.docker.io/v2/spryker/grpc-protobuf/manifests/${{ matrix.php_version }}-${{ matrix.distro }}")
if [ "$EXISTS" -eq 200 ]; then
echo "Image $IMAGE_TAG already exists in Docker Hub."
else
echo "Required base image $IMAGE_TAG does not exist. Failing workflow."
exit 1
fi
else
echo "Distro does not contain 'alpine'. Skipping."
fi

- name: Get the previous commit hash
id: previous_commit
run: |
Expand Down Expand Up @@ -115,7 +155,7 @@ jobs:

- name: Build and push
id: docker_build
uses: docker/build-push-action@v2
uses: docker/build-push-action@v5
with:
push: ${{ github.ref == 'refs/heads/master' }}
load: ${{ github.ref != 'refs/heads/master' }}
Expand Down
57 changes: 57 additions & 0 deletions .github/workflows/extensions-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Build GRPC-Protobuf Base Images

on: push

jobs:
build-base:
runs-on: ubuntu-latest
strategy:
max-parallel: 6
fail-fast: false
matrix:
php_version: [8.2, 8.3]
alpine_version: ['3.18', '3.19', '3.20']

steps:
- name: Checkout repository
uses: actions/checkout@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Check if image exists in Docker Hub
id: check-image
run: |
IMAGE_TAG="spryker/grpc-protobuf:${{ matrix.php_version }}-alpine${{ matrix.alpine_version }}"
TOKEN=$(curl -s -u "${{ secrets.DOCKER_USERNAME }}:${{ secrets.DOCKER_PASSWORD }}" \
"https://auth.docker.io/token?service=registry.docker.io&scope=repository:spryker/grpc-protobuf:pull" | jq -r .token)
EXISTS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $TOKEN" \
"https://registry-1.docker.io/v2/spryker/grpc-protobuf/manifests/${{ matrix.php_version }}-alpine${{ matrix.alpine_version }}")
if [ "$EXISTS" -eq 200 ]; then
echo "Image $IMAGE_TAG already exists in Docker Hub."
echo "exists=true" >> $GITHUB_ENV
else
echo "Image $IMAGE_TAG does not exist. Proceeding with build."
echo "exists=false" >> $GITHUB_ENV
fi

- name: Build and push base image
id: docker_build
if: env.exists == 'false'
uses: docker/build-push-action@v5
with:
push: ${{ github.ref == 'refs/heads/master' }}
tags: spryker/grpc-protobuf:${{ matrix.php_version }}-alpine${{ matrix.alpine_version }}
file: alpine/extension/Dockerfile
platforms: linux/amd64,linux/arm64
cache-to: type=gha,mode=max
build-args: |
MAJOR_PHP_VERSION=${{ matrix.php_version }}
ALPINE_VERSION=${{ matrix.alpine_version }}
29 changes: 27 additions & 2 deletions alpine/3.20/8.3/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# syntax = docker/dockerfile:1.0.2-experimental
ARG SPRYKER_PHP_VERSION=8.3.13
ARG EXTENSION_IMAGE=spryker/grpc-protobuf:8.3-alpine3.20

FROM ${EXTENSION_IMAGE} AS extension

FROM php:${SPRYKER_PHP_VERSION}-fpm-alpine3.20

ARG TARGETPLATFORM
Expand Down Expand Up @@ -65,6 +69,7 @@ ARG PHP_EXTENSIONS="\

ARG PHP_PECL_EXTENSIONS="\
apcu \
opentelemetry \
redis"

ARG ADDITIONAL_PHP_PECL_EXTENSIONS="\
Expand Down Expand Up @@ -110,7 +115,7 @@ RUN apk update \
--with-webp=/usr \
--with-xpm=/usr \
&& docker-php-ext-install -j5 ${PHP_EXTENSIONS} \
&& pecl install -o -f ${PHP_PECL_EXTENSIONS} ${ADDITIONAL_PHP_PECL_EXTENSIONS} \
&& MAKEFLAGS="-j$(nproc)" pecl install -o -f ${PHP_PECL_EXTENSIONS} ${ADDITIONAL_PHP_PECL_EXTENSIONS} \
&& rm -rf /tmp/pear \
&& docker-php-ext-enable ${PHP_EXTENSIONS} ${PHP_PECL_EXTENSIONS} \
&& apk del --no-cache .php-build-deps
Expand Down Expand Up @@ -184,6 +189,26 @@ COPY context/php/conf.d/90-opcache.ini /usr/local/etc/php/conf.d/
COPY context/php/conf.d/92-session.ini /usr/local/etc/php/conf.d/
COPY context/php/disabled /usr/local/etc/php/disabled

#OTEL: disable code instrumentattion by default
ENV OTEL_SDK_DISABLED=true
ENV OTEL_PHP_DISABLED_INSTRUMENTATIONS="all"

# Copy extensions from extension image to tpm path
COPY --from=extension /usr/local/lib/php/extensions/no-debug-non-zts-20230831/grpc.so /tmp/php-ext-dir-path/grpc.so
COPY --from=extension /usr/local/lib/php/extensions/no-debug-non-zts-20230831/protobuf.so /tmp/php-ext-dir-path/protobuf.so

#Calculate tarrget php extension folder and copy required extensions
RUN PHP_EXT_DIR=$(php-config --extension-dir) \
&& echo "Using PHP extension directory: $PHP_EXT_DIR" \
&& cp /tmp/php-ext-dir-path/grpc.so $PHP_EXT_DIR/grpc.so \
&& cp /tmp/php-ext-dir-path/protobuf.so $PHP_EXT_DIR/protobuf.so


#OTEL: Disable OTEL related extensions and keep original configs.
COPY --from=extension /usr/local/etc/php/conf.d/docker-php-ext-protobuf.ini /usr/local/etc/php/disabled/protobuf.ini
COPY --from=extension /usr/local/etc/php/conf.d/docker-php-ext-grpc.ini /usr/local/etc/php/disabled/grpc.ini
RUN mv /usr/local/etc/php/conf.d/docker-php-ext-opentelemetry.ini /usr/local/etc/php/disabled/opentelemetry.ini

WORKDIR /data

# Install composer
Expand All @@ -199,4 +224,4 @@ ENV COMPOSER_MEMORY_LIMIT=-1
RUN mkdir -p /home/spryker/.composer
RUN bash -c '[[ $COMPOSER_VERSION == "1"* ]] && composer global require hirak/prestissimo && rm -rf /home/spryker/.composer/cache || true'

USER root
USER root
84 changes: 84 additions & 0 deletions alpine/extension/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# GRPC + Protobuf extension
ARG MAJOR_PHP_VERSION=8.3
ARG ALPINE_VERSION=3.20
FROM php:${MAJOR_PHP_VERSION}-fpm-alpine${ALPINE_VERSION}

# Set environment variables
ENV srcRoot /data
ENV LD_PRELOAD /usr/lib/preloadable_libiconv.so

ARG PHP_RUN_DEPS="\
freetype \
gmp \
gnu-libiconv \
icu-libs \
libbz2 \
libc6-compat \
libjpeg-turbo \
libpng \
libwebp \
libxml2 \
libxpm \
libxslt \
libzip \
protobuf \
grpc"

ARG PHP_BUILD_DEPS="\
autoconf \
bzip2-dev \
freetype-dev \
gmp-dev \
icu-dev \
icu-data-full \
libjpeg-turbo-dev \
libpng-dev \
libwebp-dev \
libxml2-dev \
libxpm-dev \
libzip-dev \
postgresql-dev \
rabbitmq-c-dev \
protobuf-dev \
grpc-dev \
g++"

ARG PHP_PECL_EXTENSIONS="\
opentelemetry \
protobuf \
grpc"

ARG ADDITIONAL_PHP_PECL_EXTENSIONS=""

RUN apk add --no-cache \
bash \
coreutils \
curl \
git \
make \
mysql-client \
netcat-openbsd \
openssh \
postgresql-client \
procps \
python3 \
shadow \
unzip \
linux-headers \
libstdc++ \
${PHP_RUN_DEPS} \
&& apk add --no-cache --virtual .php-build-deps ${PHP_BUILD_DEPS} \
&& MAKEFLAGS="-j$(nproc)" pecl install -o -f ${PHP_PECL_EXTENSIONS} ${ADDITIONAL_PHP_PECL_EXTENSIONS} \
&& docker-php-ext-enable ${PHP_PECL_EXTENSIONS} \
&& rm -rf /tmp/pear \
&& apk del --no-cache .php-build-deps

RUN mkdir -p ${srcRoot}

# Set environment variables
ENV srcRoot /data
ENV LD_PRELOAD /usr/lib/preloadable_libiconv.so

WORKDIR ${srcRoot}

USER root
Loading