09CI/CD и DevOps для .NET
Docker и Containerization
Содержание
- Основы контейнеризации
- Multi-stage builds — минимизация размера образа
- Distroless vs Alpine vs Debian base images
- Layer caching optimization
- Health checks — liveness, readiness, startup probes
- Non-root user best practices
- docker-compose для local development
- Практические задания
Основы контейнеризации
Что такое контейнер и зачем он нужен
Контейнер — это изолированная среда выполнения с собственным файловым пространством, сетью и ограничениями ресурсов. В отличие от виртуальной машины, контейнер делится с хостом ядром ОС, что делает его лёгким и быстрым.
Контейнер vs Виртуальная машина
| Критерий | VM | Контейнер |
|---|---|---|
| Ядро | Полное (гостевое ОС) | Общее с хостом |
| Размер | GB | MB |
| Запуск | Минуты | Миллисекунды |
| Изоляция | Аппаратная | Process namespace |
| Overhead | Высокий | Минимальный |
| Образ | 5–50 GB | 50–300 MB |
Архитектура Docker
┌─────────────────────────────────────────┐
│ Docker Client │
├─────────────────────────────────────────┤
│ Docker Daemon (dockerd) │
│ ┌──────────┐ ┌──────────┐ ┌────────┐ │
│ │ Images │ │ Containers│ │Networks│ │
│ └──────────┘ └──────────┘ └────────┘ │
│ ┌──────────┐ ┌──────────┐ │
│ │ Volumes │ │ Plugins │ │
│ └──────────┘ └──────────┘ │
├─────────────────────────────────────────┤
│ Host OS Kernel │
├─────────────────────────────────────────┤
│ Hardware │
└─────────────────────────────────────────┘Multi-stage builds — минимизация размера образа
Проблема: большой образ
Первый подход — собрать и запустить в одном образе:
# Плохо: образ ~2.1 GB — содержит SDK, исходники, всё в одном слое
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /app
COPY *.sln .
COPY **/*.csproj ./
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o out
FROM mcr.microsoft.com/dotnet/aspnet:9.0
WORKDIR /app
COPY --from=build /app/out .
ENTRYPOINT ["dotnet", "MyApp.dll"]Проблема: Образ aspnet:9.0 уже содержит runtime, но если мы используем sdk:9.0 в финальном этапе — образ будет содержать SDK + runtime + исходники + всё промежуточное.
Решение: Multi-stage build
# Этап 1: Сборка
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /src
# Копируем только csproj для оптимизации кэширования слоёв
COPY *.sln .
COPY **/*.csproj ./
# Restore с кэшированием — слой не пересобирается, если csproj не изменился
RUN dotnet restore
# Копируем исходный код
COPY . .
# Сборка и публикация
RUN dotnet publish -c Release -o /app/publish --no-restore
# Этап 2: Runtime — только нужное
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS runtime
WORKDIR /app
# Копируем ТОЛЬКО опубликованный бинарник
COPY --from=build /app/publish .
# Non-root пользователь (безопасность)
RUN groupadd -r appgroup && useradd -r -g appgroup -d /app -s /sbin/nologin appuser
USER appuser
EXPOSE 8080
ENV ASPNETCORE_URLS=http://+:8080
ENV ASPNETCORE_ENVIRONMENT=Production
ENV DOTNET_SYSTEM_NET_HTTP_SOCKETSHTTPHANDLER_HTTP2UNENCRYPTEDSUPPORT=true
ENTRYPOINT ["dotnet", "MyApp.dll"]Сравнение размеров
| Подход | Размер образа | Слои |
|---|---|---|
| SDK only (один этап) | ~2.1 GB | 1 |
| Runtime + SDK (2 этапа без оптимизации) | ~500 MB | 2 |
| Multi-stage + aspnet | ~170 MB | 3+ |
| Multi-stage + distroless | ~80 MB | 2 |
Продвинутый multi-stage: сборка с кэшированием NuGet
# ---------- Build ----------
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /src
# Копируем только зависимости — восстанавливаем и кэшируем
COPY *.sln .
COPY **/*.csproj ./
RUN dotnet restore
# Копируем весь проект
COPY . .
# Тесты
RUN dotnet test --no-restore --verbosity minimal
# Публикация
RUN dotnet publish -c Release -o /app/publish --no-restore --no-build
# ---------- Test ----------
FROM build AS test
COPY . .
RUN dotnet test -c Release
# ---------- Publish ----------
FROM build AS publish
RUN dotnet publish -c Release -o /app/publish --no-restore --no-build
# ---------- Runtime ----------
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS runtime
WORKDIR /app
COPY --from=publish /app/publish .
# Настройка для production
ENV ASPNETCORE_ENVIRONMENT=Production
ENV ASPNETCORE_FORWARDEDHEADERS_ENABLED=true
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8080/health || exit 1
EXPOSE 8080
ENTRYPOINT ["dotnet", "MyApp.dll"]Distroless vs Alpine vs Debian base images
Сравнение базовых образов
| Образ | Размер | Содержимое | Безопасность | Производительность |
|---|---|---|---|---|
mcr.microsoft.com/dotnet/aspnet:9.0 | ~170 MB | Debian + runtime | Средняя | Максимальная |
mcr.microsoft.com/dotnet/aspnet:9.0-alpine | ~80 MB | Alpine Linux + runtime | Высокая | ~95% (musl vs glibc) |
mcr.microsoft.com/dotnet/aspnet:9.0-chiseled | ~110 MB | Debian chiseled (без лишних пакетов) | Очень высокая | Максимальная |
gcr.io/distroless/cc-debian12 | ~30 MB | Ничего кроме библиотек | Максимальная | Максимальная |
mcr.microsoft.com/dotnet/runtime-deps:9.0 | ~30 MB | Только зависимости .NET | Очень высокая | Максимальная |
Distroless — минимальный образ без ОС
# Distroless — только runtime + бинарник, без shell, без package manager
FROM mcr.microsoft.com/dotnet/runtime-deps:9.0 AS base
WORKDIR /app
# Копируем только published output
COPY --from=build /app/publish .
# Distroless не имеет curl — health check нужно реализовать иначе
USER appuser
ENTRYPOINT ["dotnet", "MyApp.dll"]Плюсы:
- Минимальная поверхность атаки (нет shell, package manager, utilities)
- Размер ~80–100 MB с .NET runtime
- Меньше уязвимостей (нет SSH, bash, etc.)
Минусы:
- Нет shell для отладки (невозможно
docker exec) - Нет curl — health check нужно реализовать иначе
- Нужно вручную настраивать пользователя
Alpine — компромисс
FROM mcr.microsoft.com/dotnet/aspnet:9.0-alpine AS runtime
WORKDIR /app
COPY --from=build /app/publish .
# Alpine использует musl libc вместо glibc — возможная несовместимость
# Некоторые native libraries (например, libicu) могут требовать дополнительных пакетов
RUN apk add --no-cache icu-libs
USER root
ENTRYPOINT ["dotnet", "MyApp.dll"]Плюсы:
- Маленький размер (~80 MB)
- Есть apk (package manager) для установки зависимостей
- Есть shell для отладки
Минусы:
- musl libc вместо glibc — несовместимость с некоторыми native libraries
- gRPC с HTTP/2 может не работать с mTLS
- Некоторые native dependencies требуют перекомпиляции
Chiseled Debian — рекомендация Microsoft
# Chiseled images — Debian-based, но без лишних компонентов
FROM mcr.microsoft.com/dotnet/aspnet:9.0-chiseled AS runtime
WORKDIR /app
COPY --from=build /app/publish .
# Chiseled images имеют встроенную поддержку health check
# и настроены для production использования
USER appuser
ENTRYPOINT ["dotnet", "MyApp.dll"]Выбор образа
| Сценарий | Рекомендуемый образ |
|---|---|
| Production, максимальная безопасность | runtime-deps или distroless |
| Production, баланс безопасность/удобство | aspnet:9.0-chiseled |
| Development, нужна отладка | aspnet:9.0 |
| Микросервисы с жёсткими лимитами памяти | runtime-deps + minimal hosting model |
| Нужны native libraries | aspnet:9.0 (glibc) |
| Minimal footprint, нет отладки | distroless |
Layer caching optimization — порядок имеет значение
Как работает Docker layer caching
Каждый RUN, COPY, ADD создаёт новый слой. Docker кэширует каждый слой. Если файл не изменился — используется кэш.
Правило 1: Сначала зависимости, потом код
# Плохо: каждый COPY . . инвалидирует кэш NuGet
COPY . .
RUN dotnet restore
COPY . .
RUN dotnet publish
# Хорошо: restore выполняется только при изменении csproj
COPY **/*.csproj ./
RUN dotnet restore
COPY . .
RUN dotnet publishПравило 2: Порядок COPY — от стабильного к изменчивому
# Слои (сверху = самые базовые, снизу = самые новые):
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
# Слой 1: .sln + csproj (редко меняются)
COPY *.sln .
COPY **/*.csproj ./
RUN dotnet restore # Слой 2: кэш NuGet пакетов
# Слой 3: весь остальной код (часто меняется)
COPY . .
# Слой 4: publish (инвалидируется при любом изменении кода)
RUN dotnet publish -c Release -o outПравило 3: Объединяй команды RUN
# Плохо: 3 слоя для установки пакетов
RUN apk add --no-cache curl
RUN apk add --no-cache curl-dev
RUN apk add --no-cache libicu
# Хорошо: 1 слой
RUN apk add --no-cache curl curl-dev libicuПравило 4: .dockerignore
# Исключаем ненужное из контекста сборки
**/bin/
**/obj/
*.suo
*.user
.git/
.github/
docs/
tests/
**/*.test.csproj
**/*.spec.csprojПравило 5: Многоэтапная сборка с кэшированием NuGet
# Кэширование NuGet через volume mount
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
# NuGet cache persist между сборками (Docker BuildKit)
RUN --mount=type=cache,target=/nuget/pkgdir \
dotnet nuget locals all --clear
WORKDIR /src
COPY *.sln .
COPY **/*.csproj ./
RUN dotnet restore --packages /nuget/pkgdir
COPY . .
RUN dotnet publish -c Release -o /app/publish --no-restoreПравило 6: Использование BuildKit features
# syntax=docker/dockerfile:1
# Mount secrets (безопасная передача credentials)
RUN --mount=type=secret,id=nuget_key \
dotnet nuget add source --name myget --username user --password $(cat /run/secrets/nuget_key) https://myget.org/F/my-api-key
# Mount cache для разных пакетных менеджеров
RUN --mount=type=cache,target=/root/.nuget/packages \
--mount=type=cache,target=/root/.npm \
dotnet restore && npm installHealth checks — liveness, readiness, startup probes
Типы health probes в Kubernetes
| Probe | Когда запускается | Что проверяет | Поведение при fail |
|---|---|---|---|
| Liveness | После старта | "Жив ли процесс?" | Перезапуск контейнера |
| Readiness | После старта | "Готов принимать трафик?" | Удаление из Service endpoints |
| Startup | При запуске | "Успешно ли стартовал?" | Предотвращает liveness probe |
Health check в Docker
# Docker HEALTHCHECK
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
CMD curl -f http://localhost:8080/health || exit 1Health check в ASP.NET Core
// Program.cs
var builder = WebApplication.CreateBuilder(args);
// Добавляем endpoint для health checks
builder.Services.AddHealthChecks()
.AddCheck("self", () => HealthCheckResult.Healthy())
.AddDbContextCheck<AppDbContext>("database")
.AddRedis(builder.Configuration["RedisConnection"], "redis");
var app = builder.Build();
// Map health checks
app.MapHealthChecks("/health"); // Все проверки
app.MapHealthChecks("/health/ready", // Readiness (всё готово)
new HealthCheckOptions { Predicate = check => check.Tags.Contains("ready") });
app.MapHealthChecks("/health/live", // Liveness (процесс жив)
new HealthCheckOptions { Predicate = check => check.Tags.Contains("live") });
app.Run();Custom Health Check
public class DatabaseHealthCheck : IHealthCheck
{
private readonly IDbConnection _connection;
public DatabaseHealthCheck(IDbConnection connection)
{
_connection = connection;
}
public async Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context,
CancellationToken cancellationToken = default)
{
try
{
await _connection.OpenAsync(cancellationToken);
if (_connection.State == ConnectionState.Open)
{
using var cmd = _connection.CreateCommand();
cmd.CommandText = "SELECT 1";
cmd.CommandTimeout = 5;
await cmd.ExecuteScalarAsync(cancellationToken);
return HealthCheckResult.Healthy("Database is responsive");
}
return HealthCheckResult.Unhealthy("Database connection is not open");
}
catch (Exception ex)
{
return HealthCheckResult.Unhealthy("Database check failed", ex);
}
}
}Startup Probe для медленных приложений
// Для приложений с долгим startup (загрузка моделей ML, большая инициализация)
// Startup probe даёт приложению время на запуск
// В Kubernetes:
// startupProbe:
// httpGet:
// path: /health/live
// port: 8080
// initialDelaySeconds: 0
// periodSeconds: 10
// failureThreshold: 30 // 30 * 10 = 300 секунд максимум на старт
// В .NET — можно использовать IHostedService для сигнализации готовности
public class StartupReadyService : IHostedService
{
private readonly IHostApplicationLifetime _appLifetime;
private readonly ILogger<StartupReadyService> _logger;
public StartupReadyService(
IHostApplicationLifetime appLifetime,
ILogger<StartupReadyService> logger)
{
_appLifetime = appLifetime;
_logger = logger;
}
public async Task StartAsync(CancellationToken ct)
{
_logger.LogInformation("Application starting up...");
// Долгая инициализация
await LoadMachineLearningModel(ct);
await WarmUpConnections(ct);
await PreloadCache(ct);
_logger.LogInformation("Application ready to accept traffic");
}
public Task StopAsync(CancellationToken ct) => Task.CompletedTask;
private Task LoadMachineLearningModel(CancellationToken ct) => Task.CompletedTask;
private Task WarmUpConnections(CancellationToken ct) => Task.CompletedTask;
private Task PreloadCache(CancellationToken ct) => Task.CompletedTask;
}Non-root user best practices
Почему нельзя запускать от root
# Плохо: контейнер работает от root
FROM mcr.microsoft.com/dotnet/aspnet:9.0
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "MyApp.dll"]Риски:
- Уязвимость в приложении = полный доступ к контейнеру
- Возможность escape из контейнера (в некоторых конфигурациях)
- Нарушение принципа наименьших привилегий
- Compliance issues (PCI DSS, SOC 2)
Создание non-root пользователя
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS runtime
WORKDIR /app
COPY --from=build /app/publish .
# Создаём группу и пользователя
RUN groupadd -r appgroup && \
useradd -r -g appgroup -d /app -s /sbin/nologin -G appgroup appuser && \
chown -R appuser:appgroup /app
# Переключаемся на non-root пользователя
USER appuser
EXPOSE 8080
ENV ASPNETCORE_URLS=http://+:8080
ENTRYPOINT ["dotnet", "MyApp.dll"]Правильные права на файлы
# Копируем файлы с правильными правами
COPY --from=build --chown=appuser:appgroup /app/publish .
# Или задаём права после копирования
COPY --from=build /app/publish .
RUN chown -R appuser:appgroup /app && \
chmod -R 555 /app # read + execute, no write.NET Worker Service с non-root
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /src
COPY **/*.csproj ./
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o /app/publish
FROM mcr.microsoft.com/dotnet/runtime:9.0 AS runtime
WORKDIR /app
RUN groupadd -r appgroup && useradd -r -g appgroup appuser && \
chown -R appuser:appgroup /app
COPY --from=build --chown=appuser:appgroup /app/publish .
# Worker services часто пишут логи — нужно разрешить запись
RUN mkdir -p /app/logs && chown -R appuser:appgroup /app/logs
USER appuser
ENTRYPOINT ["dotnet", "WorkerService.dll"]docker-compose для local development
Базовый docker-compose
# docker-compose.yml
version: '3.9'
services:
api:
build:
context: .
dockerfile: Dockerfile
ports:
- "5000:8080"
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ConnectionStrings__DefaultConnection=Host=postgres;Database=myapp;Username=postgres;Password=postgres
- ConnectionStrings__Redis=redis:6379
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
volumes:
- ./src:/app/src # Hot reload для разработки
restart: unless-stopped
networks:
- app-network
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: myapp
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- "5432:5432"
volumes:
- postgres-data:/var/lib/postgresql/data
- ./init-db.sql:/docker-entrypoint-initdb.d/init.sql
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 3s
retries: 5
networks:
- app-network
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
- redis-data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 5
networks:
- app-network
sqlserver:
image: mcr.microsoft.com/mssql/server:2022-latest
environment:
ACCEPT_EULA: Y
SA_PASSWORD: 'YourStrong@Passw0rd'
MSSQL_PID: Express
ports:
- "1433:1433"
volumes:
- sqlserver-data:/var/opt/mssql
healthcheck:
test: ["CMD-SHELL", "/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'YourStrong@Passw0rd' -Q 'SELECT 1'"]
interval: 10s
timeout: 5s
retries: 5
networks:
- app-network
volumes:
postgres-data:
redis-data:
sqlserver-data:
networks:
app-network:
driver: bridgedocker-compose с Visual Studio Hot Reload
# docker-compose.dev.yml
services:
api:
build:
context: .
dockerfile: Dockerfile.dev # Отдельный Dockerfile для dev
ports:
- "5000:8080"
- "5001:8081"
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_HTTP_PORTS=8080
- ASPNETCORE_HTTPS_PORTS=8081
volumes:
# Mount source для hot reload
- ./src:/app/src
- ./obj:/app/obj # obj для hot reload
- ${APPDATA}/Microsoft/UserSecrets:/home/appuser/.microsoft/usersecrets:ro
- ${APPDATA}/ASP.NET/Https:/home/appuser/.aspnet/https:ro
deploy:
resources:
limits:
memory: 2G
cpus: '1.0'Dockerfile для разработки
# Dockerfile.dev
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS base
WORKDIR /app
EXPOSE 8080
EXPOSE 8081
ENV ASPNETCORE_ENVIRONMENT=Development
ENV ASPNETCORE_URLS=http://+:8080
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /src
COPY *.sln .
COPY **/*.csproj ./
RUN dotnet restore
COPY . .
RUN dotnet publish -c Debug -o /app/publish
FROM base AS runtime
WORKDIR /app
COPY --from=build /app/publish .
# Non-root пользователь для dev
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
USER appuser
ENTRYPOINT ["dotnet", "MyApp.dll"]Практика
Задание 1: Optimized Dockerfile для ASP.NET Core API
Цель: Создать Dockerfile с multi-stage build, размер финального образа < 200MB
Требования:
- Multi-stage build (build -> publish -> runtime)
- Layer caching optimization (csproj до кода)
- Non-root пользователь
- Health check
- .dockerignore
- Финальный образ < 200 MB
Результат:
docker build -t myapp:optimized .
docker images myapp:optimized
# Размер должен быть < 200MBЗадание 2: docker-compose для local development
Цель: Создать полный стек для локальной разработки
Требования:
- ASP.NET Core API
- PostgreSQL (с health check)
- Redis (с health check)
- Все сервисы в одной сети
- Volumes для persistence данных
- Environment variables для конфигурации
Результат:
docker-compose up -d
docker-compose ps
# Все сервисы должны быть healthyЗадание 3: Health check endpoint
Цель: Реализовать health checks в ASP.NET Core
Требования:
/health— все проверки/health/ready— readiness (БД, Redis)/health/live— liveness (процесс жив)- Custom health check для БД
- Custom health check для Redis
Результат:
curl http://localhost:5000/health
curl http://localhost:5000/health/ready
curl http://localhost:5000/health/liveСводная таблица: Best Practices
| Практика | Рекомендация |
|---|---|
| Базовый образ | aspnet:9.0-chiseled для production |
| Multi-stage | Всегда использовать |
| Layer caching | COPY *.csproj до COPY . |
| Пользователь | Non-root (appuser) |
| Health checks | Liveness + Readiness + Startup |
| .dockerignore | Исключить bin/, obj/, tests/ |
| .NET Environment | ASPNETCORE_ENVIRONMENT=Production |
| Forwarded headers | ASPNETCORE_FORWARDEDHEADERS_ENABLED=true |
| Размер образа | < 200 MB для API |
| docker-compose | Health checks для всех сервисов |
CI Pipeline
Содержание
- GitHub Actions vs GitLab CI vs Azure DevOps
- Build pipeline stages: restore → build → test → publish
- Caching strategies — NuGet packages, Docker layers
- Matrix builds — multi-target framework, multi-platform
- Artifact management — versioned outputs, retention policies
- Quality gates — code coverage, security scanning
- Практические задания
GitHub Actions vs GitLab CI vs Azure DevOps
Сравнение CI/CD платформ
| Критерий | GitHub Actions | GitLab CI/CD | Azure DevOps |
|---|---|---|---|
| Интеграция с репозиторием | Нативная (GitHub) | Нативная (GitLab) | Отдельный продукт |
| Free minutes (public) | 2000 мин/мес | 400 мин/мес | Нет |
| Free minutes (private) | 500 мин/мес | 400 мин/мес | 1800 мин/мес |
| Self-hosted runners | Да | Да | Да (Agent) |
| Secret management | Actions secrets | CI/CD variables | Variable groups |
| Environments | Native support | Native support | Native support |
| Artifact storage | 10 GB/мес | 1 GB (Free) | 5 GB (Free) |
| Parallel jobs | До 20 (Free) | До 400 | Зависит от лицензии |
| YAML syntax | workflow | .gitlab-ci.yml | YAML / Classic editor |
| .NET focus | Отличная | Хорошая | Отличная |
Когда что выбирать
| Сценарий | Рекомендация |
|---|---|
| Код на GitHub | GitHub Actions |
| Код на GitLab | GitLab CI/CD |
| Enterprise + Azure | Azure DevOps |
| On-premise + GitLab | GitLab Runner |
| Multi-cloud | GitHub Actions или GitLab CI |
Build pipeline stages
Стандартный pipeline: restore → build → test → publish
# .github/workflows/dotnet-ci.yml
name: .NET CI
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
env:
DOTNET_VERSION: '9.0.x'
CONFIGURATION: Release
jobs:
# Stage 1: Restore
restore:
name: Restore dependencies
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Restore NuGet packages
run: dotnet restore --ignore-failed-sources
working-directory: ./src
- name: Cache NuGet packages
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/*.csproj') }}
restore-keys: |
${{ runner.os }}-nuget-
# Stage 2: Build
build:
name: Build application
needs: restore
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Restore
run: dotnet restore
working-directory: ./src
- name: Build
run: dotnet build --no-restore --configuration ${{ env.CONFIGURATION }}
working-directory: ./src
- name: Pack
run: dotnet pack --no-build --configuration ${{ env.CONFIGURATION }} -o ./nupkg
working-directory: ./src
# Stage 3: Test
test:
name: Run tests
needs: build
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_DB: testdb
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis:7-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Restore
run: dotnet restore
working-directory: ./src
- name: Test
run: dotnet test --no-restore --configuration ${{ env.CONFIGURATION }} --verbosity normal
working-directory: ./src
- name: Upload test results
uses: actions/upload-artifact@v4
if: always()
with:
name: test-results
path: '**/TestResults/*.trx'
retention-days: 30
# Stage 4: Publish
publish:
name: Publish artifacts
needs: test
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Publish
run: dotnet publish -c Release -o ./publish
working-directory: ./src
- name: Upload publish artifacts
uses: actions/upload-artifact@v4
with:
name: publish-output
path: ./publish
retention-days: 5Caching strategies
NuGet caching
# Базовое кэширование NuGet
- name: Cache NuGet packages
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/*.csproj') }}
restore-keys: |
${{ runner.os }}-nuget-
# Продвинутое кэширование с restore
- name: Cache and restore NuGet packages
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/Directory.Packages.props', '**/*.csproj') }}
- name: Restore dependencies (uses cache)
run: dotnet restore --packages .nuget/packagesDocker layer caching
# GitHub Actions — Docker layer cache
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
push: ${{ github.event_name == 'push' }}
tags: myapp:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
# Или с registry cache
- name: Build and push
uses: docker/build-push-action@v5
with:
push: true
tags: myapp:${{ github.sha }}
cache-from: type=registry,ref=myapp:build-cache
cache-to: type=registry,ref=myapp:build-cache,mode=maxMSBuild caching (BuildKit)
# MSBuild cache через BuildKit
- name: Build with MSBuild cache
run: |
dotnet restore --packages .nuget/packages
dotnet build --no-restore -c Release \
-p:UseSharedCompilation=false \
-p:ContinuousBuildOutputDirectoryPath=./msbuild-cacheСравнение стратегий кэширования
| Стратегия | Скорость | Надёжность | Сложность |
|---|---|---|---|
actions/cache (NuGet) | Хорошая | Средняя | Низкая |
actions/cache + --packages | Лучшая | Высокая | Низкая |
| Docker layer cache (GH Actions) | Отличная | Высокая | Низкая |
| Registry cache (Docker) | Отличная | Высокая | Средняя |
| BuildKit mount=type=cache | Лучшая | Высокая | Средняя |
Matrix builds
Multi-target framework
name: .NET CI — Matrix Build
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
dotnet-version: ['8.0.x', '9.0.x']
configuration: [Release, Debug]
include:
- dotnet-version: '9.0.x'
configuration: Release
run-tests: true
- dotnet-version: '8.0.x'
configuration: Release
run-tests: true
steps:
- uses: actions/checkout@v4
- name: Setup .NET ${{ matrix.dotnet-version }}
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ matrix.dotnet-version }}
- name: Restore
run: dotnet restore
- name: Build
run: dotnet build --no-restore -c ${{ matrix.configuration }}
- name: Test
if: matrix.run-tests
run: dotnet test --no-build -c ${{ matrix.configuration }}Multi-platform build
name: .NET CI — Multi-Platform
on: [push]
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
configuration: [Release]
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- name: Restore
run: dotnet restore
- name: Build
run: dotnet build -c ${{ matrix.configuration }}
- name: Test
run: dotnet test -c ${{ matrix.configuration }}
- name: Publish Linux
if: matrix.os == 'ubuntu-latest'
run: dotnet publish -c ${{ matrix.configuration }} -r linux-x64 -o publish/linux-x64 --self-contained
- name: Publish Windows
if: matrix.os == 'windows-latest'
run: dotnet publish -c ${{ matrix.configuration }} -r win-x64 -o publish/win-x64 --self-contained
- name: Publish macOS
if: matrix.os == 'macos-latest'
run: dotnet publish -c ${{ matrix.configuration }} -r osx-x64 -o publish/osx-x64 --self-contained
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: publish-${{ matrix.os }}
path: publish/Matrix with constraints
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
framework: ['net8.0', 'net9.0']
exclude:
# .NET 8 не поддерживается на Windows Server 2019
- os: windows-2019
framework: 'net9.0'
include:
# Дополнительные платформы для Release
- os: ubuntu-latest
framework: 'net9.0'
configuration: Release
run-publish: trueArtifact management
Publishing NuGet packages
name: Publish NuGet Package
on:
push:
tags: ['v*']
jobs:
publish-nuget:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- name: Extract version from tag
id: version
run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
- name: Pack
run: |
dotnet pack src/MyLibrary/MyLibrary.csproj \
-c Release \
-p:PackageVersion=${{ steps.version.outputs.VERSION }} \
-o ./nupkg
- name: Push to NuGet
run: dotnet nuget push ./nupkg/*.nupkg --api-key ${{ secrets.NUGET_API_KEY }} --source https://api.nuget.org/v3/index.jsonPublishing Docker images
name: Build and Push Docker Image
on:
push:
branches: [main]
tags: ['v*']
jobs:
docker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/${{ github.repository }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=maxArtifact retention policies
| Тип артефакта | Хранение | Назначение |
|---|---|---|
| Test results | 30 дней | Отладка упавших тестов |
| Build output | 5 дней | Deployment |
| NuGet package | Безлимитно | PackageManager |
| Docker image | Безлимитно | Container registry |
| Code coverage | 30 дней | Quality gates |
Quality gates
Code coverage
- name: Test with coverage
run: |
dotnet test --no-build \
-c Release \
/p:CollectCoverage=true \
/p:CoverletOutputFormat=cobertura \
/p:CoverletOutput=./coverage/cobertura.xml \
/p:Exclude="[xunit.*]*"
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
files: ./coverage/cobertura.xml
flags: unittests
name: codecov-aspnetcore
fail_ci_if_error: true.NET Analyzers
- name: Run .NET analyzers
run: dotnet build -c Release /p:AnalysisMode=Recommended /p:EnforceCodeStyleInBuild=true
# Строгий режим — build fails на warning
- name: Strict build
run: dotnet build -c Release /warnaserror /p:TreatWarningsAsErrors=trueSecurity scanning
- name: Dependency review
uses: actions/dependency-review-action@v4
if: github.event_name == 'pull_request'
- name: CodeQL analysis
uses: github/codeql-action/init@v3
with:
languages: csharp
steps:
- uses: github/codeql-action/analyze@v3Практика
Задание 1: CI pipeline с parallel test execution и caching
Требования:
- restore → build → test → publish stages
- NuGet caching с
actions/cache - Parallel test execution (unit + integration)
- Service containers (PostgreSQL, Redis)
- Artifact upload для test results
Задание 2: Matrix build для net8.0/net9.0 + Linux/Windows
Требования:
- Matrix по .NET 8.0 и 9.0
- Matrix по OS (ubuntu-latest, windows-latest)
- fail-fast: false
- Разные стратегии для Release/Debug
Задание 3: Artifact publishing с semantic versioning
Требования:
- Publish NuGet package при теге v*
- Publish Docker image с semver tags
- Multi-platform (linux/amd64, linux/arm64)
- GHCR registry
CD и Deployment Strategies
Содержание
- Blue-Green deployment
- Canary releases
- Rolling updates
- Feature flags
- Database migration в CI/CD pipeline
- Практические задания
Blue-Green deployment
Концепция
Blue-Green deployment использует две одинаковые среды:
- Blue — текущая production версия
- Green — новая версия для развёртывания
Переключение происходит через load balancer / ingress — мгновенно и обратимо.
Load Balancer
│
┌────────┴────────┐
│ │
[Blue] [Green]
v2.0 (active) v2.1 (idle)
│ │
Database (общая)Как работает
Phase 1: Blue = production, Green = idle
LB → Blue (v2.0)
Phase 2: Deploy v2.1 к Green
LB → Blue (v2.0)
Green ← v2.1 (deployed, health checked)
Phase 3: Switch LB к Green
LB → Green (v2.1) ← production traffic
Phase 4: Blue = standby / rollback target
LB → Green (v2.1)
Blue ← v2.0 (standby, ready for instant rollback)Реализация в Kubernetes
# blue-green-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-blue
spec:
replicas: 3
selector:
matchLabels:
app: myapp
version: blue
template:
metadata:
labels:
app: myapp
version: blue
spec:
containers:
- name: myapp
image: myapp:v2.0
ports:
- containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-green
spec:
replicas: 0 # Green неактивен
selector:
matchLabels:
app: myapp
version: green
template:
metadata:
labels:
app: myapp
version: green
spec:
containers:
- name: myapp
image: myapp:v2.1
ports:
- containerPort: 8080
---
# Service переключается через selector
apiVersion: v1
kind: Service
metadata:
name: myapp
spec:
selector:
app: myapp
version: blue # Переключить на 'green' для deployment
ports:
- port: 80
targetPort: 8080Health check validation перед переключением
# GitHub Actions — Blue-Green с health check
name: Blue-Green Deployment
on:
push:
tags: ['v*']
jobs:
deploy-blue-green:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy to Green
run: |
kubectl set image deployment/myapp-green \
myapp=ghcr.io/myapp:${{ github.sha }}
kubectl rollout status deployment/myapp-green --timeout=300s
- name: Health check Green
run: |
for i in {1..30}; do
if kubectl exec deployment/myapp-green -- curl -f http://localhost:8080/health/ready; then
echo "Green is healthy!"
break
fi
sleep 10
done
- name: Switch traffic to Green
if: success()
run: |
kubectl patch service myapp -p '{"spec":{"selector":{"app":"myapp","version":"green"}}}'
- name: Verify production
run: |
curl -f https://myapp.example.com/health/ready
- name: Rollback on failure
if: failure()
run: |
kubectl patch service myapp -p '{"spec":{"selector":{"app":"myapp","version":"blue"}}}'
echo "Rolled back to Blue"Преимущества и недостатки
| Плюсы | Минусы |
|---|---|
| Мгновенный rollback | Двойная инфраструктура |
| Нулевое даунтайм | Двойные затраты на ресурсы |
| Простое тестирование новой версии | Общая БД — миграции сложны |
| Нет DNS propagation | Требует stateless приложений |
Canary releases
Концепция
Canary release постепенно перенаправляет трафик на новую версию:
10% → v2.1 (canary)
90% → v2.0 (stable)
↓
5% → v2.1
95% → v2.0
↓
25% → v2.1
75% → v2.0
↓
50% → v2.1
50% → v2.0
↓
100% → v2.1Реализация с Kubernetes + Istio
# Istio VirtualService для canary
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: myapp
spec:
hosts:
- myapp.example.com
http:
- route:
- destination:
host: myapp
subset: stable
weight: 90
- destination:
host: myapp
subset: canary
weight: 10
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: myapp
spec:
host: myapp
subsets:
- name: stable
labels:
version: v2.0
- name: canary
labels:
version: v2.1Автоматический rollback на основе метрик
# GitHub Actions — Canary с автоматическим rollback
name: Canary Release with Auto-Rollback
on:
push:
branches: [main]
jobs:
canary-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy canary (5%)
run: |
kubectl set image deployment/myapp \
myapp=ghcr.io/myapp:${{ github.sha }}-canary
kubectl apply -f canary-5.yaml
- name: Wait and monitor
run: |
sleep 300 # 5 минут наблюдения
- name: Check error rate
id: error-check
run: |
ERROR_RATE=$(curl -s http://prometheus:9090/api/v1/query \
--data-urlencode 'query=rate(http_requests_total{status=~"5..",pod=~"myapp-canary.*"}[5m])' \
| jq '.data.result[0].value[1]')
echo "Error rate: $ERROR_RATE"
if (( $(echo "$ERROR_RATE > 0.01" | bc -l) )); then
echo "Error rate too high! $ERROR_RATE > 0.01"
exit 1
fi
- name: Promote canary (50%)
if: success()
run: |
kubectl apply -f canary-50.yaml
- name: Promote canary (100%)
if: success()
run: |
kubectl apply -f canary-100.yaml
kubectl delete deployment/myapp-canary
- name: Rollback on failure
if: failure()
run: |
kubectl set image deployment/myapp myapp=ghcr.io/myapp:v2.0
kubectl apply -f stable-only.yaml
echo "Rolled back to v2.0"Автоматизация с Argo Rollouts
# argo-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: myapp
spec:
replicas: 10
revisionHistoryLimit: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: ghcr.io/myapp:latest
ports:
- containerPort: 8080
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"
strategy:
canary:
steps:
- setWeight: 10
pause: {duration: 5m}
- analysis:
templates:
- templateName: canary-analysis
- setWeight: 25
pause: {duration: 5m}
- analysis:
templates:
- templateName: canary-analysis
- setWeight: 50
pause: {duration: 10m}
- analysis:
templates:
- templateName: canary-analysis
- setWeight: 100
---
# Analysis template для canary
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: canary-analysis
spec:
metrics:
- name: error-rate
provider:
prometheus:
address: http://prometheus:9090
query: >-
sum(rate(http_requests_total{job="myapp",status=~"5.."}[5m]))
/
sum(rate(http_requests_total{job="myapp"}[5m]))
successCondition: result[0] < 0.01
- name: latency-p99
provider:
prometheus:
address: http://prometheus:9090
query: >-
histogram_quantile(0.99, rate(http_request_duration_seconds_bucket{job="myapp"}[5m]))
successCondition: result[0] < 2Преимущества и недостатки
| Плюсы | Минусы |
|---|---|
| Постепенное распространение рисков | Требует observability |
| Можно откатить на любом этапе | Сложнее для отладки (два версия одновременно) |
| Реальные данные от реальных пользователей | Больше инфраструктуры |
| A/B testing possible | Нужен service mesh или advanced LB |
Rolling updates
Концепция
Kubernetes rolling update постепенно заменяет старые поды новыми:
[old] [old] [old] [old] [old]
↓
[new] [old] [old] [old] [old]
[new] [old] [old] [old] [old]
↓
[new] [new] [old] [old] [old]
[new] [new] [old] [old] [old]
↓
[new] [new] [new] [new] [new]Rolling update configuration
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1 # Максимум 1 под выше desired (всего 6)
maxUnavailable: 1 # Максимум 1 под недоступен во время обновления
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8080
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 15"] # Graceful shutdown
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 15
periodSeconds: 20Graceful shutdown в ASP.NET Core
// Program.cs — graceful shutdown
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
// Настройка graceful shutdown
app.Lifetime.ApplicationStopping.Register(() =>
{
app.Logger.LogInformation("Shutting down gracefully...");
});
app.Run();// Middleware для завершения текущих запросов
public class GracefulShutdownMiddleware
{
private readonly RequestDelegate _next;
private readonly ILogger<GracefulShutdownMiddleware> _logger;
public GracefulShutdownMiddleware(RequestDelegate next, ILogger<GracefulShutdownMiddleware> logger)
{
_next = next;
_logger = logger;
}
public async Task InvokeAsync(HttpContext context)
{
var stoppingToken = context.RequestAborted;
if (stoppingToken.IsCancellationRequested)
{
_logger.LogWarning("Rejecting request during shutdown");
context.Response.StatusCode = 503;
return;
}
await _next(context);
}
}# Kubernetes — preStop hook для drain
spec:
template:
spec:
containers:
- name: myapp
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 15"]
postStart:
exec:
command: ["/bin/sh", "-c", "sleep 5"]Преимущества и недостатки
| Плюсы | Минусы |
|---|---|
| Нет двойной инфраструктуры | Медленнее blue-green |
| Постепенное обновление | Временный mix версий |
| Стандартный подход K8s | Нужно обеспечить совместимость |
| Минимальные ресурсы | Требует readiness probes |
Feature flags
Что такое Feature Flags
Feature flag — механизм включения/выключения функциональности без деплоя нового кода.
Deployment vs Release:
- Deployment — технический процесс развёртывания кода
- Release — момент когда функциональность доступна пользователям
Feature flags разделяют эти два процесса:
Code deployed → Feature flag OFF → Feature flag ON → Feature removed
(deployment) (hidden) (release) (cleanup)Реализация в .NET
// Program.cs — настройка Feature Flags
var builder = WebApplication.CreateBuilder(args);
// Feature Flags из Configuration
builder.Services.AddFeatureFlags(options =>
{
options.Source = new ConfigurationFeatureFlagSource(
builder.Configuration.GetSection("FeatureFlags"));
});
var app = builder.Build();
app.UseFeatureFlags();
// Controller с feature flag
[ApiController]
[Route("api/[controller]")]
public class FeaturesController : ControllerBase
{
private readonly IFeatureFlagService _flags;
public FeaturesController(IFeatureFlagService flags)
{
_flags = flags;
}
[HttpGet("new-checkout")]
public IActionResult NewCheckout()
{
if (!_flags.IsEnabled("NewCheckout"))
return BadRequest("Feature not available");
return Ok("New checkout flow");
}
}Конфигурация
// appsettings.json
{
"FeatureFlags": {
"NewCheckout": false,
"DarkMode": true,
"BetaAPI": {
"Enabled": true,
"AllowList": ["user1@example.com", "user2@example.com"]
}
}
}Персистентные feature flags
// Использование Azure App Configuration для персистентных флагов
builder.Services.AddAzureAppConfiguration()
.Connect(new Uri(builder.Configuration["AppConfig:Endpoint"]))
.ConfigureKeyVault(kv => kv.SetCredential(new DefaultAzureCredential()))
.UseFeatureFlags();Feature flag lifecycle
Phase 1: Creation
FeatureFlag.Create("NewCheckout", enabled: false)
Phase 2: Deploy with flag OFF
Код в production, но функция отключена
Phase 3: Enable for internal users
FeatureFlag.Set("NewCheckout", enabled: true, targetUsers: ["@internal"])
Phase 4: Canary release (10% users)
FeatureFlag.Set("NewCheckout", enabled: true, percentage: 10)
Phase 5: Full release
FeatureFlag.Set("NewCheckout", enabled: true, percentage: 100)
Phase 6: Cleanup (после 1-2 релизов)
Удалить флаг и старый кодDatabase migration в CI/CD pipeline
Zero-downtime migration strategies
Expand-Contract pattern
Phase 1: Expand (additive)
- Добавить новую колонку
- Писать в старую И новую колонки
- Деплой приложения (читает старую, пишет обе)
Phase 2: Backfill (data migration)
- Заполнить новую колонку существующими данными
- НЕ требует даунтайма
Phase 3: Switch (cutover)
- Приложение читает из новой колонки
- Деплой приложения
Phase 4: Contract (removal)
- Перестать писать в старую колонку
- Деплой приложения
- Удалить старую колонкуMigration в CI/CD — Entity Framework
# CI/CD с миграциями БД
name: Deploy with Database Migration
on:
push:
branches: [main]
jobs:
migrate-and-deploy:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_DB: myapp
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- name: Run migrations
run: |
dotnet ef database update \
--connection "Host=localhost;Database=myapp;Username=postgres;Password=postgres"
working-directory: ./src/MigrationTool
- name: Health check
run: curl -f http://localhost:8080/health/ready
- name: Deploy
run: |
kubectl set image deployment/myapp myapp=ghcr.io/myapp:${{ github.sha }}Безопасные миграции — обратная совместимость
// BAD: Ломающая миграция
public class Migration001 : Migration
{
protected override void Up(MigrationBuilder mb)
{
mb.DropColumn("Orders", "OldTotal");
mb.RenameColumn("Orders", "NewTotal", "Total");
}
}
// GOOD: Expand-first миграция
public class Migration001_Expand : Migration
{
protected override void Up(MigrationBuilder mb)
{
mb.AddColumn("Orders", "Total", type => type.Decimal(18, 2));
}
}
public class Migration002_Backfill : Migration
{
protected override void Up(MigrationBuilder mb)
{
// mb.Sql("UPDATE Orders SET Total = OldTotal WHERE Total IS NULL");
}
}
public class Migration003_Switch : Migration
{
protected override void Up(MigrationBuilder mb)
{
// Приложение читает из Total
}
}
public class Migration004_Contract : Migration
{
protected override void Up(MigrationBuilder mb)
{
mb.DropColumn("Orders", "OldTotal");
}
}Сравнение стратегий deployment
| Стратегия | Даунтайм | Сложность | Риск | Rollback | Ресурсы |
|---|---|---|---|---|---|
| Blue-Green | 0 | Средняя | Низкий | Мгновенный | 2x |
| Canary | 0 | Высокая | Очень низкий | Быстрый | 1.5x |
| Rolling | 0 | Низкая | Средний | Средний | 1x |
| Recreate | Да | Низкая | Высокий | Мгновенный | 1x |
Практика
Задание 1: Blue-Green deployment с health check
Требования:
- Два deployment (blue + green)
- Service для переключения
- Health check перед переключением
- Автоматический rollback при fail
Задание 2: Canary release с automatic rollback
Требования:
- Istio или nginx для weight-based routing
- Постепенное увеличение трафика (5% → 25% → 50% → 100%)
- Проверка error rate на каждом этапе
- Автоматический rollback при error rate > 1%
Задание 3: Zero-downtime database migration
Требования:
- Expand: добавить новую колонку
- Backfill: заполнить данные
- Switch: переключить приложение
- Contract: удалить старую колонку
- Каждый этап — отдельный migration
Kubernetes для .NET Developers
Содержание
- Pods, Deployments, Services, Ingress
- ConfigMap и Secret — configuration management
- Resource requests/limits — CPU, memory, QoS classes
- Horizontal Pod Autoscaler (HPA)
- Pod Disruption Budgets (PDB)
- Graceful shutdown с preStop hook
- Практические задания
Core concepts
Kubernetes Architecture
┌─────────────────── Control Plane ───────────────────┐
│ │
│ API Server etcd Scheduler Controller │
│ (REST API) (DB) (decides) (reconcile) │
│ │
└──────────────────────────────────────────────────────┘
│
┌────────┴────────┐
│ │
Node 1 Node 2
┌────────┐ ┌────────┐
│ Kubelet│ │ Kubelet│
│ Kube-proxy │ │ Kube-proxy │
│ Container Runtime │ │ Container Runtime │
│ [Pod] [Pod] [Pod] │ │ [Pod] [Pod] │
└────────┘ └────────┘Pod
Минимальная единица развёртывания. Содержит один или более контейнеров.
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8080
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"Deployment
Управляет Pod'ами с rolling updates, rollback, scaling.
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8080
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 10Service
Стабильный network endpoint для Pod'ов.
# ClusterIP — внутренний доступ
apiVersion: v1
kind: Service
metadata:
name: myapp
spec:
type: ClusterIP
selector:
app: myapp
ports:
- port: 80
targetPort: 8080
protocol: TCP
---
# NodePort — доступ через порт ноды
apiVersion: v1
kind: Service
metadata:
name: myapp-nodeport
spec:
type: NodePort
selector:
app: myapp
ports:
- port: 80
targetPort: 8080
nodePort: 30080
---
# LoadBalancer — внешний IP (cloud provider)
apiVersion: v1
kind: Service
metadata:
name: myapp-lb
spec:
type: LoadBalancer
selector:
app: myapp
ports:
- port: 80
targetPort: 8080Ingress
HTTP/HTTPS routing, TLS termination, virtual hosting.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
# nginx ingress
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "true"
# Rate limiting
nginx.ingress.kubernetes.io/limit-rps: "100"
spec:
ingressClassName: nginx
tls:
- hosts:
- myapp.example.com
secretName: myapp-tls
rules:
- host: myapp.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp
port:
number: 80
- path: /api
pathType: Prefix
backend:
service:
name: myapp-api
port:
number: 80ConfigMap и Secret
ConfigMap — конфигурация
# configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: myapp-config
data:
# Простые key-value
ASPNETCORE_ENVIRONMENT: Production
ConnectionStrings__DefaultConnection: "Host=postgres;Database=myapp;"
Logging__LogLevel__Default: Information
FeatureFlags__NewCheckout: "false"
# Файлы
appsettings.production.json: |
{
"Logging": {
"LogLevel": {
"Default": "Warning"
}
},
"Serilog": {
"Using": ["Serilog.Sinks.File"],
"WriteTo": [{"Name": "File", "Args": {"path": "/logs/log.txt"}}]
}
}Использование ConfigMap в Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
template:
spec:
containers:
- name: myapp
image: myapp:latest
# Env из ConfigMap
envFrom:
- configMapRef:
name: myapp-config
# Отдельные env vars
env:
- name: DB_HOST
valueFrom:
configMapKeyRef:
name: myapp-config
key: ConnectionStrings__DefaultConnection
# Mount как файл
volumeMounts:
- name: config-volume
mountPath: /app/appsettings.production.json
subPath: appsettings.production.json
volumes:
- name: config-volume
configMap:
name: myapp-configSecret — чувствительные данные
# secret.yaml — Base64 encoded
apiVersion: v1
kind: Secret
metadata:
name: myapp-secret
type: Opaque
stringData: # Не base64 — автоматически кодируется
ConnectionStrings__DefaultConnection: "Host=postgres;Database=myapp;Username=admin;Password=S3cret!"
ApiKey__Stripe: "sk_live_abc123"
Jwt__Key: "my-super-secret-jwt-key-12345"Secret с Kubernetes external secrets (AWS Secrets Manager)
# ExternalSecret — синхронизация с AWS Secrets Manager
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: myapp-secrets
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secrets-manager
kind: ClusterSecretStore
target:
name: myapp-secret
data:
- secretKey: ConnectionStrings__DefaultConnection
remoteRef:
key: production/myapp/connection-string
- secretKey: ApiKey__Stripe
remoteRef:
key: production/myapp/stripe-api-keyResource requests/limits — QoS classes
Resource requests vs limits
| Параметр | Значение | Описание |
|---|---|---|
| requests.memory | 128Mi | Минимум памяти для запуска |
| requests.cpu | 100m | Минимум CPU (0.1 ядра) |
| limits.memory | 256Mi | Максимум памяти (OOM Kill при превышении) |
| limits.cpu | 200m | Максимум CPU (throttling при превышении) |
QoS Classes
| Class | Requirements | Preemption | Описание |
|---|---|---|---|
| Guaranteed | requests == limits для всех контейнеров | Нет | Максимальный приоритет |
| Burstable | requests < limits | Возможно | Стандартный класс |
| BestEffort | нет requests/limits | Да | Минимальный приоритет |
Примеры QoS
# Guaranteed — requests == limits
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-guaranteed
spec:
template:
spec:
containers:
- name: myapp
image: myapp:latest
resources:
requests:
memory: "256Mi"
cpu: "200m"
limits:
memory: "256Mi" # == requests
cpu: "200m" # == requests
# Burstable — requests < limits
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-burstable
spec:
template:
spec:
containers:
- name: myapp
image: myapp:latest
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "512Mi" # > requests
cpu: "500m" # > requests
# BestEffort — нет requests/limits
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-besteffort
spec:
template:
spec:
containers:
- name: myapp
image: myapp:latest
# Нет ресурсов — BestEffort.NET-specific resource tuning
// Program.cs — настройка для контейнерной среды
var builder = WebApplication.CreateBuilder(args);
// Ограничение ThreadPool для контейнеров
var cpuCount = Environment.ProcessorCount;
var threadPoolSize = Math.Max(cpuCount * 4, 256);
ThreadPool.SetMinThreads(threadPoolSize, threadPoolSize);
// Memory limits awareness
var memoryLimit = Process.GetCurrentProcess().WorkingSet64;
// Или через cgroup:
// var memLimit = File.ReadAllText("/sys/fs/cgroup/memory/memory.limit_in_bytes");
builder.Services.AddDbContext<AppDbContext>(options =>
options.UseSqlServer(builder.Configuration.GetConnectionString("Default")));
var app = builder.Build();
app.Run();Horizontal Pod Autoscaler (HPA)
Базовый HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
behavior:
scaleUp:
stabilizationWindowSeconds: 60
policies:
- type: Percent
value: 100
periodSeconds: 60
- type: Pods
value: 4
periodSeconds: 60
selectPolicy: Max
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 10
periodSeconds: 60HPA на основе custom metrics (RPS)
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa-rps
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 50
metrics:
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "100" # 100 RPS на под
behavior:
scaleUp:
stabilizationWindowSeconds: 30
policies:
- type: Pods
value: 5
periodSeconds: 60
scaleDown:
stabilizationWindowSeconds: 300KEDA — event-driven autoscaling
# KEDA ScaledObject для .NET Worker Service с RabbitMQ
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: myapp-worker-scaledobject
spec:
scaleTargetRef:
name: myapp-worker
triggers:
- type: rabbitmq
metadata:
protocol: amqp
host: amqp://rabbitmq:5672
queueName: orders
consumptionPerPod: "10"
activationValue: "50"
cooldownPeriod: 300
minReplicaCount: 0 # Scale to zero
maxReplicaCount: 20Pod Disruption Budgets (PDB)
Зачем нужен PDB
Предотвращает одновременное удаление слишком многих подов при maintenance, upgrades, node drains.
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: myapp-pdb
spec:
minAvailable: 2 # Минимум 2 пода доступны
selector:
matchLabels:
app: myapp
---
# Или через maxUnavailable
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: myapp-pdb
spec:
maxUnavailable: 1 # Максимум 1 под недоступен
selector:
matchLabels:
app: myappPDB с HPA
# PDB должен учитывать minReplicas HPA
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: myapp-pdb
spec:
minAvailable: 2 # Совпадает с HPA minReplicas
selector:
matchLabels:
app: myappGraceful shutdown с preStop hook
preStop hook для drain
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
template:
spec:
containers:
- name: myapp
image: myapp:latest
lifecycle:
preStop:
exec:
# Ключевой момент: preStop должен быть >= readiness probe timeout
command: ["/bin/sh", "-c", "sleep 15"]
postStart:
exec:
command: ["/bin/sh", "-c", "sleep 5"]Graceful shutdown в ASP.NET Core
// Program.cs
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
// Отказываем новым запросам при shutdown
app.Lifetime.ApplicationStopping.Register(() =>
{
app.Logger.LogInformation("Application stopping — rejecting new requests");
});
// Обработка текущих запросов
app.Use(async (context, next) =>
{
if (app.Lifetime.ApplicationStopping.IsCancellationRequested)
{
context.Response.StatusCode = 503;
return;
}
await next();
});
app.Run();Worker Service graceful shutdown
public class OrderProcessingWorker : BackgroundService
{
private readonly ILogger<OrderProcessingWorker> _logger;
private readonly CancellationTokenSource _shutdownCts;
public OrderProcessingWorker(ILogger<OrderProcessingWorker> logger)
{
_logger = logger;
_shutdownCts = CancellationTokenSource.CreateLinkedTokenSource(
Context.StopToken);
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
_logger.LogInformation("Worker started");
while (!stoppingToken.IsCancellationRequested)
{
try
{
var order = await GetNextOrderAsync(stoppingToken);
if (order != null)
{
await ProcessOrderAsync(order, stoppingToken);
}
else
{
await Task.Delay(1000, stoppingToken);
}
}
catch (OperationCanceledException)
{
_logger.LogInformation("Worker stopping gracefully");
break;
}
}
_logger.LogInformation("Worker stopped");
}
}Полный manifest для stateless API
# myapp-full.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
labels:
app: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: ghcr.io/myorg/myapp:latest
ports:
- containerPort: 8080
name: http
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"
envFrom:
- configMapRef:
name: myapp-config
- secretRef:
name: myapp-secret
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
failureThreshold: 3
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 3
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 15"]
---
apiVersion: v1
kind: Service
metadata:
name: myapp
spec:
type: ClusterIP
selector:
app: myapp
ports:
- port: 80
targetPort: 8080
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: myapp-pdb
spec:
minAvailable: 2
selector:
matchLabels:
app: myapp
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- myapp.example.com
secretName: myapp-tls
rules:
- host: myapp.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp
port:
number: 80Практика
Задание 1: Kubernetes manifest для stateless API с HPA
Требования:
- Deployment с 3 репликами
- Service (ClusterIP)
- HPA (CPU 70%, min 2, max 20)
- PDB (minAvailable: 2)
- Resource requests/limits
- Liveness и readiness probes
- ConfigMap + Secret
Задание 2: Ingress controller с TLS termination
Требования:
- Ingress с TLS
- Cert-manager для auto TLS
- Rate limiting
- SSL redirect
Задание 3: Graceful shutdown с preStop hook и drain period
Требования:
- preStop hook с sleep 15s
- ASP.NET Core middleware для 503 при shutdown
- Worker Service с graceful cancellation
Advanced CI/CD Patterns
Содержание
- Trunk-based development
- Conventional Commits
- Semantic Release
- Environment promotion
- GitOps — ArgoCD, Flux
- Практические задания
Trunk-based development
Что такое Trunk-based development
Trunk-based development — практика, при которой все разработчики работают с короткоживущими ветками (менее 1-2 дней) от основной ветки (trunk/main).
main ─────────────────────────────────────────────────────────
└─ feature-a (2 часа) ─────────┐
├─ merge
└─ feature-b (1 день) ─────────┘
│
└─ feature-c (3 часа) ────────┘vs Git-flow
| Критерий | Git-flow | Trunk-based |
|---|---|---|
| Длительность веток | Дни/недели | Часы/дни |
| Merge frequency | Редко | Часто |
| Feature flags | Нет | Да |
| CI/CD | Сложный | Простой |
| Конфликты merge | Частые | Редкие |
| Подходит для | Стабильные релизы | Continuous delivery |
Trunk-based workflow
# .github/workflows/trunk-ci.yml
name: Trunk CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- name: Restore
run: dotnet restore
- name: Build
run: dotnet build -c Release
- name: Test
run: dotnet test -c Release --collect:"XPlat Code Coverage"
- name: Upload coverage
uses: codecov/codecov-action@v4
with:
fail_ci_if_error: true
- name: Security scan
run: |
dotnet tool install -g dotnet-outdated-tool
dotnet outdated --highest-minor
- name: Docker build (PR only)
if: github.event_name == 'pull_request'
uses: docker/build-push-action@v5
with:
push: false
tags: pr-${{ github.event.pull_request.number }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Docker build + push (main only)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
with:
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=maxFeature flags для trunk-based
// Код в main, но функциональность скрыта
[ApiController]
[Route("api/[controller]")]
public class CheckoutController : ControllerBase
{
private readonly IFeatureFlagService _flags;
public CheckoutController(IFeatureFlagService flags)
{
_flags = flags;
}
[HttpPost]
public async Task<IActionResult> Checkout([FromBody] CheckoutRequest request)
{
// Функциональность в main, но доступна только при включённом флаге
if (!_flags.IsEnabled("NewCheckoutV2"))
return BadRequest("Feature not available");
return await ProcessNewCheckoutAsync(request);
}
}Conventional Commits
Спецификация
<type>(<scope>): <description>
[optional body]
[optional footer(s)]Types
| Type | Описание | Пример |
|---|---|---|
| feat | Новая функциональность | feat(auth): add OAuth2 login |
| fix | Исправление бага | fix(api): handle null user |
| docs | Документация | docs(readme): update setup guide |
| style | Форматирование (не код) | style: fix indentation |
| refactor | Рефакторинг (не фикс, не фича) | refactor(payment): simplify logic |
| perf | Улучшение производительности | perf(query): add index on users |
| test | Тесты | test(auth): add unit tests |
| chore | Рутинные изменения | chore(deps): update nuget packages |
| ci | CI/CD | ci(actions): add docker build |
| build | Build system | build(docker): optimize layers |
| revert | Откат предыдущего коммита | revert: feat(auth): add OAuth2 |
Примеры
feat(api): add user registration endpoint
fix(payment): handle expired credit cards
docs: update API documentation for v2
refactor(order): extract validation logic
perf(db): add index on orders.created_at
test(auth): add integration tests for login
chore(deps): update Microsoft.Extensions to 9.0.0
ci: add GitHub Actions workflow for deployment
build(docker): add multi-stage build
revert: revert "feat: add dark mode"Commit message validation
# .github/workflows/commit-lint.yml
name: Commit Lint
on: [pull_request]
jobs:
lint-commits:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Conventional commits check
uses: webiny/action-conventional-commits@v1.3.0Semantic Release
Автоматическое версионирование
// .releaserc.json
{
"branches": ["main"],
"plugins": [
["@semantic-release/commit-analyzer", {
"preset": "conventionalcommits"
}],
["@semantic-release/release-notes-generator", {
"preset": "conventionalcommits"
}],
"@semantic-release/changelog",
["@semantic-release/npm", {
"npmPublish": false
}],
["@semantic-release/git", {
"assets": [
"package.json",
"CHANGELOG.md"
],
"message": "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}"
}],
["@semantic-release/github", {
"successComment": false,
"failTitle": false
}]
]
}Semantic Release в GitHub Actions
name: Semantic Release
on:
push:
branches: [main]
jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.RELEASE_TOKEN }}
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- name: Install semantic-release
run: |
npm init -y
npm install --save-dev semantic-release @semantic-release/git @semantic-release/changelog @semantic-release/github @semantic-release/commit-analyzer @semantic-release/release-notes-generator conventionalcommits.org
- name: Semantic Release
env:
GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}
run: npx semantic-releaseVersion bump rules
| Commit type | Version change | Example |
|---|---|---|
fix: | patch | 1.0.0 → 1.0.1 |
feat: | minor | 1.0.0 → 1.1.0 |
feat!(breaking): | major | 1.0.0 → 2.0.0 |
fix!(breaking): | major | 1.0.0 → 2.0.0 |
Environment promotion
Environment promotion workflow
dev → staging → production
Каждый environment — отдельная инфраструктураGitHub Actions — Environment promotion
name: Environment Promotion
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build and test
run: |
dotnet restore
dotnet build -c Release
dotnet test -c Release
- name: Docker build
uses: docker/build-push-action@v5
with:
push: false
tags: myapp:${{ github.sha }}
cache-from: type=gha
- name: Deploy to Dev
uses: azure/k8s-deploy@v4
with:
namespace: dev
manifests: k8s/dev/
images: |
myapp:${{ github.sha }}
environment: dev
deploy-staging:
needs: build
runs-on: ubuntu-latest
environment: staging
steps:
- uses: actions/checkout@v4
- name: Deploy to Staging
uses: azure/k8s-deploy@v4
with:
namespace: staging
manifests: k8s/staging/
images: |
myapp:${{ github.sha }}
deploy-production:
needs: deploy-staging
runs-on: ubuntu-latest
environment: production
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Deploy to Production
uses: azure/k8s-deploy@v4
with:
namespace: production
manifests: k8s/production/
images: |
myapp:${{ github.sha }}Manual approval gates
# GitHub Environments с manual approval
# settings → Environments → production → Required reviewers
deploy-production:
needs: deploy-staging
environment:
name: production
url: https://myapp.example.com
steps:
- name: Deploy to Production
run: |
kubectl set image deployment/myapp myapp=ghcr.io/myapp:${{ github.sha }}
kubectl rollout status deployment/myapp --timeout=300sGitOps — ArgoCD
GitOps концепция
GitOps — подход, при котором желаемое состояние инфраструктуры хранится в Git. GitOps-оператор (ArgoCD/Flux) автоматически синхронизирует кластер с Git.
Git Repository Kubernetes Cluster
┌─────────────────┐ ┌──────────────────┐
│ k8s/ │ │ │
│ staging/ │ ArgoCD │ staging/ │
│ production/ │ ──────────► │ Deployment │
│ values.yaml │ (sync) │ Service │
└─────────────────┘ │ ConfigMap │
│ Secret │
└──────────────────┘ArgoCD Application
# argo-cd-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp-production
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/myorg/myapp.git
targetRevision: main
path: k8s/production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true # Удалить ресурсы, которых нет в Git
selfHeal: true # Автоматически исправлять drift
syncOptions:
- CreateNamespace=trueArgoCD with Kustomize
k8s/
├── base/
│ ├── deployment.yaml
│ ├── service.yaml
│ └── kustomization.yaml
├── staging/
│ ├── overlays.yaml
│ └── kustomization.yaml
└── production/
├── overlays.yaml
└── kustomization.yaml# k8s/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml
images:
- name: myapp
newTag: v1.0.0# k8s/production/overlays.yaml
replicas: 5
resources:
- ../base
patches:
- target:
kind: Deployment
name: myapp
patch: |
- op: replace
path: /spec/template/spec/containers/0/resources/limits/memory
value: "512Mi"
- op: replace
path: /spec/template/spec/containers/0/resources/limits/cpu
value: "500m"Flux CD
# flux-sync.yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: myapp
namespace: default
spec:
interval: 1m
url: https://github.com/myorg/myapp.git
ref:
branch: main
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: myapp-production
namespace: default
spec:
interval: 5m
path: ./k8s/production
prune: true
wait: true
sourceRef:
kind: GitRepository
name: myapp
decryption:
provider: sops
secretRef:
name: sops-secretПрактика
Задание 1: Semantic release pipeline с conventional commits
Требования:
- Conventional commits validation в PR
- Автоматическое версионирование при merge в main
- Автоматическое создание GitHub Release
- CHANGELOG.md generation
Задание 2: Multi-environment promotion workflow
Требования:
- 3 environments: dev, staging, production
- Manual approval gate для production
- Каждый environment — отдельный namespace в K8s
- Environment-specific ConfigMap
Задание 3: GitOps deployment с ArgoCD
Требования:
- ArgoCD Application manifest
- Kustomize для overlay per environment
- Automated sync + self-heal
- Prune unmanaged resources
Infrastructure as Code
Содержание
- Terraform — state management, modules, workspaces
- Bicep / ARM templates для Azure resources
- Helm charts — packaging Kubernetes applications
- Environment parity — dev/staging/production consistency
- Практические задания
Terraform
State management
# backend.tf — Remote state в Azure Blob Storage
terraform {
backend "azurerm" {
resource_group_name = "terraform-state"
storage_account_name = "tfstatedev123"
container_name = "terraform"
key = "production/terraform.tfstate"
}
}State locking
terraform {
backend "azurerm" {
resource_group_name = "terraform-state"
storage_account_name = "tfstatedev123"
container_name = "terraform"
key = "production/terraform.tfstate"
# State locking через Blob Lease
use_oidc = true
}
}Module structure
infra/
├── modules/
│ ├── resource-group/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── postgresql/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── redis/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── kubernetes/
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
├── environments/
│ ├── dev/
│ │ ├── main.tf
│ │ └── variables.tf
│ ├── staging/
│ │ ├── main.tf
│ │ └── variables.tf
│ └── production/
│ ├── main.tf
│ └── variables.tf
└── shared/
├── networking.tf
└── monitoring.tfResource Group module
# modules/resource-group/main.tf
resource "azurerm_resource_group" "this" {
name = var.name
location = var.location
tags = var.tags
lifecycle {
prevent_destroy = false
}
}
# modules/resource-group/variables.tf
variable "name" {
description = "Resource group name"
type = string
}
variable "location" {
description = "Azure region"
type = string
default = "eastus"
}
variable "tags" {
description = "Resource tags"
type = map(string)
default = {}
}
# modules/resource-group/outputs.tf
output "id" {
description = "Resource group ID"
value = azurerm_resource_group.this.id
}
output "name" {
description = "Resource group name"
value = azurerm_resource_group.this.name
}PostgreSQL module
# modules/postgresql/main.tf
resource "azurerm_resource_group" "this" {
name = var.rg_name
location = var.location
tags = var.tags
}
resource "azurerm_postgresql_flexible_server" "this" {
name = var.name
resource_group_name = azurerm_resource_group.this.name
location = azurerm_resource_group.this.location
administrator_login = var.admin_username
administrator_password = var.admin_password
sku_name = var.sku_name
storage_mb = var.storage_mb
version = var.version
zone = var.zone
backup_retention_days = var.backup_retention_days
geo_redundant_backup = var.geo_redundant
tags = var.tags
network {
public_access = var.public_access
}
}
resource "azurerm_postgresql_flexible_server_database" "main" {
name = var.database_name
server_id = azurerm_postgresql_flexible_server.this.id
collation = var.collation
charset = var.charset
}Environment: Dev
# environments/dev/main.tf
terraform {
backend "azurerm" {
resource_group_name = "terraform-state"
storage_account_name = "tfstatedev123"
container_name = "terraform"
key = "dev/terraform.tfstate"
}
}
provider "azurerm" {
features {}
}
locals {
common_tags = {
Environment = "dev"
ManagedBy = "terraform"
Project = "myapp"
}
}
module "resource_group" {
source = "../../modules/resource-group"
name = "rg-myapp-dev"
location = "eastus"
tags = local.common_tags
}
module "postgresql" {
source = "../../modules/postgresql"
rg_name = module.resource_group.name
location = "eastus"
name = "psql-myapp-dev"
admin_username = "admin"
admin_password = var.db_password
sku_name = "B_Standard_B1s"
storage_mb = 32768
version = "16"
database_name = "myapp_dev"
public_access = true
zone = 1
tags = local.common_tags
}
module "redis" {
source = "../../modules/redis"
rg_name = module.resource_group.name
location = "eastus"
name = "redis-myapp-dev"
capacity = 0
family = "C"
tier = "Basic"
maxmemory_percent = 25
tags = local.common_tags
}
variable "db_password" {
description = "Database admin password"
type = string
sensitive = true
}Terraform Workspaces
# Создание workspace
terraform workspace new dev
terraform workspace new staging
terraform workspace new production
# Переключение
terraform workspace select dev
# Список workspace
terraform workspace list# Использование workspace для environment-specific config
variable "environment" {
type = string
default = ""
}
locals {
environment = var.environment != "" ? var.environment : terraform.workspace
common_tags = {
Environment = local.environment
ManagedBy = "terraform"
}
}
resource "azurerm_resource_group" "this" {
name = "rg-myapp-${local.environment}"
location = "eastus"
tags = local.common_tags
}Bicep / ARM templates
Bicep — Azure-native IaC
// main.bicep
param location string = resourceGroup().location
param environment string = 'dev'
param dbPassword secureString
resource resourceGroup 'Microsoft.Resources/resourceGroups@2023-07-01' = {
name: 'rg-myapp-${environment}'
location: location
tags: {
Environment: environment
ManagedBy: 'bicep'
}
}
resource postgresql 'Microsoft.DBforPostgreSQL/flexibleServers@2024-08-01' = {
name: 'psql-myapp-${environment}'
location: location
resourceGroupName: resourceGroup.name
sku: {
name: 'B_Standard_B1s'
tier: 'Basic'
}
properties: {
administratorLogin: 'admin'
administratorLoginPassword: dbPassword
highAvailability: {
mode: 'Disabled'
}
storage: {
storageSizeGB: 32
}
version: '16'
}
}
resource database 'Microsoft.DBforPostgreSQL/flexibleServers/databases@2024-08-01' = {
name: 'myapp_${environment}'
properties: {
charset: 'utf8'
collation: 'en_US.utf8'
}
parent: postgresql
}
output postgresqlFqdn string = postgresql.properties.fqdn
output databaseName string = database.nameBicep vs Terraform
| Критерий | Terraform | Bicep |
|---|---|---|
| Cloud support | Multi-cloud | Azure only |
| Language | HCL | Bicep (DSL) / ARM JSON |
| State management | Native | Нет (ARM) |
| Module system | Отличный | Отличный |
| Learning curve | Средняя | Низкая (для Azure devs) |
| .NET integration | Средний | Нативный (Azure CLI) |
Helm charts
Структура Helm chart
charts/myapp/
├── Chart.yaml
├── values.yaml
├── values.dev.yaml
├── values.staging.yaml
├── values.production.yaml
├── templates/
│ ├── deployment.yaml
│ ├── service.yaml
│ ├── hpa.yaml
│ ├── pdb.yaml
│ ├── ingress.yaml
│ └── tests/
│ └── test-connection.yaml
└── .helmignoreChart.yaml
apiVersion: v2
name: myapp
description: .NET ASP.NET Core application
type: application
version: 1.0.0
appVersion: "1.0.0"
dependencies:
- name: postgresql
version: 15.0.0
repository: https://charts.bitnami.com/bitnami
condition: postgresql.enabled
- name: redis
version: 18.0.0
repository: https://charts.bitnami.com/bitnami
condition: redis.enabledvalues.yaml
replicaCount: 2
image:
repository: ghcr.io/myorg/myapp
pullPolicy: IfNotPresent
tag: ""
service:
type: ClusterIP
port: 80
ingress:
enabled: true
className: nginx
hosts:
- host: myapp.local
paths:
- path: /
pathType: Prefix
tls: []
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"
autoscaling:
enabled: true
minReplicas: 2
maxReplicas: 20
targetCPUUtilizationPercentage: 70
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
env:
ASPNETCORE_ENVIRONMENT: Production
DOTNET_SYSTEM_NET_HTTP_SOCKETSHTTPHANDLER_HTTP2UNENCRYPTEDSUPPORT: "true"
postgresql:
enabled: true
auth:
database: myapp
username: postgres
password: postgres
redis:
enabled: true
auth:
enabled: falsetemplates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "myapp.fullname" . }}
labels:
{{- include "myapp.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "myapp.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "myapp.selectorLabels" . | nindent 8 }}
spec:
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 8080
protocol: TCP
env:
{{- range $key, $value := .Values.env }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
{{- with .Values.extraEnvFrom }}
{{- toYaml . | nindent 8 }}
{{- end }}
livenessProbe:
{{- toYaml .Values.livenessProbe | nindent 10 }}
readinessProbe:
{{- toYaml .Values.readinessProbe | nindent 10 }}
resources:
{{- toYaml .Values.resources | nindent 10 }}
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 15"]Override per environment
# Dev
helm install myapp-dev ./charts/myapp \
-f values.yaml \
-f values.dev.yaml \
--namespace dev \
--create-namespace
# Staging
helm install myapp-staging ./charts/myapp \
-f values.yaml \
-f values.staging.yaml \
--namespace staging \
--create-namespace
# Production
helm install myapp-prod ./charts/myapp \
-f values.yaml \
-f values.production.yaml \
--namespace production \
--create-namespacevalues.dev.yaml
replicaCount: 1
image:
tag: "latest"
ingress:
hosts:
- host: myapp-dev.local
resources:
requests:
memory: "64Mi"
cpu: "50m"
limits:
memory: "128Mi"
cpu: "100m"
autoscaling:
enabled: false
env:
ASPNETCORE_ENVIRONMENT: DevelopmentEnvironment parity
Parity checklist
| Компонент | Dev | Staging | Production |
|---|---|---|---|
| Kubernetes version | 1.30 | 1.30 | 1.30 |
| Node count | 2 | 3 | 5+ |
| Node size | Small | Medium | Large |
| Database tier | Basic | Standard | Premium |
| Redis tier | Basic | Standard | Premium |
| Monitoring | Basic | Full | Full + alerts |
| Backup | Daily | Hourly | Continuous |
| SSL/TLS | Self-signed | Let's Encrypt | Managed cert |
Infrastructure per environment
infra/
├── environments/
│ ├── dev/
│ │ ├── main.tf # Terraform config
│ │ ├── variables.tf # Dev-specific vars
│ │ └── outputs.tf # Dev outputs
│ ├── staging/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── production/
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
└── modules/ # Shared modules
├── resource-group/
├── postgresql/
├── redis/
└── kubernetes/CI/CD для IaC
name: Terraform CI/CD
on:
push:
paths:
- 'infra/**'
pull_request:
paths:
- 'infra/**'
jobs:
terraform-dev:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.7.0
- name: Terraform Init (dev)
run: terraform init
working-directory: infra/environments/dev
- name: Terraform Plan (dev)
run: terraform plan -out=tfplan
working-directory: infra/environments/dev
- name: Terraform Apply (dev)
if: github.ref == 'refs/heads/main'
run: terraform apply -auto-approve tfplan
working-directory: infra/environments/dev
terraform-production:
needs: terraform-dev
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
- name: Terraform Init (prod)
run: terraform init
working-directory: infra/environments/production
- name: Terraform Plan (prod)
run: terraform plan -out=tfplan
working-directory: infra/environments/production
- name: Terraform Apply (prod)
if: github.ref == 'refs/heads/main'
run: terraform apply -auto-approve tfplan
working-directory: infra/environments/productionПрактика
Задание 1: Terraform module для managed database
Требования:
- Azure PostgreSQL Flexible Server
- Автоматическое создание БД
- Network rules
- Backup configuration
- Outputs: connection string, FQDN
Задание 2: Helm chart с values override per environment
Требования:
- Chart.yaml с dependencies
- templates/ для deployment, service, ingress, HPA
- values.yaml (default)
- values.dev.yaml, values.staging.yaml, values.production.yaml
Задание 3: Infrastructure diff preview в PR comments
Требования:
- Terraform plan в PR
- Comment с diff
- Auto-approve для dev
- Manual approval для production
Platform Engineering
Содержание
- Internal Developer Platform (IDP)
- Backstage — developer portal
- Golden paths — standardized scaffolding
- Platform APIs — abstracting cloud complexity
- Практические задания
Internal Developer Platform (IDP)
Проблема: Developer Friction
Developer wants to deploy a new microservice:
1. Create Azure resource group ← manual
2. Set up PostgreSQL ← ask DBA team
3. Configure Redis ← ask infra team
4. Write Kubernetes manifests ← learn K8s YAML
5. Set up CI/CD pipeline ← copy from another project
6. Configure monitoring ← ask SRE team
7. Set up logging ← ask SRE team
8. Configure TLS/SSL ← ask security team
9. Set up secrets management ← ask security team
Total time: 2-3 daysРешение: Internal Developer Platform
Developer wants to deploy a new microservice:
1. Run: platform-cli create-service my-new-api
2. Platform creates everything automatically
3. Developer gets:
- Azure resources provisioned
- CI/CD pipeline ready
- Monitoring configured
- TLS/SSL set up
- Secrets managed
Total time: 30 secondsIDP Architecture
┌─────────────────────────────────────────────────────┐
│ Developer │
│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ Backstage│ │CLI/SDK │ │ Self-Service API │ │
│ └────┬─────┘ └────┬─────┘ └────────┬─────────┘ │
└───────┼──────────────┼─────────────────┼────────────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────┐
│ Platform Layer │
│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ Provision│ │ Deploy │ │ Observability │ │
│ │ Engine │ │ Engine │ │ Platform │ │
│ └────┬─────┘ └────┬─────┘ └────────┬─────────┘ │
└───────┼──────────────┼─────────────────┼────────────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────┐
│ Cloud Infrastructure │
│ Azure │ Kubernetes │ Networking │ Security │
└─────────────────────────────────────────────────────┘Backstage — Developer Portal
Backstage Architecture
┌──────────────────────────────────────────┐
│ Backstage App │
│ ┌──────────┐ ┌──────────┐ ┌─────────┐ │
│ │ Catalog │ │Scaffolder│ │TechDocs │ │
│ └──────────┘ └──────────┘ └─────────┘ │
│ ┌──────────┐ ┌──────────┐ ┌─────────┐ │
│ │ Search │ │Plugins │ │Groups │ │
│ └──────────┘ └──────────┘ └─────────┘ │
└──────────────────────────────────────────┘
│
┌───────┴──────────────────────────────────┐
│ Backstage Backend │
│ ┌──────────┐ ┌──────────┐ ┌─────────┐ │
│ │ Software │ │Entity │ │User │ │
│ │ Catalog │ │Proxies │ │Auth │ │
│ └──────────┘ └──────────┘ └─────────┘ │
└──────────────────────────────────────────┘
│
┌───────┴──────────────────────────────────┐
│ Data Sources │
│ GitHub │ Azure DevOps │ Jira │ etc│
└──────────────────────────────────────────┘Software Catalog — component.yaml
# catalog.yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: myapp-api
description: "Order processing API service"
annotations:
github.com/project-slug: myorg/myapp-api
backstage.io/techdocs-ref: dir:.
spec:
type: service
lifecycle: production
owner: team-platform
system: order-management
providesApis:
- myapp-api
---
apiVersion: backstage.io/v1alpha1
kind: API
metadata:
name: myapp-api
spec:
type: openapi
owner: team-platform
system: order-management
definition:
$text: ./openapi.yaml
---
apiVersion: backstage.io/v1alpha1
kind: System
metadata:
name: order-management
spec:
owner: team-orders
---
apiVersion: backstage.io/v1alpha1
kind: Group
metadata:
name: team-platform
spec:
type: team
members:
- user:alice
- user:bobScaffolder — templates
# scaffolder-templates/.scaffolder-template.yaml
apiVersion: scaffolder.backstage.io/v1beta2
kind: Template
metadata:
name: create-dotnet-api
title: Create .NET API Service
description: Scaffold a new .NET API service with full platform integration
spec:
owner: team-platform
type: service
parameters:
- title: Provide information about the new service
required:
- serviceName
- owner
properties:
serviceName:
title: Service Name
type: string
description: Unique name for your service
owner:
title: Team Owner
type: string
description: Team that owns this service
language:
title: .NET Version
type: string
default: net9.0
enum:
- net8.0
- net9.0
database:
title: Database
type: string
default: none
description: Database to provision
enum:
- none
- postgresql
- sqlserver
- redis
- title: Configure CI/CD
required: []
properties:
ciProvider:
title: CI Provider
type: string
default: github
enum:
- github
- azure-devops
deployTarget:
title: Deployment Target
type: string
default: kubernetes
enum:
- kubernetes
- azure-app-service
- title: Configure Observability
required: []
properties:
enableMonitoring:
title: Enable Monitoring
type: boolean
default: true
enableLogging:
title: Enable Logging
type: boolean
default: true
enableTracing:
title: Enable Distributed Tracing
type: boolean
default: true
steps:
- id: generate
name: Generate code
action: fetch:template
input:
url: ./template
values:
serviceName: {{ parameters.serviceName }}
owner: {{ parameters.owner }}
language: {{ parameters.language }}
database: {{ parameters.database }}
- id: provision-azure
name: Provision Azure resources
action: azure:arm:deploy
input:
armTemplate: ./infra/arm-template.json
parameters:
serviceName: {{ parameters.serviceName }}
owner: {{ parameters.owner }}
database: {{ parameters.database }}
- id: create-repo
name: Create GitHub repository
action: github:repo:create
input:
repoUrl: github.com?repo={{ parameters.serviceName }}&owner=myorg
- id: add-catalog
name: Add to software catalog
action: catalog:register
input:
catalogInfoPath: /catalog.yaml
- id: publish
name: Publish template
action: publish:github:pull-request
input:
repoUrl: github.com?repo={{ parameters.serviceName }}&owner=myorg
output:
- title: Service URL
icon: link
output: link
- title: OpenAPI Documentation
icon: docs
output:
$text: '{{ steps.generate.output.repoRelativeUrl }}/openapi.yaml'Golden path template
template/
├── src/
│ ├── {{ serviceName }}/
│ │ ├── {{ serviceName }}.csproj
│ │ ├── Program.cs
│ │ ├── Controllers/
│ │ │ └── WeatherForecastController.cs
│ │ ├── Services/
│ │ │ └── IHealthCheck.cs
│ │ ├── Middleware/
│ │ │ └── CorrelationIdMiddleware.cs
│ │ └── Filters/
│ │ └── GlobalExceptionFilter.cs
├── tests/
│ └── {{ serviceName }}.Tests/
│ ├── UnitTests.cs
│ └── IntegrationTests.cs
├── Dockerfile
├── docker-compose.yml
├── .github/
│ └── workflows/
│ ├── ci.yml
│ └── cd.yml
├── k8s/
│ ├── deployment.yaml
│ ├── service.yaml
│ └── ingress.yaml
├── catalog.yaml
├── openapi.yaml
└── README.mdGolden paths — standardized scaffolding
Что такое Golden Path
Golden Path — это заранее определённый, проверенный и поддерживаемый путь для типичных задач. Developer следует шаблону и получает:
- Рабочий CI/CD pipeline
- Настроенное monitoring
- Готовые best practices
- Поддержку от Platform team
Golden Path: .NET API Service
// Golden path template — Program.cs
using System.Diagnostics;
using Microsoft.AspNetCore.Mvc;
using OpenTelemetry.Trace;
var builder = WebApplication.CreateBuilder(args);
// Standard middleware pipeline
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddHealthChecks();
// OpenTelemetry — стандартный tracing
builder.Services.AddOpenTelemetry()
.WithTracing(tracing => tracing
.AddSource(builder.Environment.ApplicationName)
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddDbContextInstrumentation()
.SetResourceBuilder(ResourceBuilder.CreateDefault()
.AddService(builder.Environment.ApplicationName)));
// Serilog — стандартный logging
builder.Host.UseSerilog((context, services, configuration) => configuration
.ReadFrom.Configuration(context.Configuration)
.ReadFrom.Services(services)
.Enrich.WithProperty("Service", builder.Environment.ApplicationName)
.Enrich.WithProperty("Environment", builder.Environment.EnvironmentName)
.WriteTo.Console()
.WriteTo.OpenTelemetry());
// Resilience — Polly
builder.Services.AddHttpClient()
.AddResiliencePipeline("default", builder =>
{
builder.AddCircuitBreaker(new()
{
SamplingDuration = TimeSpan.FromSeconds(10),
FailureRatio = 0.5,
MinimumThroughput = 3,
SamplingPeriod = TimeSpan.FromSeconds(10)
})
.AddRetry(new()
{
MaxRetryAttempts = 3,
Delay = TimeSpan.FromMilliseconds(500),
BackoffType = DelayBackOffType.Exponential
});
});
var app = builder.Build();
// Standard middleware
app.UseMiddleware<CorrelationIdMiddleware>();
app.UseMiddleware<GlobalExceptionMiddleware>();
if (app.Environment.IsProduction())
{
app.UseHttpsRedirection();
}
app.MapHealthChecks("/health/live");
app.MapHealthChecks("/health/ready");
app.MapGet("/", () => Results.Ok(new { service = builder.Environment.ApplicationName }));
app.Run();Golden Path: Dockerfile
# Golden path Dockerfile
ARG BASE_IMAGE=mcr.microsoft.com/dotnet/sdk:9.0
# Build
FROM ${BASE_IMAGE} AS build
WORKDIR /src
COPY **/*.csproj ./
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o /app/publish --no-restore --no-build
# Test
FROM build AS test
RUN dotnet test -c Release --no-build
# Runtime
FROM mcr.microsoft.com/dotnet/aspnet:9.0-chiseled AS runtime
WORKDIR /app
COPY --from=build /app/publish .
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
USER appuser
ENV ASPNETCORE_ENVIRONMENT=Production
ENV ASPNETCORE_FORWARDEDHEADERS_ENABLED=true
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8080/health/live || exit 1
ENTRYPOINT ["dotnet", "MyApp.dll"]Golden Path: CI/CD
# Golden path CI/CD pipeline
name: CI/CD
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '9.0.x'
- run: dotnet restore
- run: dotnet build -c Release
- run: dotnet test -c Release --collect:"XPlat Code Coverage"
- uses: codecov/codecov-action@v4
- name: Run security scan
run: |
dotnet tool install -g dotnet-outdated-tool
dotnet outdated --fail-on-updates
docker:
needs: build-and-test
runs-on: ubuntu-latest
if: github.event_name == 'push'
steps:
- uses: actions/checkout@v4
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/build-push-action@v5
with:
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy:
needs: docker
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
environment: production
steps:
- uses: azure/k8s-deploy@v4
with:
namespace: production
manifests: k8s/
images: |
ghcr.io/${{ github.repository }}:${{ github.sha }}Platform APIs — abstracting cloud complexity
Self-Service API
// Platform API — .NET Web API для self-service provisioning
[ApiController]
[Route("api/platform/[controller]")]
public class ServicesController : ControllerBase
{
private readonly IProvisioningService _provisioning;
private readonly ILogger<ServicesController> _logger;
public ServicesController(
IProvisioningService provisioning,
ILogger<ServicesController> logger)
{
_provisioning = provisioning;
_logger = logger;
}
[HttpPost]
[ProducesResponseType(StatusCodes.Status202Accepted)]
public async Task<IActionResult> CreateService(
[FromBody] CreateServiceRequest request,
CancellationToken ct)
{
var job = await _provisioning.ProvisionAsync(new ProvisionRequest
{
ServiceName = request.ServiceName,
Owner = request.Owner,
Database = request.Database,
Environment = request.Environment
}, ct);
return Accepted(new
{
JobId = job.Id,
StatusUrl = Url.Action("GetJobStatus", new { job.Id }),
EstimatedTime = job.EstimatedDuration
});
}
[HttpGet("{jobId}")]
public async Task<IActionResult> GetJobStatus(Guid jobId, CancellationToken ct)
{
var status = await _provisioning.GetJobStatusAsync(jobId, ct);
return Ok(status);
}
}
public record CreateServiceRequest(
string ServiceName,
string Owner,
string Database,
string Environment);Provisioning Service
public interface IProvisioningService
{
Task<ProvisionJob> ProvisionAsync(ProvisionRequest request, CancellationToken ct);
Task<ProvisionStatus> GetJobStatusAsync(Guid jobId, CancellationToken ct);
}
public class AzureProvisioningService : IProvisioningService
{
private readonly IResourceManager _resourceManager;
private readonly IKubernetesManager _kubernetesManager;
private readonly ISecretsManager _secretsManager;
public async Task<ProvisionJob> ProvisionAsync(ProvisionRequest request, CancellationToken ct)
{
var job = new ProvisionJob { Id = Guid.NewGuid(), Status = ProvisionStatus.Running };
try
{
// 1. Azure resources
var resources = await _resourceManager.ProvisionAsync(request, ct);
// 2. Kubernetes deployment
await _kubernetesManager.DeployAsync(resources, request.Environment, ct);
// 3. Secrets
await _secretsManager.StoreAsync(resources.ConnectionStrings, ct);
job.Status = ProvisionStatus.Completed;
job.Outputs = resources;
}
catch (Exception ex)
{
job.Status = ProvisionStatus.Failed;
job.ErrorMessage = ex.Message;
}
return job;
}
}CLI для developers
# platform-cli — self-service CLI
platform-cli create-service my-api \
--owner team-platform \
--database postgresql \
--environment production
# Output:
# ✓ Azure resource group created: rg-my-api-prod
# ✓ PostgreSQL server provisioned: psql-my-api-prod
# ✓ Database created: my_api_prod
# ✓ Kubernetes deployment created
# ✓ Secrets stored in Azure Key Vault
# ✓ CI/CD pipeline configured
#
# Connection string:
# export DATABASE_URL="Host=psql-my-api-prod.postgres.database.azure.com;Database=my_api_prod"
#
# Dashboard: https://backstage.myorg.com/catalog/default/component/my-apiПрактика
Задание 1: Project template с pre-configured CI/CD
Требования:
- Golden path шаблон для .NET API
- Включает: Dockerfile, CI/CD, monitoring, logging, health checks
- Scaffolder template для Backstage
Задание 2: Self-service environment provisioning
Требования:
- Platform API для создания сервисов
- Provisioning Service с Azure integration
- CLI для developers
Задание 3: Platform health dashboard
Требования:
- Metrics: deployment frequency, lead time, change failure rate
- Dashboard в Grafana
- SLA tracking
Advanced Deployment Patterns
Содержание
- Database-first deployment — expand then contract
- Feature flag lifecycle
- A/B testing infrastructure
- Multi-region deployment
- Disaster recovery automation
- Практические задания
Expand-Contract migration для breaking schema change
Expand-Contract Pattern — 4 фазы
Фаза 1: EXPAND (100% back compatible)
- Добавить новую колонку/таблицу
- Начать писать в старую И новую
- Деплой приложения (читает старую, пишет обе)
Фаза 2: BACKFILL (data migration)
- Заполнить новую колонку данными из старой
- Отдельный job, НЕ требует deployment
Фаза 3: SWITCH (cutover)
- Приложение читает из новой колонки
- Деплой приложения (читает новую, пишет обе)
Фаза 4: CONTRACT (cleanup)
- Перестать писать в старую колонку
- Деплой приложения (читает новую, пишет новую)
- Удалить старую колонку (отдельный migration)Практическая реализация
// Migration 1: Expand — добавить новую колонку
public class Migration_001_Expand : Migration
{
protected override void Up(MigrationBuilder mb)
{
// Добавить новую колонку (nullable)
mb.AddColumn("Orders", "NewTotal", c => c.Decimal(18, 2));
// Приложение деплоится и начинает писать в обе колонки
}
protected override void Down(MigrationBuilder mb)
{
mb.DropColumn("Orders", "NewTotal");
}
}
// Migration 2: Backfill — заполнить данные
public class Migration_002_Backfill : Migration
{
protected override void Up(MigrationBuilder mb)
{
// Заполняем новую колонку данными из старой
// Важно: это может занять время для больших таблиц
mb.Sql(@"
UPDATE Orders
SET NewTotal = OldTotal
WHERE NewTotal IS NULL
");
// Для больших таблиц — batch update:
// WHILE EXISTS (SELECT 1 FROM Orders WHERE NewTotal IS NULL)
// BEGIN
// UPDATE TOP (10000) Orders SET NewTotal = OldTotal WHERE NewTotal IS NULL
// END
}
}
// Migration 3: Switch — переключить приложение
public class Migration_003_Switch : Migration
{
protected override void Up(MigrationBuilder mb)
{
// Приложение уже читает из NewTotal
// Этот migration только помечает переключение
}
}
// Migration 4: Contract — удалить старую колонку
public class Migration_004_Contract : Migration
{
protected override void Up(MigrationBuilder mb)
{
// Старая колонка больше не используется
mb.DropColumn("Orders", "OldTotal");
}
}
// Migration 5: Final cleanup
public class Migration_005_Finalize : Migration
{
protected override void Up(MigrationBuilder mb)
{
// Переименовываем новую колонку
mb.RenameColumn("Orders", "NewTotal", "Total");
}
}Dual-write в приложении
// Приложение пишет в обе колонки
public class OrderRepository : IOrderRepository
{
private readonly AppDbContext _context;
public async Task SaveAsync(Order order, CancellationToken ct = default)
{
// Старый формат (читается)
order.OldTotal = order.CalculateOldTotal();
// Новый формат (пишется)
order.NewTotal = order.CalculateNewTotal();
// При switch — читаем NewTotal
await _context.SaveChangesAsync(ct);
}
public async Task<Order?> GetByIdAsync(int id, CancellationToken ct = default)
{
var order = await _context.Orders.FindAsync(new object[] { id }, ct);
// До switch: читаем OldTotal
// После switch: читаем NewTotal
order.Total = order.NewTotal ?? order.OldTotal;
return order;
}
}Feature flag lifecycle
Полное управление флагами
// Feature Flag Service
public interface IFeatureFlagService
{
Task<bool> IsEnabledAsync(string flagName, CancellationToken ct = default);
Task<bool> IsEnabledAsync(string flagName, UserContext user, CancellationToken ct = default);
Task<IReadOnlyList<FeatureFlag>> GetAllAsync(CancellationToken ct = default);
}
public record UserContext(string UserId, IReadOnlyList<string> Groups, IReadOnlyList<string> Tags);
public record FeatureFlag(
string Name,
bool Enabled,
DateTimeOffset? CreatedAt,
DateTimeOffset? TargetReleaseDate,
string Owner,
IReadOnlyList<string> RolloutStrategy);
// Реализация с Azure App Configuration
public class AppConfigFeatureFlagService : IFeatureFlagService
{
private readonly AzureAppConfigurationClient _client;
private readonly ICacheService _cache;
public async Task<bool> IsEnabledAsync(string flagName, UserContext? user = null, CancellationToken ct = default)
{
// 1. Check cache
var cached = await _cache.GetAsync<bool>($"flag:{flagName}");
if (cached.HasValue) return cached.Value;
// 2. Check global flag
var globalEnabled = await _client.GetFeatureFlagAsync(flagName, ct);
if (!globalEnabled.Enabled) return false;
// 3. Check targeting rules
if (user != null && globalEnabled.Targeting != null)
{
return EvaluateTargeting(globalEnabled.Targeting, user);
}
return globalEnabled.Enabled;
}
private bool EvaluateTargeting(TargetingRules targeting, UserContext user)
{
// Group-based rollout
foreach (var rule in targeting.Groups)
{
if (user.Groups.Contains(rule.Group) && rule.Percentage > 0)
{
var hash = Hash(user.UserId + rule.Group);
if (hash % 100 < rule.Percentage) return true;
}
}
// User-based rollout
foreach (var userId in targeting.Users)
{
if (userId == user.UserId) return true;
}
return false;
}
}Feature flag lifecycle management
Phase 1: Creation (Day 0)
POST /api/platform/feature-flags
{
"name": "DarkMode",
"enabled": false,
"owner": "team-ui",
"targetReleaseDate": "2026-06-01"
}
Phase 2: Deploy with flag OFF (Day 1)
Код в production, но flag.enabled = false
Никто не видит новую функциональность
Phase 3: Internal testing (Day 7)
PATCH /api/platform/feature-flags/DarkMode
{
"enabled": true,
"targeting": {
"groups": ["@internal"],
"users": ["alice", "bob"]
}
}
Phase 4: Canary (Day 14)
PATCH /api/platform/feature-flags/DarkMode
{
"targeting": {
"groups": ["@internal"],
"percentage": 10
}
}
Phase 5: Gradual rollout (Day 21)
PATCH /api/platform/feature-flags/DarkMode
{ "targeting": { "percentage": 50 } }
Phase 6: Full release (Day 28)
PATCH /api/platform/feature-flags/DarkMode
{ "enabled": true, "targeting": null }
Phase 7: Cleanup (Day 60)
Удалить flag и старый код в следующем релизеA/B testing infrastructure
A/B testing с traffic splitting
# Istio VirtualService для A/B testing
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: checkout-ab-test
spec:
hosts:
- myapp.example.com
http:
- match:
- headers:
x-ab-test:
exact: "variant-a"
route:
- destination:
host: checkout
subset: variant-a
weight: 100
- match:
- headers:
x-ab-test:
exact: "variant-b"
route:
- destination:
host: checkout
subset: variant-b
weight: 100
- route:
- destination:
host: checkout
subset: control
weight: 70
- destination:
host: checkout
subset: variant-a
weight: 15
- destination:
host: checkout
subset: variant-b
weight: 15
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: checkout-variants
spec:
host: checkout
subsets:
- name: control
labels:
variant: control
version: v1.0
- name: variant-a
labels:
variant: test-a
version: v1.1-a
- name: variant-b
labels:
variant: test-b
version: v1.1-bA/B testing middleware в .NET
// Middleware для A/B testing
public class AbTestMiddleware
{
private readonly RequestDelegate _next;
private readonly ILogger<AbTestMiddleware> _logger;
public AbTestMiddleware(RequestDelegate next, ILogger<AbTestMiddleware> logger)
{
_next = next;
_logger = logger;
}
public async Task InvokeAsync(HttpContext context)
{
var userId = context.User.FindFirstValue(ClaimTypes.NameIdentifier)
?? context.Connection.RemoteIpAddress?.ToString()
?? Guid.NewGuid().ToString();
// Assign variant based on hash
var variant = GetVariant(userId);
context.Response.Headers.Append("x-ab-test", variant);
// Track experiment
_logger.LogInformation("User {UserId} assigned to variant {Variant}", userId, variant);
await _next(context);
}
private string GetVariant(string userId)
{
var hash = Math.Abs(userId.GetHashCode()) % 100;
if (hash < 15) return "variant-a";
if (hash < 30) return "variant-b";
return "control";
}
}Статистическая значимость
// Статистический анализ A/B теста
public class AbTestAnalyzer
{
public AbTestResult Analyze(
int controlConversions, int controlVisitors,
int variantConversions, int variantVisitors)
{
var controlRate = (double)controlConversions / controlVisitors;
var variantRate = (double)variantConversions / variantVisitors;
// Two-proportion z-test
var pooledRate = (controlConversions + variantConversions) / (double)(controlVisitors + variantVisitors);
var standardError = Math.Sqrt(
pooledRate * (1 - pooledRate) * (1.0 / controlVisitors + 1.0 / variantVisitors));
var zScore = (variantRate - controlRate) / standardError;
var pValue = 2 * (1 - NormalCDF(Math.Abs(zScore)));
var lift = (variantRate - controlRate) / controlRate * 100;
return new AbTestResult
{
ControlRate = controlRate,
VariantRate = variantRate,
Lift = lift,
ZScore = zScore,
PValue = pValue,
IsSignificant = pValue < 0.05,
Confidence = (1 - pValue) * 100
};
}
private double NormalCDF(double x)
{
// Approximation of the standard normal CDF
var a1 = 0.254829592;
var a2 = -0.284496736;
var a3 = 1.421413741;
var a4 = -1.453152027;
var a5 = 1.061405429;
var p = 0.3275911;
var sign = x < 0 ? -1 : 1;
x = Math.Abs(x) / Math.Sqrt(2);
var t = 1.0 / (1.0 + p * x);
var y = 1.0 - (((((a5 * t + a4) * t) + a3) * t + a2) * t + a1) * t * Math.Exp(-x * x);
return 0.5 * (1.0 + sign * y);
}
}
public record AbTestResult(
double ControlRate,
double VariantRate,
double Lift,
double ZScore,
double PValue,
bool IsSignificant,
double Confidence);Multi-region deployment
Active-Passive
Region 1 (Active) Region 2 (Passive)
┌─────────────────┐ ┌─────────────────┐
│ Kubernetes │ │ Kubernetes │
│ (3 replicas) │ │ (0 replicas) │
│ │ │ │
│ PostgreSQL │ │ PostgreSQL │
│ (Primary) │ │ (Read Replica) │
│ │ │ │
│ Redis │ │ Redis │
│ (Primary) │ │ (Sync) │
└─────────────────┘ └─────────────────┘
│ │
└──── DNS: Primary ─────────┘
Failover → SecondaryActive-Active
Region 1 Region 2
┌─────────────────┐ ┌─────────────────┐
│ Kubernetes │ │ Kubernetes │
│ (3 replicas) │◄──────►│ (3 replicas) │
│ │ Sync │ │
│ PostgreSQL │ │ PostgreSQL │
│ (Primary) │ │ (Primary) │
│ │ │ │
│ Redis │ │ Redis │
│ (Primary) │ │ (Primary) │
└─────────────────┘ └─────────────────┘
│ │
└──── DNS: GeoDNS ──────────┘
Route to nearestDisaster recovery automation
RTO и RPO
| Метрика | Описание | Active-Passive | Active-Active |
|---|---|---|---|
| RTO | Recovery Time Objective | 15-30 минут | < 1 минута |
| RPO | Recovery Point Objective | 5-15 минут | 0 (zero data loss) |
Automated failover
# Azure Traffic Manager для failover
# traffic-manager.json
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.Network/trafficManagerProfiles",
"apiVersion": "2022-04-01",
"name": "myapp-traffic-manager",
"location": "global",
"properties": {
"profileStatus": "Enabled",
"trafficRoutingMethod": "Priority",
"dnsConfig": {
"relativeName": "myapp",
"ttl": 30
},
"monitorConfig": {
"protocol": "HTTPS",
"port": 443,
"path": "/health/live",
"intervalInSeconds": 30,
"toleratedNumberOfFailures": 3,
"timeoutInSeconds": 10
},
"endpoints": [
{
"name": "primary-eastus",
"type": "Microsoft.Network/trafficManagerProfiles/azureEndpoints",
"properties": {
"targetResourceId": "[resourceId('Microsoft.Web/sites', 'myapp-eastus')]",
"priority": 1,
"weight": 1000,
"status": "Enabled"
}
},
{
"name": "secondary-westus",
"type": "Microsoft.Network/trafficManagerProfiles/azureEndpoints",
"properties": {
"targetResourceId": "[resourceId('Microsoft.Web/sites', 'myapp-westus')]",
"priority": 2,
"weight": 1000,
"status": "Enabled"
}
}
]
}
}
]
}Disaster recovery drill
# GitHub Actions — DR drill
name: Disaster Recovery Drill
on:
schedule:
- cron: '0 0 1 * *' # 1-го числа каждого месяца
workflow_dispatch:
jobs:
dr-drill:
runs-on: ubuntu-latest
environment: production
steps:
- name: Simulate failover
run: |
# Включить secondary region
az traffic-manager endpoint update \
--name secondary-westus \
--resource-group rg-infra \
--profile-name myapp-traffic-manager \
--type azureEndpoints \
--priority 1 \
--set targetResourceId=/subscriptions/.../myapp-westus
- name: Wait for DNS propagation
run: sleep 60
- name: Verify secondary is serving traffic
run: |
curl -f https://myapp.example.com/health/live
- name: Measure RTO
id: rto
run: |
START=$(date +%s)
# ... failover logic ...
END=$(date +%s)
RTO=$((END - START))
echo "RTO: ${RTO}s"
echo "RTO=$RTO" >> $GITHUB_OUTPUT
- name: Failover back to primary
if: always()
run: |
az traffic-manager endpoint update \
--name primary-eastus \
--resource-group rg-infra \
--profile-name myapp-traffic-manager \
--type azureEndpoints \
--priority 1 \
--set targetResourceId=/subscriptions/.../myapp-eastus
- name: Report results
if: always()
run: |
echo "DR Drill Results:" >> $GITHUB_STEP_SUMMARY
echo "- RTO: ${{ steps.rto.outputs.RTO }}s" >> $GITHUB_STEP_SUMMARY
echo "- Status: ${{ job.status }}" >> $GITHUB_STEP_SUMMARYПрактика
Задание 1: Expand-Contract migration для breaking schema change
Требования:
- Migration 1: Expand (новая колонка)
- Migration 2: Backfill (заполнение данных)
- Migration 3: Switch (переключение чтения)
- Migration 4: Contract (удаление старой колонки)
- Dual-write в репозитории
Задание 2: A/B testing framework с traffic splitting
Требования:
- Istio VirtualService для routing
- A/B testing middleware в .NET
- Статистический анализ результатов
- Автоматическое определение winner
Задание 3: Automated disaster recovery drill
Требования:
- Traffic Manager с priority-based failover
- Health check для detection
- Automated failover + failback
- RTO measurement и reporting
Cost Optimization
Содержание
- Cloud cost monitoring — allocation, tagging, anomaly detection
- Right-sizing — CPU/memory request tuning
- Spot/preemptible instances для stateless workloads
- Auto-scaling policies — minimizing idle resources
- Reserved instances vs pay-as-you-go
- Практические задания
Cloud cost monitoring
Cost allocation через tagging
Тегирование ресурсов для cost allocation:
┌─────────────────────────────────────────────┐
│ Resource │ Tags │
├─────────────────────────────────────────────┤
│ rg-myapp-prod │ team=platform │
│ env=production │
│ app=myapp │
│ cost-center=engineering │
├─────────────────────────────────────────────┤
│ psql-myapp-prod │ team=platform │
│ env=production │
│ app=myapp │
│ cost-center=engineering │
├─────────────────────────────────────────────┤
│ aks-myapp │ team=platform │
│ env=production │
│ app=myapp │
│ cost-center=engineering │
└─────────────────────────────────────────────┘Azure Cost Management
# Azure CLI — cost analysis
az costmanagement export create \
--resource-group rg-infra \
--name "monthly-cost-report" \
--type Scheduled \
--schedule-state Enabled \
--recurrence-type Monthly \
--recurrence-monthly 1 \
--format Csv \
--timeframe CurrentMonth \
--dataset-type Total \
--granularity Daily \
--grouping-type Tag \
--grouping-names "tags.cost-center" "tags.env" "tags.team"Cost anomaly detection
# Azure Anomaly Detection Rule
# cost-anomaly-monitor.json
{
"$schema": "https://schema.management.azure.com/schemas/2021-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.CostManagement/anomalyAlerts",
"apiVersion": "2023-08-01",
"name": "cost-anomaly-alert",
"properties": {
"description": "Alert when monthly cost exceeds baseline by 20%",
"threshold": 1.2,
"frequency": "Daily",
"recipients": ["ops-team@myorg.com"],
"filters": {
"tags": {
"env": ["production"]
}
}
}
}
]
}Right-sizing — CPU/memory tuning
VPA — Vertical Pod Autoscaler
# VPA Recommendation
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: myapp-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
updatePolicy:
updateMode: "Auto" # Auto | Init | Recreate
resourcePolicy:
containerPolicies:
- containerName: myapp
minAllowed:
cpu: 50m
memory: 64Mi
maxAllowed:
cpu: "1"
memory: 1Gi
controlledResources:
- cpu
- memoryАнализ потребления с Prometheus
# CPU usage per pod
container_cpu_usage_seconds_total{namespace="production", pod=~"myapp.*"}
# Memory usage per pod
container_memory_working_set_bytes{namespace="production", pod=~"myapp.*"}
# Request vs Usage
kube_pod_resource_request{resource="cpu", namespace="production"}
kube_pod_resource_limit{resource="cpu", namespace="production"}
# Recommendation: запросы на 40% выше фактического использования
# → уменьшить requestsRight-sizing пример
| Метрика | До оптимизации | После оптимизации | Экономия |
|---|---|---|---|
| CPU request | 500m | 150m | 70% |
| CPU limit | 1000m | 300m | 70% |
| Memory request | 512Mi | 256Mi | 50% |
| Memory limit | 1Gi | 512Mi | 50% |
| Cost/pod/мес | $45 | $18 | 60% |
Spot/Preemptible instances
Spot VMs для stateless workloads
# AKS Spot nodes
apiVersion: aks.azure.com/v1
kind: NodePool
metadata:
name: spot-pool
spec:
mode: User
nodeCount: 2
maxCount: 10
minCount: 0
vmSize: Standard_D2s_v3
type: VirtualMachineScaleSets
osType: Linux
nodeLabels:
workload-type: spot
nodeTaints:
- key: workload-type
value: spot
effect: NoSchedule
priority: Spot
evictionPolicy: Delete
spotMaxPrice: -1 # -1 = current price
scaleDownMode: DeleteTolerations для spot nodes
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
template:
spec:
tolerations:
- key: "workload-type"
operator: "Equal"
value: "spot"
effect: "NoSchedule"
containers:
- name: myapp
image: myapp:latestCost comparison
| Тип | Цена/час (D2s_v3) | Надёжность | Подходит для |
|---|---|---|---|
| On-Demand | $0.096 | 99.99% | Production (critical) |
| Reserved (1yr) | $0.058 | 99.99% | Production (stable) |
| Spot | $0.024-0.048 | ~95% | Dev, staging, batch |
| Spot (1yr RI discount) | $0.016-0.032 | ~95% | Non-critical workloads |
Auto-scaling policies
HPA с custom metrics
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 50
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75
behavior:
scaleUp:
stabilizationWindowSeconds: 30
policies:
- type: Percent
value: 50
periodSeconds: 60
- type: Pods
value: 5
periodSeconds: 60
selectPolicy: Max
scaleDown:
stabilizationWindowSeconds: 300 # 5 минут перед scale down
policies:
- type: Percent
value: 10
periodSeconds: 120KEDA для event-driven scaling
# KEDA ScaledObject — scale to zero
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: myapp-worker
spec:
scaleTargetRef:
name: myapp-worker
triggers:
- type: rabbitmq
metadata:
host: amqp://rabbitmq:5672
queueName: orders
consumptionPerPod: "10"
activationValue: "50"
cooldownPeriod: 300
minReplicaCount: 0 # Scale to zero!
maxReplicaCount: 50Azure App Service Auto-scale
{
"name": "myapp-autoscale",
"type": "Microsoft.Insights/autoscaleSettings",
"properties": {
"enabled": true,
"profiles": [
{
"name": "Default",
"capacity": {
"minimum": "2",
"maximum": "10",
"default": "2"
},
"rules": [
{
"metricTrigger": {
"metricName": "CpuPercentage",
"metricResourceUri": "[resourceId('Microsoft.Web/sites', 'myapp')]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "GreaterThan",
"threshold": 70
},
"scaleAction": {
"direction": "Increase",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
"metricTrigger": {
"metricName": "CpuPercentage",
"metricResourceUri": "[resourceId('Microsoft.Web/sites', 'myapp')]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT15M",
"timeAggregation": "Average",
"operator": "LessThan",
"threshold": 20
},
"scaleAction": {
"direction": "Decrease",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT15M"
}
}
]
}
],
"enabled": true,
"targetResourceUri": "[resourceId('Microsoft.Web/serverfarms', 'myapp-plan')]"
}
}Reserved instances vs pay-as-you-go
Сравнение моделей оплаты
| Модель | Скидка | Commitment | Гибкость |
|---|---|---|---|
| Pay-as-you-go | 0% | Нет | Максимальная |
| Reserved (1yr) | ~37% | 1 год | Средняя |
| Reserved (3yr) | ~54% | 3 года | Низкая |
| Spot | до 70% | Нет (preemptible) | Минимальная |
| Savings Plan | до 40% | $/час | Высокая |
Reserved Instance strategy
Production workloads (стабильные):
- 1yr Reserved: 40% экономия
- 3yr Reserved: 54% экономия
Development/Staging:
- Pay-as-you-go: высокая вариативность
Batch processing:
- Spot instances: до 70% экономия
Hybrid approach:
- 60% Reserved (baseline)
- 30% Pay-as-you-go (peak)
- 10% Spot (flexible)Практика
Задание 1: Cost alerting на budget threshold breach
Требования:
- Azure Budget с alert на 50%, 75%, 100%
- Notification через email + Teams
- Tag-based allocation
Задание 2: Оптимизация Kubernetes resource requests
Требования:
- VPA для анализа потребления
- Правильные requests/limits
- HPA для автомасштабирования
Задание 3: Cost report dashboard
Требования:
- Grafana dashboard с cost metrics
- Per-service breakdown
- Trend analysis
- Anomaly detection
Контрольная точка модуля 9
Содержание
- Обзор проекта
- Архитектура решения
- Требования
- Пошаговая реализация
- Критерии прохождения
- Чек-лист
Обзор проекта
Цель
Создать production-ready deployment pipeline для .NET platform, который покрывает все аспекты CI/CD, контейнеризации, оркестрации и мониторинга.
Сценарий
Вы — Senior DevOps Engineer в компании, которая разрабатывает микросервисную platform на .NET. Необходимо настроить полный цикл от commit до production с zero-downtime deployment.
Архитектура решения
┌─────────────────────────────────────────────────────────────┐
│ GitHub Repository │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ .NET API │ │ Worker │ │ Migration Tool │ │
│ │ Service │ │ Service │ │ │ │
│ └──────┬──────┘ └──────┬──────┘ └──────────┬──────────┘ │
│ │ │ │ │
│ ┌──────┴────────────────┴─────────────────────┴──────────┐ │
│ │ GitHub Actions Pipeline │ │
│ │ CI: restore → build → test → security scan │ │
│ │ CD: docker build → push → deploy → verify │ │
│ └─────────────────────────┬──────────────────────────────┘ │
└────────────────────────────┼────────────────────────────────┘
│
┌────────┴────────┐
│ GitHub CR │
│ (images) │
└────────┬────────┘
│
┌────────┴────────┐
│ Kubernetes │
│ Cluster │
│ │
│ ┌──────────┐ │
│ │ API Pod │ │
│ │ x3 │ │
│ └──────────┘ │
│ ┌──────────┐ │
│ │Worker Pod│ │
│ │ x2 │ │
│ └──────────┘ │
└─────────────────┘
│
┌────────┴────────┐
│ Azure Cloud │
│ PostgreSQL │
│ Redis │
│ Key Vault │
│ Monitoring │
└─────────────────┘Требования
Обязательные (Must Have)
| # | Требование | Описание |
|---|---|---|
| 1 | Multi-stage Dockerfile | Build → Publish → Runtime, < 200MB |
| 2 | docker-compose | API + PostgreSQL + Redis для local dev |
| 3 | Health checks | Liveness + Readiness + Startup |
| 4 | CI pipeline | restore → build → test → publish |
| 5 | NuGet caching | actions/cache |
| 6 | Parallel tests | Unit + Integration |
| 7 | Docker image push | GHCR с semver tags |
| 8 | Kubernetes manifests | Deployment + Service + HPA + PDB |
| 9 | Ingress + TLS | nginx + cert-manager |
| 10 | Blue-Green или Canary | Zero-downtime deployment |
| 11 | ConfigMap + Secret | Externalized configuration |
| 12 | Graceful shutdown | preStop hook + middleware |
| 13 | Non-root user | appuser в Docker |
| 14 | .dockerignore | Исключить bin/, obj/, tests/ |
Желательные (Nice to Have)
| # | Требование | Описание |
|---|---|---|
| 15 | Code coverage | > 80% с gate |
| 16 | Security scanning | CodeQL + dependency review |
| 17 | Semantic release | Автоматическое версионирование |
| 18 | Feature flags | Встроенная поддержка |
| 19 | Terraform | IaC для Azure resources |
| 20 | Helm chart | Packaging K8s manifests |
| 21 | Cost monitoring | VPA recommendations |
| 22 | GitOps | ArgoCD sync |
Пошаговая реализация
Шаг 1: .NET Application с health checks
// Program.cs
var builder = WebApplication.CreateBuilder(args);
// Health checks
builder.Services.AddHealthChecks()
.AddCheck("self", () => HealthCheckResult.Healthy())
.AddDbContextCheck<AppDbContext>("database");
// Serilog
builder.Host.UseSerilog((ctx, sp, cfg) => cfg
.ReadFrom.Configuration(ctx.Configuration)
.Enrich.FromLogContext()
.WriteTo.Console());
// OpenTelemetry
builder.Services.AddOpenTelemetry()
.WithTracing(t => t
.AddSource(builder.Environment.ApplicationName)
.AddAspNetCoreInstrumentation());
var app = builder.Build();
// Middleware
app.UseMiddleware<CorrelationIdMiddleware>();
app.UseMiddleware<GlobalExceptionMiddleware>();
// Health checks
app.MapHealthChecks("/health/live", new HealthCheckOptions
{
Predicate = check => check.Tags.Contains("live")
});
app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
Predicate = check => check.Tags.Contains("ready")
});
app.MapHealthChecks("/health", new HealthCheckOptions
{
Predicate = _ => true
});
app.MapGet("/", () => Results.Ok(new { service = "myapp-api", version = "1.0.0" }));
app.Run();Шаг 2: Multi-stage Dockerfile
# syntax=docker/dockerfile:1
# ---------- Build ----------
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /src
COPY *.sln .
COPY **/*.csproj ./
RUN dotnet restore
COPY . .
# Тесты
RUN dotnet test --no-restore --verbosity minimal
# Публикация
RUN dotnet publish -c Release -o /app/publish --no-restore --no-build
# ---------- Runtime ----------
FROM mcr.microsoft.com/dotnet/aspnet:9.0-chiseled AS runtime
WORKDIR /app
COPY --from=build --chown=appuser:appgroup /app/publish .
RUN groupadd -r appgroup && \
useradd -r -g appgroup -d /app -s /sbin/nologin appuser && \
chown -R appuser:appgroup /app
USER appuser
ENV ASPNETCORE_ENVIRONMENT=Production
ENV ASPNETCORE_FORWARDEDHEADERS_ENABLED=true
ENV DOTNET_SYSTEM_NET_HTTP_SOCKETSHTTPHANDLER_HTTP2UNENCRYPTEDSUPPORT=true
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8080/health/live || exit 1
ENTRYPOINT ["dotnet", "MyApp.dll"]Шаг 3: docker-compose
# docker-compose.yml
services:
api:
build:
context: .
dockerfile: Dockerfile
ports:
- "5000:8080"
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ConnectionStrings__DefaultConnection=Host=postgres;Port=5432;Database=myapp;Username=postgres;Password=postgres
- ConnectionStrings__Redis=redis:6379
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
networks:
- app-network
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: myapp
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- "5432:5432"
volumes:
- postgres-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 3s
retries: 5
networks:
- app-network
redis:
image: redis:7-alpine
ports:
- "6379:6379"
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 5
networks:
- app-network
volumes:
postgres-data:
networks:
app-network:
driver: bridgeШаг 4: GitHub Actions CI/CD
# .github/workflows/cicd.yml
name: CI/CD Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
env:
DOTNET_VERSION: '9.0.x'
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
ci:
name: CI
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_DB: testdb
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis:7-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Cache NuGet
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/*.csproj') }}
restore-keys: |
${{ runner.os }}-nuget-
- name: Restore
run: dotnet restore
- name: Build
run: dotnet build --no-restore -c Release
- name: Test
run: dotnet test --no-build -c Release --collect:"XPlat Code Coverage"
- name: Upload coverage
uses: codecov/codecov-action@v4
with:
fail_ci_if_error: true
docker:
name: Docker
needs: ci
runs-on: ubuntu-latest
if: github.event_name == 'push'
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha
type=semver,pattern={{version}}
type=ref,event=branch
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy:
name: Deploy to Kubernetes
needs: docker
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
environment: production
steps:
- uses: actions/checkout@v4
- name: Deploy to K8s
uses: azure/k8s-deploy@v4
with:
namespace: production
manifests: k8s/
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
- name: Verify deployment
run: |
kubectl rollout status deployment/myapp -n production --timeout=300s
kubectl get pods -n productionШаг 5: Kubernetes manifests
# k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: ghcr.io/myorg/myapp:latest
ports:
- containerPort: 8080
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "200m"
envFrom:
- configMapRef:
name: myapp-config
- secretRef:
name: myapp-secret
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 15"]
---
# k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: production
spec:
type: ClusterIP
selector:
app: myapp
ports:
- port: 80
targetPort: 8080
---
# k8s/hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
---
# k8s/pdb.yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: myapp-pdb
namespace: production
spec:
minAvailable: 2
selector:
matchLabels:
app: myapp
---
# k8s/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
namespace: production
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- myapp.example.com
secretName: myapp-tls
rules:
- host: myapp.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp
port:
number: 80Критерии прохождения
Production-ready pipeline
| Критерий | Цель | Как проверить |
|---|---|---|
| Deploy time | < 5 минут от commit to production | GitHub Actions timing |
| Zero-downtime | Verified через load test | kubectl rollout status + curl |
| Rollback time | < 2 минут на health check failure | kubectl rollout undo |
| Docker image size | < 200MB | docker images |
| IaC coverage | 100% через Terraform/Bicep | terraform plan |
| Health checks | Все 3 probe работают | kubectl get pods |
| Auto-scaling | HPA scales on CPU > 70% | kubectl top pods |
| PDB | Минимум 2 пода доступны | kubectl get pdb |
| Graceful shutdown | preStop hook + 503 middleware | kubectl delete pod |
Чек-лист
- [ ] Multi-stage Dockerfile < 200MB
- [ ] docker-compose с API + PostgreSQL + Redis
- [ ] Health checks: /health/live, /health/ready, /health
- [ ] CI: restore → build → test → publish
- [ ] NuGet caching
- [ ] Parallel test execution
- [ ] Docker push на GHCR
- [ ] Kubernetes: Deployment + Service + HPA + PDB
- [ ] Ingress с TLS
- [ ] Blue-Green или Canary deployment
- [ ] ConfigMap + Secret
- [ ] Graceful shutdown (preStop + middleware)
- [ ] Non-root user в Docker
- [ ] .dockerignore
- [ ] Code coverage > 80%
- [ ] Security scanning
- [ ] Semantic release
- [ ] Feature flags
- [ ] Terraform для Azure
- [ ] Helm chart
- [ ] Cost monitoring dashboard
Оценка
Прохождение модуля
| Критерий | Баллы |
|---|---|
| Все Must Have требования | 70 баллов |
| Code coverage > 80% | +5 |
| Security scanning | +5 |
| Semantic release | +5 |
| Feature flags | +5 |
| Terraform IaC | +5 |
| Helm chart | +5 |
| Максимум | 100 баллов |
Проходной балл: 80/100
Демонстрация
Для прохождения модуля необходимо продемонстрировать:
- Commit → Deploy pipeline - Показать GitHub Actions workflow - Показать deploy time < 5 минут - Показать zero-downtime deployment
- Health checks - Показать liveness, readiness, startup probes - Показать rollback на health check failure
- Auto-scaling - Нагрузить приложение - Показать HPA scale up - Показать HPA scale down
- Rollback - Deploy bad version - Показать автоматический rollback - Показать rollback time < 2 минут
- Docker image - Показать размер < 200MB - Показать multi-stage build - Показать non-root user