06Тестирование и Quality Assurance
Уровень 1: Foundation
Unit Testing
Обзор
Unit testing — фундамент quality assurance. Изолированная проверка минимальных единиц кода (методов, классов) без внешних зависимостей.
Фреймворки: xUnit vs NUnit vs MSTest
Сравнительная таблица
| Характеристика | xUnit | NUnit | MSTest |
|---|---|---|---|
| Создание | 2007 (Brad Wilson, Jim Newkirk) | 2002 (port Java JUnit) | 2007 (Microsoft) |
| Архитектура | Extensible, attributes + interfaces | Attribute-driven | Visual Studio integration |
| [Fact] / [Test] | [Fact] | [Test] | [TestMethod] |
| [Theory] | [Theory] + [InlineData] | [TestCase] | [DataTestMethod] + [DataRow] |
| Setup/Teardown | Constructor + IDisposable / IAsyncLifetime | [SetUp] / [TearDown] | [TestInitialize] / [TestCleanup] |
| One-time setup | IClassFixture<T> / ICollectionFixture<T> | [OneTimeSetUp] | [ClassInitialize] |
| Assertions | Assert.Equal, Assert.Throws | Assert.That(actual, Is.EqualTo(expected)) | Assert.AreEqual, Assert.ThrowsException |
| Parallel execution | Встроенная (по умолчанию) | Через [Parallelizable] | Ограниченная |
| Extensibility | IXunitTestCase, custom attributes | ITestAction | Custom attributes |
Почему xUnit — стандарт индустрии
// xUnit: каждый тест — изолированный экземпляр класса
public class CalculatorTests
{
// Constructor = SetUp (выполняется ПЕРЕД каждым тестом)
public CalculatorTests()
{
_calculator = new Calculator();
}
// IDisposable = TearDown (выполняется ПОСЛЕ каждого теста)
public void Dispose()
{
_calculator?.Dispose();
}
[Fact]
public void Add_TwoPositiveNumbers_ReturnsSum()
{
// xUnit использует точные assertions
var result = _calculator.Add(2, 3);
Assert.Equal(5, result);
}
}Ключевое отличие: xUnit создаёт НОВЫЙ экземпляр тестового класса для каждого [Fact]. Это гарантирует изоляцию — нет shared state между тестами.
AAA Pattern (Arrange, Act, Assert)
Структура
[Fact]
public void TransferMoney_SufficientFunds_DeductsFromSource()
{
// ARRANGE — подготовка данных, mocks, expected values
var sourceAccount = new Account { Balance = 1000 };
var targetAccount = new Account { Balance = 500 };
var transferService = new TransferService();
decimal expectedSourceBalance = 700; // 1000 - 300
// ACT — вызов тестируемого метода
var result = transferService.Transfer(sourceAccount, targetAccount, 300);
// ASSERT — проверка результата
Assert.True(result.Success);
Assert.Equal(expectedSourceBalance, sourceAccount.Balance);
Assert.Equal(800, targetAccount.Balance); // 500 + 300
}Почему AAA важен
- Читаемость — любой разработчик понимает структуру за 3 секунды
- Поддерживаемость — легко найти что сломалось
- Debugging — чёткое разделение подготовки, действия, проверки
Anti-patterns
// BAD: смешанный AAA
[Fact]
public void BadTest()
{
var account = new Account { Balance = 1000 };
var result = account.Withdraw(200);
account.Deposit(100); // ACT после ASSERT?!
Assert.Equal(800, account.Balance); // Непонятно что проверяем
Assert.True(result.Success);
}
// BAD: multiple asserts без clear intent
[Fact]
public void AlsoBad()
{
var result = _service.Process();
Assert.NotNull(result);
Assert.Equal(42, result.Id);
Assert.NotEmpty(result.Items);
Assert.True(result.IsValid);
Assert.Equal(DateTime.UtcNow.Date, result.CreatedAt.Date);
// ... ещё 10 assertions — какой из них fail?
}Test Doubles: Dummy, Fake, Stub, Spy, Mock
Классификация
| Тип | Назначение | Пример |
|---|---|---|
| Dummy | Placeholder, не используется в тесте | new DummyLogger() для конструктора |
| Fake | Working implementation, упрощённый | In-memory database, fake email service |
| Stub | Возвращает predetermined responses | stubUserService.GetUser(id) => adminUser |
| Spy | Записывает interactions для последующей проверки | spyLogger.LoggedMessages list |
| Mock | Ожидает specific calls, fail если не вызван | mockPaymentProcessor.Verify(p => p.Charge(...)) |
Примеры каждого типа
// DUMMY — просто placeholder
public class DummyEmailService : IEmailService
{
public Task SendAsync(string to, string subject, string body) => Task.CompletedTask;
}
// FAKE — рабочая но упрощённая реализация
public class InMemoryUserRepository : IUserRepository
{
private readonly Dictionary<int, User> _users = new();
private int _nextId = 1;
public async Task<User> GetByIdAsync(int id) =>
_users.TryGetValue(id, out var user) ? user : null;
public async Task<User> CreateAsync(User user)
{
user.Id = _nextId++;
_users[user.Id] = user;
return user;
}
}
// STUB — возвращает заранее заданные значения
public class StubDiscountCalculator : IDiscountCalculator
{
public decimal GetDiscount(User user) => 0.15m; // Всегда 15%
}
// SPY — записывает вызовы
public class SpyEmailService : IEmailService
{
public List<(string To, string Subject, string Body)> SentEmails { get; } = new();
public Task SendAsync(string to, string subject, string body)
{
SentEmails.Add((to, subject, body));
return Task.CompletedTask;
}
}
// Использование spy в тесте
[Fact]
public async Task RegisterUser_SendsWelcomeEmail()
{
var spyEmail = new SpyEmailService();
var service = new UserService(spyEmail);
await service.RegisterAsync("test@example.com", "John");
Assert.Single(spyEmail.SentEmails);
Assert.Equal("test@example.com", spyEmail.SentEmails[0].To);
Assert.Contains("Welcome", spyEmail.SentEmails[0].Subject);
}Mock Frameworks: Moq
Основы Moq
// Создание mock
var mockUserService = new Mock<IUserService>();
// Setup — как mock реагирует на вызовы
mockUserService
.Setup(s => s.GetByIdAsync(1))
.ReturnsAsync(new User { Id = 1, Name = "Alice" });
// Setup с любым аргументом
mockUserService
.Setup(s => s.GetByIdAsync(It.IsAny<int>()))
.ReturnsAsync(new User { Id = 99, Name = "Unknown" });
// Setup с условием на аргумент
mockUserService
.Setup(s => s.GetByIdAsync(It.Is<int>(id => id > 0)))
.ReturnsAsync(new User { Id = 1, Name = "Valid" });
mockUserService
.Setup(s => s.GetByIdAsync(It.Is<int>(id => id <= 0)))
.ThrowsAsync(new ArgumentException("Invalid ID"));
// Callback — выполнить код при вызове
var callCount = 0;
mockUserService
.Setup(s => s.GetByIdAsync(It.IsAny<int>()))
.Callback<int>(id => callCount++)
.ReturnsAsync(new User { Id = 1 });
// Verify — проверить что метод был вызван
mockUserService.Verify(s => s.GetByIdAsync(1), Times.Once);
mockUserService.Verify(s => s.GetByIdAsync(It.IsAny<int>()), Times.AtLeastOnce);
mockUserService.Verify(s => s.GetByIdAsync(999), Times.Never);
// Verify с таймаутом (для async)
mockUserService.Verify(s => s.GetByIdAsync(1), Times.Once, TimeSpan.FromSeconds(1));Mocking с 3+ dependencies
public interface IUserRepository { Task<User> GetByIdAsync(int id); }
public interface IEmailService { Task SendAsync(string to, string subject, string body); }
public interface IAuditLogger { Task LogAsync(string message); }
public interface IDiscountCalculator { decimal GetDiscount(User user); }
public class OrderService
{
private readonly IUserRepository _users;
private readonly IEmailService _email;
private readonly IAuditLogger _audit;
private readonly IDiscountCalculator _discount;
public OrderService(IUserRepository users, IEmailService email,
IAuditLogger audit, IDiscountCalculator discount)
{
_users = users; _email = email; _audit = audit; _discount = discount;
}
public async Task<OrderResult> CreateOrderAsync(int userId, decimal amount)
{
var user = await _users.GetByIdAsync(userId);
if (user == null) throw new UserNotFoundException(userId);
var discount = _discount.GetDiscount(user);
var finalAmount = amount * (1 - discount);
await _email.SendAsync(user.Email, "Order Created", $"Amount: {finalAmount}");
await _audit.LogAsync($"Order created for user {userId}: {finalAmount}");
return new OrderResult { UserId = userId, FinalAmount = finalAmount };
}
}
// Тест с 4 mocks
[Fact]
public async Task CreateOrder_ValidUser_AppliesDiscountAndSendsEmail()
{
// Arrange
var mockUsers = new Mock<IUserRepository>();
var mockEmail = new Mock<IEmailService>();
var mockAudit = new Mock<IAuditLogger>();
var mockDiscount = new Mock<IDiscountCalculator>();
var testUser = new User { Id = 1, Name = "John", Email = "john@test.com" };
mockUsers.Setup(u => u.GetByIdAsync(1)).ReturnsAsync(testUser);
mockDiscount.Setup(d => d.GetDiscount(testUser)).Returns(0.1m); // 10%
var service = new OrderService(
mockUsers.Object, mockEmail.Object, mockAudit.Object, mockDiscount.Object);
// Act
var result = await service.CreateOrderAsync(1, 1000);
// Assert
Assert.Equal(900, result.FinalAmount); // 1000 - 10%
mockEmail.Verify(e => e.SendAsync(
"john@test.com", "Order Created", "Amount: 900"), Times.Once);
mockAudit.Verify(a => a.LogAsync(
"Order created for user 1: 900"), Times.Once);
}Edge Cases Testing
Null, Empty, Boundary Values
public class StringProcessor
{
public static string Truncate(string input, int maxLength)
{
if (input == null) throw new ArgumentNullException(nameof(input));
if (maxLength < 0) throw new ArgumentOutOfRangeException(nameof(maxLength));
if (maxLength == 0) return string.Empty;
if (input.Length <= maxLength) return input;
return input[..maxLength];
}
}
public class StringProcessorTests
{
[Fact]
public void Truncate_NullInput_ThrowsArgumentNullException()
{
Assert.Throws<ArgumentNullException>(() => StringProcessor.Truncate(null, 5));
}
[Theory]
[InlineData(-1)]
[InlineData(int.MinValue)]
public void Truncate_NegativeMaxLength_ThrowsArgumentOutOfRangeException(int maxLength)
{
Assert.Throws<ArgumentOutOfRangeException>(() => StringProcessor.Truncate("test", maxLength));
}
[Theory]
[InlineData("hello", 5, "hello")] // Equal length
[InlineData("hello", 6, "hello")] // maxLength > length
[InlineData("hello", 3, "hel")] // Truncate
[InlineData("hello", 1, "h")] // Single char
[InlineData("hello", 0, "")] // Zero length
[InlineData("a", 1, "a")] // Single char input
[InlineData("", 5, "")] // Empty input
public void Truncate_BoundaryValues_CorrectResult(string input, int maxLength, string expected)
{
Assert.Equal(expected, StringProcessor.Truncate(input, maxLength));
}
}Test Fixtures
Class Fixture (shared state для тестов в одном классе)
// Fixture
public class DatabaseFixture : IDisposable
{
public string ConnectionString { get; }
public SqlConnection Connection { get; }
public DatabaseFixture()
{
ConnectionString = "Server=localhost;Database=TestDb;Trusted_Connection=True;";
Connection = new SqlConnection(ConnectionString);
Connection.Open();
// Seed test data
using var cmd = new SqlCommand("DELETE FROM Users; INSERT INTO Users (Name) VALUES ('Test')", Connection);
cmd.ExecuteNonQuery();
}
public void Dispose()
{
Connection.Close();
Connection.Dispose();
}
}
// Использование
public class UserTests : IClassFixture<DatabaseFixture>
{
private readonly DatabaseFixture _fixture;
public UserTests(DatabaseFixture fixture) => _fixture = fixture;
[Fact]
public void Test1()
{
Assert.NotNull(_fixture.Connection);
// Тесты используют shared connection
}
[Fact]
public void Test2()
{
// Тот же connection, тот же state
}
}Collection Fixture (shared state между НЕСКОЛЬКИМИ классами)
// 1. Define fixture
public class SharedDatabaseFixture : IAsyncLifetime
{
public PostgreSqlContainer Postgres { get; private set; }
public string ConnectionString => Postgres.GetConnectionString();
public async Task InitializeAsync()
{
Postgres = new PostgreSqlBuilder().Build();
await Postgres.StartAsync();
}
public async Task DisposeAsync()
{
await Postgres.StopAsync();
await Postgres.DisposeAsync();
}
}
// 2. Define collection
[CollectionDefinition("Database Collection")]
public class DatabaseCollection : ICollectionFixture<SharedDatabaseFixture> { }
// 3. Use in multiple test classes
[Collection("Database Collection")]
public class UserRepositoryTests
{
private readonly SharedDatabaseFixture _fixture;
public UserRepositoryTests(SharedDatabaseFixture fixture) => _fixture = fixture;
[Fact]
public async Task CanInsertAndRetrieve()
{
// Использует _fixture.ConnectionString
}
}
[Collection("Database Collection")]
public class OrderRepositoryTests
{
private readonly SharedDatabaseFixture _fixture;
public OrderRepositoryTests(SharedDatabaseFixture fixture) => _fixture = fixture;
[Fact]
public async Task CanCreateOrder()
{
// Тот же PostgreSQL instance
}
}Async Testing
[Fact]
public async Task AsyncMethod_Completes_ReturnsExpectedResult()
{
// Act
var result = await _service.ProcessAsync();
// Assert
Assert.NotNull(result);
}
// Testing async exceptions
[Fact]
public async Task AsyncMethod_InvalidInput_ThrowsException()
{
await Assert.ThrowsAsync<InvalidOperationException>(
async () => await _service.ProcessAsync(-1));
}
// Testing Task-returning methods без await
[Fact]
public void AsyncMethod_ReturnsTask_DoesNotThrow()
{
var task = _service.ProcessAsync();
Assert.NotNull(task);
Assert.False(task.IsFaulted);
}Практика
Упражнение 1: Service с 3+ dependencies
// Создайте OrderProcessingService с зависимостями:
// - IInventoryService
// - IPaymentGateway
// - INotificationService
// - IOrderRepository
// Напишите 5+ unit tests покрывающих happy path и error casesУпражнение 2: Edge cases
// Для метода CalculateShipping(decimal weight, string country, bool isExpress)
// Напишите tests для:
// - weight = 0, weight < 0, weight > max
// - country = null, empty, invalid, valid
// - isExpress = true/false combinationsУпражнение 3: Test fixture
// Создайте SharedRedisFixture для integration tests
// Реализуйте IAsyncLifetime
// Используйте в 3+ test classes через ICollectionFixtureIntegration Testing
Обзор
Integration testing проверяет взаимодействие компонентов: API endpoints, базы данных, message brokers, external services. В отличие от unit tests, использует реальные (или containerized) зависимости.
Testcontainers
Что это
Testcontainers — библиотека для создания ephemeral (временных) Docker containers в тестах. Каждый тест получает свежий instance базы данных, message broker, или другого сервиса.
Test Run
|
+-- Docker.Start(postgres:16) ~ 2-5 сек
| |
| +-- Container ready
| +-- Connection string available
| |
+-- Run tests against real PostgreSQL
|
+-- Docker.Stop() ~ 1 сек
|
+-- Container destroyed, no traceУстановка
dotnet add package Testcontainers
dotnet add package Testcontainers.PostgreSql
dotnet add package Testcontainers.MsSql
dotnet add package Testcontainers.Redis
dotnet add package Testcontainers.RabbitMq
dotnet add package Testcontainers.KafkaБазовое использование
using Testcontainers.PostgreSql;
public class PostgreSqlIntegrationTests : IAsyncLifetime
{
private readonly PostgreSqlContainer _postgresContainer;
public PostgreSqlIntegrationTests()
{
_postgresContainer = new PostgreSqlBuilder()
.WithDatabase("testdb")
.WithUsername("postgres")
.WithPassword("postgres")
.WithCleanUp(true) // Auto-remove container
.Build();
}
public async Task InitializeAsync()
{
await _postgresContainer.StartAsync();
// Container ready, connection string available
var connectionString = _postgresContainer.GetConnectionString();
// Server=localhost;Port=5432;Database=testdb;Username=postgres;Password=postgres
}
public async Task DisposeAsync()
{
await _postgresContainer.DisposeAsync();
}
[Fact]
public async Task CanConnectToRealPostgres()
{
var connectionString = _postgresContainer.GetConnectionString();
await using var connection = new NpgsqlConnection(connectionString);
await connection.OpenAsync();
Assert.Equal(ConnectionState.Open, connection.State);
}
[Fact]
public async Task CanExecuteQuery()
{
await using var connection = new NpgsqlConnection(_postgresContainer.GetConnectionString());
await connection.OpenAsync();
await using var command = new NpgsqlCommand("SELECT 1 + 1", connection);
var result = await command.ExecuteScalarAsync();
Assert.Equal(2L, result);
}
}EF Core + Testcontainers
public class EfCoreIntegrationTests : IAsyncLifetime
{
private readonly PostgreSqlContainer _postgres;
private readonly AppDbContext _dbContext;
public EfCoreIntegrationTests()
{
_postgres = new PostgreSqlBuilder().Build();
}
public async Task InitializeAsync()
{
await _postgres.StartAsync();
var options = new DbContextOptionsBuilder<AppDbContext>()
.UseNpgsql(_postgres.GetConnectionString())
.Options;
_dbContext = new AppDbContext(options);
await _dbContext.Database.MigrateAsync(); // Применить миграции
}
public async Task DisposeAsync()
{
await _dbContext.DisposeAsync();
await _postgres.DisposeAsync();
}
[Fact]
public async Task CanSaveAndRetrieveUser()
{
// Arrange
var user = new User { Name = "Alice", Email = "alice@test.com" };
// Act
_dbContext.Users.Add(user);
await _dbContext.SaveChangesAsync();
// Assert
var retrieved = await _dbContext.Users.FindAsync(user.Id);
Assert.NotNull(retrieved);
Assert.Equal("Alice", retrieved.Name);
Assert.Equal("alice@test.com", retrieved.Email);
}
[Fact]
public async Task MigrationCreatesTables()
{
var tables = await _dbContext.Database.GetDbConnection()
.QueryAsync<string>(
"SELECT tablename FROM pg_tables WHERE schemaname = 'public'");
Assert.Contains("Users", tables);
Assert.Contains("Orders", tables);
}
}Multiple Containers (PostgreSQL + Redis + RabbitMQ)
public class MultiContainerFixture : IAsyncLifetime
{
public PostgreSqlContainer Postgres { get; private set; }
public RedisContainer Redis { get; private set; }
public RabbitMqContainer RabbitMq { get; private set; }
public async Task InitializeAsync()
{
var tasks = new[]
{
StartPostgres(),
StartRedis(),
StartRabbitMq()
};
await Task.WhenAll(tasks);
}
private async Task StartPostgres()
{
Postgres = new PostgreSqlBuilder()
.WithDatabase("testdb")
.Build();
await Postgres.StartAsync();
}
private async Task StartRedis()
{
Redis = new RedisBuilder().Build();
await Redis.StartAsync();
}
private async Task StartRabbitMq()
{
RabbitMq = new RabbitMqBuilder()
.WithUsername("guest")
.WithPassword("guest")
.Build();
await RabbitMq.StartAsync();
}
public async Task DisposeAsync()
{
var tasks = new IDisposableAsync[] { Postgres, Redis, RabbitMq }
.Select(c => c.DisposeAsync());
await Task.WhenAll(tasks);
}
}Custom Container Image
// Custom PostgreSQL с pre-loaded data
var postgres = new PostgreSqlBuilder()
.WithImage("postgres:16-alpine") // Конкретная версия
.WithEnvironment("POSTGRES_INITDB_ARGS", "--encoding=UTF8")
.WithBindMount("./test-data/init.sql", "/docker-entrypoint-initdb.d/init.sql")
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilPortIsAvailable(5432)
.UntilMessageIsLogged("database system is ready"))
.Build();WebApplicationFactory
Что это
WebApplicationFactory<TEntryPoint> создаёт in-memory test server для ASP.NET Core приложения. Позволяет тестировать HTTP endpoints без запуска реального сервера.
Базовое использование
// Program.cs должен быть accessible
// <Project Sdk="Microsoft.NET.Sdk.Web"> для test project
public class ApiIntegrationTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly HttpClient _client;
public ApiIntegrationTests(WebApplicationFactory<Program> factory)
{
_client = factory.CreateClient();
}
[Fact]
public async Task GetUsers_ReturnsOkWithUsers()
{
// Act
var response = await _client.GetAsync("/api/users");
// Assert
response.EnsureSuccessStatusCode();
var content = await response.Content.ReadAsStringAsync();
var users = JsonSerializer.Deserialize<List<User>>(content,
new JsonSerializerOptions { PropertyNameCaseInsensitive = true });
Assert.NotNull(users);
}
[Fact]
public async Task CreateUser_PostValidUser_ReturnsCreated()
{
// Arrange
var newUser = new { Name = "John", Email = "john@test.com" };
var content = new StringContent(
JsonSerializer.Serialize(newUser),
Encoding.UTF8,
"application/json");
// Act
var response = await _client.PostAsync("/api/users", content);
// Assert
Assert.Equal(HttpStatusCode.Created, response.StatusCode);
var location = response.Headers.Location;
Assert.NotNull(location);
}
[Fact]
public async Task GetUser_InvalidId_ReturnsNotFound()
{
var response = await _client.GetAsync("/api/users/999999");
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
}
}Кастомизация WebApplicationFactory
public class CustomWebApplicationFactory : WebApplicationFactory<Program>
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
// Заменить реальный database на in-memory
var descriptor = services.SingleOrDefault(
d => d.ServiceType == typeof(DbContextOptions<AppDbContext>));
if (descriptor != null)
{
services.Remove(descriptor);
}
services.AddDbContext<AppDbContext>(options =>
options.UseInMemoryDatabase("TestDb"));
});
builder.UseEnvironment("Testing");
}
}
// Или с Testcontainers (real database!)
public class TestContainerFactory : WebApplicationFactory<Program>, IAsyncLifetime
{
private PostgreSqlContainer _postgres;
public async Task InitializeAsync()
{
_postgres = new PostgreSqlBuilder().Build();
await _postgres.StartAsync();
// Переопределить connection string
Environment.SetEnvironmentVariable(
"ConnectionStrings:DefaultConnection",
_postgres.GetConnectionString());
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Testing");
}
public async Task DisposeAsync()
{
await _postgres.DisposeAsync();
}
}Testing с Authentication
public class AuthenticatedClientTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly WebApplicationFactory<Program> _factory;
public AuthenticatedClientTests(WebApplicationFactory<Program> factory)
{
_factory = factory;
}
private HttpClient CreateAuthenticatedClient(string userId = "test-user")
{
return _factory.WithWebHostBuilder(builder =>
{
builder.ConfigureTestServices(services =>
{
// Mock authentication
services.AddAuthentication("Test")
.AddScheme<AuthenticationSchemeOptions, TestAuthHandler>(
"Test", options => options.UserId = userId);
});
}).CreateClient(new WebApplicationFactoryClientOptions
{
BaseAddress = new Uri("https://localhost")
});
}
[Fact]
public async Task GetProfile_Authenticated_ReturnsUserProfile()
{
var client = CreateAuthenticatedClient("user-123");
var response = await client.GetAsync("/api/profile");
response.EnsureSuccessStatusCode();
var profile = await response.Content.ReadFromJsonAsync<UserProfile>();
Assert.Equal("user-123", profile.Id);
}
}
public class TestAuthHandler : AuthenticationHandler<AuthenticationSchemeOptions>
{
public string UserId { get; set; }
public TestAuthHandler(IOptionsMonitor<AuthenticationSchemeOptions> options,
ILoggerFactory logger, UrlEncoder encoder, ISystemClock clock)
: base(options, logger, encoder, clock) { }
protected override Task<AuthenticateResult> HandleAuthenticateAsync()
{
var claims = new[] { new Claim(ClaimTypes.NameIdentifier, UserId) };
var identity = new ClaimsIdentity(claims, "Test");
var principal = new ClaimsPrincipal(identity);
var ticket = new AuthenticationTicket(principal, "Test");
return Task.FromResult(AuthenticateResult.Success(ticket));
}
}In-Memory vs Real Database
Сравнение
| Аспект | In-Memory | Real (Testcontainers) |
|---|---|---|
| Скорость | Быстро (~10ms setup) | Медленнее (~2-5 сек setup) |
| Точность | Низкая (не все SQL features) | Высокая (реальный engine) |
| SQL совместимость | Нет (EF Core LINQ only) | Полная |
| Migrations | Частично | Полностью |
| Constraints | Не проверяет | Проверяет (FK, unique, check) |
| Stored Procedures | Нет | Да |
| Рекомендация | Быстрые smoke tests | Production-like tests |
Когда использовать In-Memory
// Быстрые тесты бизнес логики без database-specific features
[Fact]
public async Task UserRegistration_ValidData_CreatesUser()
{
var options = new DbContextOptionsBuilder<AppDbContext>()
.UseInMemoryDatabase(Guid.NewGuid().ToString()) // Unique per test
.Options;
await using var context = new AppDbContext(options);
var service = new UserService(context);
await service.RegisterAsync("test@test.com", "password");
var user = await context.Users.FirstOrDefaultAsync();
Assert.NotNull(user);
Assert.Equal("test@test.com", user.Email);
}Когда использовать Real Database
// Тесты с raw SQL, stored procedures, specific SQL features
[Fact]
public async Task UserRepository_FullTextSearch_ReturnsResults()
{
// In-Memory НЕ поддерживает Full-Text Search
// Testcontainers с PostgreSQL — единственный вариант
await using var connection = new NpgsqlConnection(_postgres.GetConnectionString());
await connection.OpenAsync();
// Seed data
await connection.ExecuteAsync(@"
INSERT INTO products (name, description) VALUES
('Wireless Mouse', 'Ergonomic wireless mouse with USB receiver'),
('Mechanical Keyboard', 'RGB mechanical keyboard with Cherry MX switches')
");
// Full-text search query
var results = await connection.QueryAsync<Product>(@"
SELECT * FROM products
WHERE to_tsvector('english', name || ' ' || description)
@@ plainto_tsquery('english', 'wireless mouse')
");
Assert.Single(results);
Assert.Contains("Wireless Mouse", results.First().Name);
}Test Database Seeding и Cleanup
Seeding Strategies
// 1. Seed через EF Core
public static async Task SeedTestData(AppDbContext context)
{
if (await context.Users.AnyAsync()) return; // Already seeded
var users = new[]
{
new User { Name = "Alice", Email = "alice@test.com", Role = "Admin" },
new User { Name = "Bob", Email = "bob@test.com", Role = "User" },
new User { Name = "Charlie", Email = "charlie@test.com", Role = "User" }
};
await context.Users.AddRangeAsync(users);
await context.SaveChangesAsync();
}
// 2. Seed через SQL scripts
public static async Task SeedWithSql(AppDbContext context)
{
var sql = await File.ReadAllTextAsync("test-data/seed.sql");
await context.Database.ExecuteSqlRawAsync(sql);
}
// 3. Seed через Bogus (fake data generator)
public static List<User> GenerateFakeUsers(int count = 100)
{
var faker = new Faker<User>()
.RuleFor(u => u.Name, f => f.Name.FullName())
.RuleFor(u => u.Email, f => f.Internet.Email())
.RuleFor(u => u.Age, f => f.Random.Number(18, 80));
return faker.Generate(count);
}Cleanup Strategies
// 1. Transaction rollback (быстрый, но не для всех сценариев)
public class TransactionalTests : IAsyncLifetime
{
private IDbContextTransaction _transaction;
public async Task InitializeAsync()
{
_transaction = await _dbContext.Database.BeginTransactionAsync();
}
public async Task DisposeAsync()
{
await _transaction.RollbackAsync(); // Все изменения отменены
await _dbContext.DisposeAsync();
}
[Fact]
public void Test1() { /* Изменения будут rolled back */ }
}
// 2. Delete all после каждого теста
public async Task CleanupDatabase()
{
await _dbContext.Database.ExecuteSqlRawAsync(@"
DELETE FROM OrderItems;
DELETE FROM Orders;
DELETE FROM Users;
");
}
// 3. Per-test database (Testcontainers — каждый тест в fresh container)
// Самый надёжный, но самый медленныйПрактика
Упражнение 1: Integration test с Testcontainers + PostgreSQL
// 1. Создайте PostgreSqlContainer
// 2. Примените EF Core migrations
// 3. Seed test data
// 4. Напишите тесты для UserRepository
// 5. Проверьте что данные сохраняются и извлекаются корректноУпражнение 2: API integration test через WebApplicationFactory
// 1. Создайте WebApplicationFactory<Program>
// 2. Замените database на Testcontainers PostgreSQL
// 3. Протестируйте CRUD endpoints:
// - GET /api/users — возвращает список
// - POST /api/users — создаёт пользователя
// - GET /api/users/{id} — возвращает пользователя
// - PUT /api/users/{id} — обновляет
// - DELETE /api/users/{id} — удаляет
// 4. Проверьте status codes и response bodiesУпражнение 3: EF Core migration correctness test
// 1. Создайте fresh Testcontainers PostgreSQL
// 2. Примените все migrations
// 3. Проверьте что все таблицы созданы
// 4. Проверьте что constraints работают (FK, unique, not null)
// 5. Проверьте что seed data загружаетсяTDD (Test-Driven Development)
Обзор
TDD — методология разработки, где тесты пишутся ДО реализации. Цикл Red-Green-Refactor обеспечивает дизайн кода через тесты, а не наоборот.
Red-Green-Refactor Cycle
Фазы
RED GREEN REFACTOR
---- ------ --------
1. Написать тест 3. Написать код 5. Улучшить код
FAILING чтобы тест сохраняя
PASSING passing tests
| | |
v v v
[FAIL] Test fails [PASS] Test passes [PASS] Tests still pass
because code with minimal with cleaner design
doesn't exist implementation
or is wrongДетальный цикл
RED PHASE
---------
1. Понять requirement
2. Написать ТЕСТ для одного аспекта requirement
3. Запустить тест — он FAIL (код не написан или некорректен)
4. Убедиться что failure message понятный
GREEN PHASE
-----------
5. Написать MINIMAL код чтобы тест прошёл
6. НЕ оптимизировать, НЕ улучшать дизайн
7. Запустить тест — он PASS
8. Если не pass — исправить
REFACTOR PHASE
--------------
9. Улучшить код: extract method, rename, remove duplication
10. Запустить ВСЕ тесты — все должны pass
11. Если какой-то fail — revert refactoring, fix, retryПример: FizzBuzz через TDD
Шаг 1: RED — число 1 возвращает "1"[Fact]
public void Convert_1_Returns1()
{
var fizzBuzz = new FizzBuzz();
Assert.Equal("1", fizzBuzz.Convert(1));
}
// FAIL: FizzBuzz.Convert не существуетШаг 2: GREEN — минимальная реализацияpublic class FizzBuzz
{
public string Convert(int number) => "1";
}
// PASS: тест проходит (хотя реализация hardcoded)Шаг 3: RED — число 2 возвращает "2"[Fact]
public void Convert_2_Returns2()
{
var fizzBuzz = new FizzBuzz();
Assert.Equal("2", fizzBuzz.Convert(2));
}
// FAIL: возвращает "1" вместо "2"Шаг 4: GREEN — обобщитьpublic class FizzBuzz
{
public string Convert(int number) => number.ToString();
}
// PASS: оба теста проходятШаг 5: RED — число 3 возвращает "Fizz"[Fact]
public void Convert_3_ReturnsFizz()
{
var fizzBuzz = new FizzBuzz();
Assert.Equal("Fizz", fizzBuzz.Convert(3));
}
// FAIL: возвращает "3"Шаг 6: GREENpublic class FizzBuzz
{
public string Convert(int number)
{
if (number % 3 == 0) return "Fizz";
return number.ToString();
}
}
// PASS: все 3 тестаШаг 7: RED — число 5 возвращает "Buzz"[Fact]
public void Convert_5_ReturnsBuzz()
{
var fizzBuzz = new FizzBuzz();
Assert.Equal("Buzz", fizzBuzz.Convert(5));
}
// FAIL: возвращает "5"Шаг 8: GREENpublic class FizzBuzz
{
public string Convert(int number)
{
if (number % 3 == 0) return "Fizz";
if (number % 5 == 0) return "Buzz";
return number.ToString();
}
}
// PASS: все 4 тестаШаг 9: RED — число 15 возвращает "FizzBuzz"[Fact]
public void Convert_15_ReturnsFizzBuzz()
{
var fizzBuzz = new FizzBuzz();
Assert.Equal("FizzBuzz", fizzBuzz.Convert(15));
}
// FAIL: возвращает "Fizz" (первое условие срабатывает)Шаг 10: GREENpublic class FizzBuzz
{
public string Convert(int number)
{
if (number % 15 == 0) return "FizzBuzz";
if (number % 3 == 0) return "Fizz";
if (number % 5 == 0) return "Buzz";
return number.ToString();
}
}
// PASS: все 5 тестовШаг 11: REFACTORpublic class FizzBuzz
{
public string Convert(int number)
{
var result = string.Empty;
if (number % 3 == 0) result += "Fizz";
if (number % 5 == 0) result += "Buzz";
return string.IsNullOrEmpty(result) ? number.ToString() : result;
}
}
// PASS: все тесты всё ещё проходят, код чищеTriangulation
Что это
Triangulation — техника в TDD где вы добавляете multiple test cases чтобы "вынудить" код стать generic. Один test case можно захардкодить, два — сложнее, три — нужна общая логика.
Пример: Max функция
// Test 1: можно захардкодить
[Fact]
public void Max_2And3_Returns3()
{
Assert.Equal(3, MathEx.Max(2, 3));
}
// Реализация может быть:
public static int Max(int a, int b) => 3; // Хардкод!
// Test 2: хардкод не работает
[Fact]
public void Max_5And1_Returns5()
{
Assert.Equal(5, MathEx.Max(5, 1));
}
// Теперь нужна логика:
public static int Max(int a, int b) => a > b ? a : b;
// Test 3: triangulation подтверждает
[Fact]
public void Max_EqualValues_ReturnsValue()
{
Assert.Equal(5, MathEx.Max(5, 5));
}
// Test 4: negative numbers
[Fact]
public void Max_NegativeNumbers_ReturnsGreater()
{
Assert.Equal(-3, MathEx.Max(-5, -3));
}Когда использовать triangulation
| Ситуация | Triangulation нужна? |
|---|---|
| Простая логика (if/else) | 2-3 теста достаточно |
| Алгоритмы (сортировка, поиск) | 5+ тестов, включая edge cases |
| Business rules | Тест per rule |
| String parsing | Empty, null, valid, invalid, boundary |
Characterization Tests
Что это
Characterization tests — тесты для legacy кода без документации. Они НЕ проверяют что код правилен, они фиксируют что код ДЕЛАЕТ. Это "snapshot" текущего поведения.
Когда использовать
- Legacy code без тестов
- Undocumented behavior
- Перед refactoring legacy кода
- При migration на новую версию библиотеки
Процесс
1. Найти legacy method без тестов
2. Написать тест который вызывает method с конкретными inputs
3. Записать actual output (даже если он странный)
4. Сделать output expected value
5. Повторить для множества inputs
6. Теперь можно refactoring — tests catch behavior changesПример
// Legacy method без документации
public class LegacyPricingEngine
{
public decimal CalculatePrice(decimal basePrice, string customerType, int quantity)
{
decimal price = basePrice;
if (customerType == "VIP") price *= 0.9m;
if (quantity > 10) price *= 0.95m;
if (quantity > 50) price *= 0.9m;
if (customerType == "VIP" && quantity > 100) price -= 10;
if (price < 0) price = 0;
return Math.Round(price, 2, MidpointRounding.AwayFromZero);
}
}
// Characterization tests
public class LegacyPricingEngineCharacterizationTests
{
private readonly LegacyPricingEngine _engine = new();
[Theory]
[InlineData(100, "Regular", 1, 100)]
[InlineData(100, "VIP", 1, 90)]
[InlineData(100, "Regular", 11, 95)]
[InlineData(100, "Regular", 51, 90)]
[InlineData(100, "VIP", 11, 85.5)] // 100 * 0.9 * 0.95
[InlineData(100, "VIP", 101, 75.5)] // 100 * 0.9 * 0.9 - 10
[InlineData(5, "VIP", 101, 0)] // 5 * 0.9 * 0.9 - 10 = -5.95 -> 0
[InlineData(100, "VIP", 50, 90)] // boundary: quantity = 50
[InlineData(100, "VIP", 51, 81)] // 100 * 0.9 * 0.9
public void Characterize_CalculationMatchesCurrentBehavior(
decimal basePrice, string customerType, int quantity, decimal expected)
{
// Этот тест ДОКУМЕНТИРУЕТ текущее поведение
// Если behavior изменится — тест fail и мы узнаем
var result = _engine.CalculatePrice(basePrice, customerType, quantity);
Assert.Equal(expected, result);
}
}Автоматическая генерация characterization tests
// Инструмент: ApprovalTests или ручная генерация
[Fact]
public void GenerateCharacterizationTests()
{
var engine = new LegacyPricingEngine();
var testCases = new[]
{
(100m, "Regular", 1),
(100m, "VIP", 1),
(100m, "Regular", 11),
(100m, "Regular", 51),
(100m, "VIP", 11),
(100m, "VIP", 101),
(5m, "VIP", 101),
(100m, "VIP", 50),
(100m, "VIP", 51),
(0m, "Regular", 0),
(1000m, "VIP", 200),
};
foreach (var (price, type, qty) in testCases)
{
var result = engine.CalculatePrice(price, type, qty);
Console.WriteLine($"[InlineData({price}, \"{type}\", {qty}, {result})]");
}
}When NOT to Use TDD
Trade-offs
| Ситуация | TDD полезен? | Почему |
|---|---|---|
| UI/Frontend | Нет | Дизайн меняется быстро, тесты fragile |
| Exploratory prototyping | Нет | Сначала нужно понять проблему |
| Legacy code без тестов | Частично | Characterization tests сначала |
| Simple CRUD | Нет | Boilerplate, мало business logic |
| Complex algorithms | Да | Тесты помогают понять requirements |
| Business rules | Да | Тесты документируют правила |
| Public APIs | Да | Тесты = contract documentation |
| Security-critical code | Да | Тесты catch edge cases |
Альтернативы TDD
1. Test-After Development
- Написать код
- Потом написать тесты
- Риск: код может быть untestable
2. Test-First (не full TDD)
- Написать тесты перед кодом
- Без strict Red-Green-Refactor
- Быстрее, но менее disciplined
3. Property-Based Testing
- Тестировать properties/invariants
- Не конкретные inputs/outputs
- Хорошо для алгоритмов
4. Integration-First
- Тестировать через API/UI
- Unit tests потом
- Хорошо для web appsПрактика
Упражнение 1: Алгоритм сортировки через TDD
// Реализуйте Bubble Sort или Quick Sort через чистый TDD:
//
// Шаг 1: Sort empty array -> empty array
// Шаг 2: Sort single element -> same element
// Шаг 3: Sort two elements (sorted) -> same
// Шаг 4: Sort two elements (reversed) -> sorted
// Шаг 5: Sort multiple elements
// Шаг 6: Sort with duplicates
// Шаг 7: Sort negative numbers
// Шаг 8: REFACTOR — extract helper methods
//
// Каждый шаг: RED -> GREEN -> REFACTORУпражнение 2: Characterization test для legacy method
// Найдите или создайте legacy method с complex logic
// Напишите 10+ characterization tests
// Задокументируйте все edge cases
// Теперь refactoring — tests должны passУпражнение 3: Domain logic module через TDD
// Создайте ShoppingCart module через TDD:
//
// Rules:
// - Добавить товар
// - Удалить товар
// - Применить скидку (10% если сумма > 1000)
// - Применить промокод
// - Рассчитать итог
// - Не более 10 items
// - Не более 5 одинаковых items
//
// Каждый rule: сначала тест, потом implementationTest Pyramid и Strategy
Обзор
Test Pyramid — стратегия распределения тестов по уровням абстракции. Правильное соотношение обеспечивает быструю feedback loop и надёжное покрытие.
Test Pyramid: Unit → Integration → E2E
Правильное соотношение (70-20-10)
/\
/ \
/ E2E \ 10% — End-to-End тесты
/-------\ Медленные, дорогие, fragile
/ \
/ Integration \ 20% — Integration тесты
/---------------\ Средние по скорости и стоимости
/ \
/ Unit \ 70% — Unit тесты
/---------------------\ Быстрые, дешёвые, надёжныеПочему 70-20-10
| Уровень | Скорость | Стоимость | Надёжность | Покрытие |
|---|---|---|---|---|
| Unit (70%) | ~100ms | Дешёвые | Высокая | Code logic |
| Integration (20%) | ~1-5s | Средние | Средняя | Component interaction |
| E2E (10%) | ~10-60s | Дорогие | Низкая | Full user journey |
Anti-pattern: Ice Cream Cone
E2E ████████████████████ 60% — слишком много!
Int ████ 10%
Unit ██ 30% — слишком мало!
Проблемы:
- Тесты run 30+ минут
- Flaky tests из-за UI/browser
- Hard to debug failures
- Expensive CI/CDМетрики Test Pyramid
// Анализ распределения тестов в проекте
public class TestPyramidAnalyzer
{
public TestPyramidMetrics Analyze(Assembly testAssembly)
{
var tests = testAssembly.GetTypes()
.SelectMany(t => t.GetMethods())
.Where(m => m.GetCustomAttributes(typeof(FactAttribute), true).Any()
|| m.GetCustomAttributes(typeof(TheoryAttribute), true).Any())
.ToList();
int unit = 0, integration = 0, e2e = 0;
foreach (var test in tests)
{
var declaringType = test.DeclaringType;
if (declaringType.Name.Contains("E2E") ||
declaringType.Name.Contains("Ui"))
e2e++;
else if (declaringType.Name.Contains("Integration"))
integration++;
else
unit++;
}
return new TestPyramidMetrics
{
Total = tests.Count,
Unit = unit,
Integration = integration,
E2E = e2e,
UnitPercent = (double)unit / tests.Count * 100,
IntegrationPercent = (double)integration / tests.Count * 100,
E2EPercent = (double)e2e / tests.Count * 100
};
}
}
public record TestPyramidMetrics
{
public int Total { get; init; }
public int Unit { get; init; }
public int Integration { get; init; }
public int E2E { get; init; }
public double UnitPercent { get; init; }
public double IntegrationPercent { get; init; }
public double E2EPercent { get; init; }
public bool IsBalanced =>
UnitPercent >= 60 &&
IntegrationPercent >= 15 &&
E2EPercent <= 15;
}Contract Testing — Pact
Что это
Contract testing проверяет что producer и consumer сервисы совместимы на уровне API contracts. Consumer-driven contracts: consumer определяет expectations, producer verifies.
Pact.NET установка
dotnet add package PactNetConsumer test (определяет contract)
public class UserApiConsumerTests : IClassFixture<PactBuilderFixture>
{
private readonly PactBuilder _pactBuilder;
public UserApiConsumerTests(PactBuilderFixture fixture)
{
_pactBuilder = fixture.PactBuilder;
}
[Fact]
public async Task GetUser_Expectations()
{
// Arrange: Define expected interaction
_pactBuilder
.UponReceiving("A GET request for a user")
.Given("a user exists")
.WithRequest(HttpMethod.Get, "/api/users/1")
.WillRespond()
.WithStatus(HttpStatusCode.OK)
.WithHeader("Content-Type", "application/json")
.WithBody(new
{
id = PactBuilder.MatchInteger(1),
name = PactBuilder.MatchRegex("^[A-Za-z]+$", "Alice"),
email = PactBuilder.MatchRegex("^[\\w.+-]+@[\\w-]+\\.[\\w.]+$", "alice@test.com")
});
await _pactBuilder.Build();
// Act: Call the mock provider
var mockProviderUrl = _pactBuilder.MockProviderUri;
using var client = new HttpClient { BaseAddress = mockProviderUrl };
var response = await client.GetAsync("/api/users/1");
// Assert: Verify response matches expectations
response.EnsureSuccessStatusCode();
var user = await response.Content.ReadFromJsonAsync<User>();
Assert.Equal(1, user.Id);
Assert.NotNull(user.Name);
Assert.NotNull(user.Email);
}
}Producer test (verifies contract)
public class UserApiProducerTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly WebApplicationFactory<Program> _factory;
public UserApiProducerTests(WebApplicationFactory<Program> factory)
{
_factory = factory;
}
[Fact]
public async Task VerifyPactContract()
{
var config = new PactVerifierConfig
{
PublishVerificationResults = true,
ProviderVersion = "1.0.0"
};
var pactUri = Path.Combine(Directory.GetCurrentDirectory(),
"pacts", "Consumer-Provider.json");
IPactVerifier verifier = new PactVerifier(config);
verifier
.Provider("Provider")
.WithHttpEndpoint(new Uri("http://localhost"))
.WithFileSource(new FileInfo(pactUri))
.Verify();
}
}Pact Broker (centralized contract registry)
// Producer публикует verification results в Pact Broker
var config = new PactVerifierConfig
{
PublishVerificationResults = true,
ProviderVersion = "1.0.0",
PactBrokerUri = new Uri("https://pact-broker.example.com"),
PactBrokerUsername = "user",
PactBrokerPassword = "password"
};
// Consumer может проверить can-i-deploy
// pact-broker can-i-deploy --pacticipant Consumer --version 1.0.0Property-Based Testing — FsCheck
Что это
Property-based testing генерирует hundreds of random inputs и проверяет invariants (properties) которые должны всегда hold. В отличие от example-based testing, не нужно думать о конкретных test cases.
Установка
dotnet add package FsCheck.XunitПримеры
using FsCheck;
using FsCheck.Xunit;
public class PropertyBasedTests
{
// Property: reversing a list twice returns the original list
[Property]
public Property ReverseTwice_ReturnsOriginal(List<int> list)
{
var reversed = list.Reverse().Reverse().ToList();
return (list.SequenceEqual(reversed)).ToProperty();
}
// Property: sorting is idempotent
[Property]
public Property SortIsIdempotent(List<int> list)
{
var sorted1 = list.OrderBy(x => x).ToList();
var sorted2 = sorted1.OrderBy(x => x).ToList();
return sorted1.SequenceEqual(sorted2).ToProperty();
}
// Property: max element is always >= any element
[Property]
public Property MaxIsGreatestElement(NonEmptyList<int> list)
{
var max = list.Get.Max();
return list.Get.All(x => x <= max).ToProperty();
}
// Property: string concatenation length
[Property]
public Property ConcatLength(string a, string b)
{
return (a + b).Length == a.Length + b.Length;
}
// Property: binary search correctness
[Property]
public Property BinarySearchFindsElement(NonEmptyList<int> sortedList, int searchValue)
{
var sorted = sortedList.Get.OrderBy(x => x).ToList();
sorted.Add(searchValue);
sorted.Sort();
var index = sorted.BinarySearch(searchValue);
return (index >= 0 && sorted[index] == searchValue).ToProperty();
}
}Custom generators
public class CustomGenerators
{
// Генератор valid email
public static Arb<string> EmailGenerator()
{
var localPart = Gen.Elements("user", "test", "admin")
.Select(s => s + Gen.Elements(".", "_") + Gen.Alpha(3));
var domain = Gen.Elements("example.com", "test.org", "mail.net");
return Arb.From(localPart.Zip(domain, (l, d) => $"{l}@{d}"));
}
[Property(Arbitrary = new[] { typeof(CustomGenerators) })]
public Property ValidEmail_ContainsAtSymbol(string email)
{
return email.Contains('@').ToProperty();
}
}Когда использовать property-based testing
| Сценарий | Подходит? | Пример property |
|---|---|---|
| Алгоритмы сортировки | Да | sorted[i] <= sorted[i+1] |
| Парсинг/сериализация | Да | parse(serialize(x)) == x |
| Математические функции | Да | f(f(x)) == x |
| Business rules | Частично | discount <= originalPrice |
| UI interactions | Нет | Слишком complex |
Mutation Testing — Stryker.NET
Что это
Mutation testing проверяет QUALITY тестов (не code coverage). Мутанты — небольшие изменения в коде (изменить > на <, + на -, удалить condition). Если тесты detect mutation — мутант killed. Если нет — mutation survived (тесты неэффективны).
Установка
dotnet tool install -g dotnet-stryker
# или в проект
dotnet add package Stryker.NETЗапуск
cd YourProject.Tests
dotnet strykerКак работает
Original code:
if (age >= 18) return "adult";
Mutants:
1. if (age > 18) return "adult"; // >= changed to >
2. if (age <= 18) return "adult"; // >= negated
3. if (true) return "adult"; // condition removed
4. return "adult"; // entire if removed
Если тесты detect ALL mutants — mutation score 100%
Если тесты detect 8/10 mutants — mutation score 80%Stryker конфигурация
// stryker-config.json
{
"stryker-config": {
"project": "YourProject.csproj",
"thresholds": {
"high": 80,
"low": 60,
"break": 40
},
"reporters": [
"progress",
"html",
"markdown"
],
"mutate": [
"**/Services/*.cs",
"!**/Migrations/*.cs"
],
"language-version": "preview"
}
}dotnet stryker --config stryker-config.jsonMutation Score интерпретация
| Score | Статус | Действие |
|---|---|---|
| 90-100% | Excellent | Тесты очень эффективны |
| 70-89% | Good | Можно улучшить |
| 50-69% | Warning | Добавить тесты для uncovered mutants |
| <50% | Critical | Тесты неэффективны, high risk |
Пример mutation report
Mutation Score: 75% (150/200 mutants killed)
Surviving Mutants:
1. OrderService.cs:42 - if (quantity > 0) → if (quantity >= 0)
Status: Survived
Test coverage: None
2. DiscountCalculator.cs:15 - price * 0.9 → price / 0.9
Status: Survived
Test coverage: Covered by DiscountTests but assertion too weak
Killed Mutants:
1. UserService.cs:28 - if (user == null) → if (user != null)
Status: Killed by UserServiceTests.Create_NullUser_ThrowsExceptionCoverage Quality vs Quantity
Code Coverage 90% но Mutation Score 40%:
var result = CalculateDiscount(price);
Assert.NotNull(result); // Weak assertion!
// Coverage: line executed
// Mutation: если CalculateDiscount изменён — тест не detect
Code Coverage 60% но Mutation Score 80%:
var result = CalculateDiscount(100);
Assert.Equal(10, result); // Strong assertion!
// Coverage: меньше lines executed
// Mutation: любая change в logic — test failsВывод: Mutation score > Code coverage как метрика качества тестов.
Практика
Упражнение 1: Test Pyramid для проекта
// 1. Проанализируйте существующий проект
// 2. Посчитайте Unit/Integration/E2E тесты
// 3. Визуализируйте distribution
// 4. Если imbalance — создайте план rebalancing
// 5. Implement new tests для достижения 70-20-10Упражнение 2: Contract test между producer и consumer
// 1. Создайте Consumer project с Pact tests
// 2. Определите 3+ API interactions
// 3. Создайте Producer project
// 4. Producer verifies Pact contracts
// 5. Измените Producer API — contract test должен failУпражнение 3: Mutation analysis
// 1. Установите Stryker.NET
// 2. Запустите mutation analysis на проекте
// 3. Проанализируйте surviving mutants
// 4. Напишите tests для killing surviving mutants
// 5. Цель: mutation score > 80%Advanced Mocking Patterns
Обзор
Advanced mocking patterns решают сложные проблемы тестирования: partial mocks, time-dependent code, async race conditions, fake implementations с replay capability.
Partial Mocking
Что это
Partial mocking — mock где некоторые методы mocked, другие используют real implementation. Полезно для testing abstract classes или когда нужно mock только часть поведения.
Когда нужно
// Abstract class с partial implementation
public abstract class PaymentProcessor
{
public abstract Task<bool> ValidatePayment(Payment payment);
public virtual async Task<PaymentResult> Process(Payment payment)
{
if (!await ValidatePayment(payment))
return PaymentResult.Failed("Invalid payment");
var amount = CalculateAmount(payment);
var receipt = GenerateReceipt(payment, amount);
return PaymentResult.Success(receipt);
}
protected virtual decimal CalculateAmount(Payment payment) =>
payment.Amount * (1 - payment.Discount);
protected virtual string GenerateReceipt(Payment payment, decimal amount) =>
$"Receipt #{Guid.NewGuid()}: {amount:C}";
}
// Partial mock — test ValidatePayment, rest is real
[Fact]
public async Task Process_InvalidPayment_ReturnsFailed()
{
var mock = new Mock<PaymentProcessor>();
mock.CallBase = true; // Use real implementation by default
mock.Setup(p => p.ValidatePayment(It.IsAny<Payment>()))
.ReturnsAsync(false); // Only mock this method
var result = await mock.Object.Process(new Payment { Amount = 100 });
Assert.False(result.Success);
Assert.Equal("Invalid payment", result.ErrorMessage);
}Когда антипаттерн
// BAD: partial mocking чтобы test private behavior через backdoor
var mock = new Mock<MyService>();
mock.CallBase = true;
mock.Setup(s => s.SomePublicMethod()).CallBase = false;
mock.Setup(s => s.AnotherPublicMethod()).CallBase = false;
// ... mocking 80% of class — это значит class слишком сложный!
// Решение: refactor class, extract dependencies, test smaller unitsRule of thumb: Если partial mocking нужен для >2 methods — class violates SRP.
Testing Private Methods
Почему это плохая идея
// BAD: testing private method через reflection
[Fact]
public void PrivateMethod_Test()
{
var service = new OrderService();
var method = typeof(OrderService).GetMethod(
"CalculateTax", BindingFlags.NonPublic | BindingFlags.Instance);
var result = method.Invoke(service, new object[] { 100m, "US" });
Assert.Equal(10m, result);
}
// Проблемы:
// 1. Refactoring breaks tests (rename private method)
// 2. Tests implementation details, not behavior
// 3. Hard to maintain
// 4. Doesn't catch integration issuesАльтернативы
// 1. Test через public API
[Fact]
public void ProcessOrder_IncludesTaxInTotal()
{
var order = new Order { Subtotal = 100m, Country = "US" };
var result = _service.ProcessOrder(order);
Assert.Equal(110m, result.Total); // 100 + 10% tax
}
// 2. Extract private method в отдельный class
public class TaxCalculator
{
public decimal Calculate(decimal amount, string country) { ... }
}
// Теперь TaxCalculator можно test напрямую
[Fact]
public void Calculate_USTax_Returns10Percent()
{
var calculator = new TaxCalculator();
Assert.Equal(10m, calculator.Calculate(100m, "US"));
}
// 3. InternalVisibleTo для test project
// В production project .csproj:
// <ItemGroup>
// <InternalsVisibleTo Include="YourProject.Tests" />
// </ItemGroup>
// internal method теперь visible для tests
internal decimal CalculateTax(decimal amount, string country) { ... }Time-Dependent Code Testing
Проблема
// BAD: код зависит от DateTime.Now — невозможно test deterministically
public class NotificationService
{
public async Task SendReminder(User user)
{
if (user.LastLogin > DateTime.UtcNow.AddDays(-7))
return; // Don't remind active users
await _email.SendAsync(user.Email, "Come back!", "...");
}
}
// Как test "user logged in 8 days ago" без mocking time?SystemClock Abstraction
// 1. Define abstraction
public interface ISystemClock
{
DateTime UtcNow { get; }
}
// 2. Production implementation
public class SystemClock : ISystemClock
{
public DateTime UtcNow => DateTime.UtcNow;
}
// 3. Refactor code to use abstraction
public class NotificationService
{
private readonly ISystemClock _clock;
private readonly IEmailService _email;
public NotificationService(ISystemClock clock, IEmailService email)
{
_clock = clock;
_email = email;
}
public async Task SendReminder(User user)
{
var sevenDaysAgo = _clock.UtcNow.AddDays(-7);
if (user.LastLogin > sevenDaysAgo)
return;
await _email.SendAsync(user.Email, "Come back!", "...");
}
}
// 4. Test с fake clock
[Fact]
public async Task SendReminder_UserInactiveFor8Days_SendsEmail()
{
// Arrange
var fakeClock = new Mock<ISystemClock>();
fakeClock.Setup(c => c.UtcNow).Returns(new DateTime(2024, 1, 15));
var mockEmail = new Mock<IEmailService>();
var user = new User { LastLogin = new DateTime(2024, 1, 5) }; // 10 days ago
var service = new NotificationService(fakeClock.Object, mockEmail.Object);
// Act
await service.SendReminder(user);
// Assert
mockEmail.Verify(e => e.SendAsync(
user.Email, "Come back!", It.IsAny<string>()), Times.Once);
}
[Fact]
public async Task SendReminder_UserActive_Last3Days_NoEmail()
{
var fakeClock = new Mock<ISystemClock>();
fakeClock.Setup(c => c.UtcNow).Returns(new DateTime(2024, 1, 15));
var mockEmail = new Mock<IEmailService>();
var user = new User { LastLogin = new DateTime(2024, 1, 13) }; // 2 days ago
var service = new NotificationService(fakeClock.Object, mockEmail.Object);
await service.SendReminder(user);
mockEmail.Verify(e => e.SendAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>()), Times.Never);
}Testable Clock с control over time
public class TestableClock : ISystemClock
{
private DateTime _currentTime;
public TestableClock(DateTime initialTime)
{
_currentTime = initialTime;
}
public DateTime UtcNow => _currentTime;
public void Advance(TimeSpan duration)
{
_currentTime = _currentTime.Add(duration);
}
public void Set(DateTime time)
{
_currentTime = time;
}
}
// Использование
[Fact]
public async Task Reminder_SentAfter7Days()
{
var clock = new TestableClock(new DateTime(2024, 1, 1));
var mockEmail = new Mock<IEmailService>();
var user = new User { LastLogin = new DateTime(2024, 1, 1) };
var service = new NotificationService(clock, mockEmail.Object);
// Day 1: No reminder (just logged in)
await service.SendReminder(user);
mockEmail.Verify(e => e.SendAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>()), Times.Never);
// Day 5: Still no reminder
clock.Advance(TimeSpan.FromDays(4));
await service.SendReminder(user);
mockEmail.Verify(e => e.SendAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>()), Times.Never);
// Day 8: Reminder sent!
clock.Advance(TimeSpan.FromDays(3));
await service.SendReminder(user);
mockEmail.Verify(e => e.SendAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>()), Times.Once);
}Async Method Testing
Race Conditions в тестах
// BAD: non-deterministic async test
[Fact]
public async Task BadAsyncTest()
{
var results = new List<int>();
var task1 = Task.Run(async () =>
{
await Task.Delay(100);
results.Add(1);
});
var task2 = Task.Run(async () =>
{
await Task.Delay(50);
results.Add(2);
});
await Task.WhenAll(task1, task2);
// NON-DETERMINISTIC: order depends on timing
Assert.Equal(new[] { 2, 1 }, results); // Sometimes fails!
}Deterministic Async Testing
// 1. Use controlled Task completion
[Fact]
public async Task ProcessAsync_CompletesInOrder()
{
var tcs1 = new TaskCompletionSource<bool>();
var tcs2 = new TaskCompletionSource<bool>();
var results = new ConcurrentBag<int>();
var task1 = ProcessAsync(1, tcs1.Task);
var task2 = ProcessAsync(2, tcs2.Task);
// Complete in controlled order
tcs1.SetResult(true);
await task1;
Assert.Contains(1, results);
tcs2.SetResult(true);
await task2;
Assert.Contains(2, results);
}
private async Task ProcessAsync(int id, Task trigger)
{
await trigger;
// Process...
}
// 2. Use Task.Delay с predictable timing
[Fact]
public async Task Retry_SucceedsOnThirdAttempt()
{
var mockService = new Mock<IService>();
var callCount = 0;
mockService
.Setup(s => s.CallAsync())
.ReturnsAsync(() =>
{
callCount++;
if (callCount < 3) throw new InvalidOperationException("Fail");
return "Success";
});
var retryService = new RetryService(mockService.Object, maxRetries: 3);
var result = await retryService.ExecuteAsync();
Assert.Equal("Success", result);
Assert.Equal(3, callCount);
}Testing Concurrent Operations
[Fact]
public async Task ConcurrentProcessing_AllItemsProcessed()
{
// Arrange
var items = Enumerable.Range(1, 100).ToList();
var processedItems = new ConcurrentBag<int>();
// Act
var tasks = items.Select(async item =>
{
var result = await _processor.ProcessAsync(item);
processedItems.Add(result);
});
await Task.WhenAll(tasks);
// Assert
Assert.Equal(100, processedItems.Count);
Assert.Equal(items.OrderBy(x => x), processedItems.OrderBy(x => x));
}
// Testing timeout behavior
[Fact]
public async Task ProcessAsync_ExceedsTimeout_ThrowsTimeoutException()
{
var mockService = new Mock<IService>();
mockService
.Setup(s => s.CallAsync())
.ReturnsAsync(async () =>
{
await Task.Delay(TimeSpan.FromSeconds(10));
return "result";
});
var service = new TimeoutService(mockService.Object, TimeSpan.FromSeconds(1));
await Assert.ThrowsAsync<TimeoutException>(
async () => await service.ExecuteAsync());
}Fake Implementation с Replay Capability
Что это
Fake implementation которая записывает interactions и может replay их для deterministic testing.
public class FakeApiServer : IExternalApi
{
private readonly List<RecordedCall> _recordedCalls = new();
private readonly Dictionary<string, Func<HttpRequest, HttpResponse>> _handlers = new();
public IReadOnlyList<RecordedCall> RecordedCalls => _recordedCalls.AsReadOnly();
public void SetupHandler(string path, Func<HttpRequest, HttpResponse> handler)
{
_handlers[path] = handler;
}
public async Task<HttpResponse> SendAsync(HttpRequest request)
{
if (_handlers.TryGetValue(request.Path, out var handler))
{
var response = handler(request);
_recordedCalls.Add(new RecordedCall(request, response));
return response;
}
throw new KeyNotFoundException($"No handler for {request.Path}");
}
public void Reset()
{
_recordedCalls.Clear();
}
}
public record RecordedCall(HttpRequest Request, HttpResponse Response);
public record HttpRequest(string Method, string Path, string Body);
public record HttpResponse(int StatusCode, string Body);
// Использование
[Fact]
public async Task OrderService_CallsPaymentApi_Correctly()
{
// Arrange
var fakeApi = new FakeApiServer();
fakeApi.SetupHandler("/api/payments", request =>
{
var payment = JsonSerializer.Deserialize<PaymentRequest>(request.Body);
return new HttpResponse(200, $"{{\"transactionId\": \"txn-123\"}}");
});
var service = new OrderService(fakeApi);
// Act
await service.ProcessPaymentAsync(new PaymentRequest { Amount = 100 });
// Assert
var call = Assert.Single(fakeApi.RecordedCalls);
Assert.Equal("POST", call.Request.Method);
Assert.Equal("/api/payments", call.Request.Path);
Assert.Contains("100", call.Request.Body);
}
// Replay capability — record once, replay many times
[Fact]
public async Task Replay_PreRecordedResponses_Deterministic()
{
var fakeApi = new FakeApiServer();
// Record mode
fakeApi.SetupHandler("/api/users/1", _ =>
new HttpResponse(200, "{\"id\":1,\"name\":\"Alice\"}"));
var service = new UserService(fakeApi);
var user1 = await service.GetUserAsync(1);
Assert.Equal("Alice", user1.Name);
// Replay mode — same response every time
fakeApi.Reset();
var user2 = await service.GetUserAsync(1);
Assert.Equal("Alice", user2.Name);
var user3 = await service.GetUserAsync(1);
Assert.Equal("Alice", user3.Name);
}Практика
Упражнение 1: Testable clock abstraction
// Создайте TimeSensitiveService который:
// - Блокирует аккаунт после 3 failed login attempts за 15 минут
// - Разблокирует через 24 часа
// - Отправляет security alert если attempts > 5 за 1 час
//
// Используйте ISystemClock abstraction
// Напишите deterministic tests с TestableClockУпражнение 2: Async concurrent operation test
// Создайте RateLimiter который:
// - Allows N requests per second
// - Queues excess requests
// - Times out after 5 seconds
//
// Напишите test для:
// - Normal load (under limit)
// - Overload (over limit, queuing)
// - Timeout scenario
// - Concurrent requests from multiple threadsУпражнение 3: Fake API с replay capability
// Создайте FakeGitHubApi который:
// - Записывает все calls
// - Может replay pre-recorded responses
// - Поддерживает setup handlers для разных endpoints
//
// Напишите tests для GitHubService который использует этот fakePerformance Testing
Обзор
Performance testing — измерение и анализ производительности кода под различными нагрузками. Включает microbenchmarks, load testing, soak testing, spike testing.
BenchmarkDotNet
Установка
dotnet new console -n Benchmarks
cd Benchmarks
dotnet add package BenchmarkDotNetБазовое использование
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
namespace MyBenchmarks;
public class Program
{
public static void Main(string[] args) => BenchmarkRunner.Run<StringBenchmarks>();
}
[MemoryDiagnoser] // Показывает allocation
public class StringBenchmarks
{
private string _longString = null!;
[Params(100, 1000, 10000)]
public int Length { get; set; }
[GlobalSetup]
public void Setup()
{
_longString = new string('a', Length);
}
[Benchmark]
public string StringConcat()
{
var result = "";
foreach (var c in _longString)
result += c;
return result;
}
[Benchmark]
public string StringBuilder()
{
var sb = new StringBuilder(Length);
foreach (var c in _longString)
sb.Append(c);
return sb.ToString();
}
[Benchmark]
public string StringCreate()
{
return string.Create(Length, _longString, (span, source) =>
{
for (int i = 0; i < source.Length; i++)
span[i] = source[i];
});
}
}Результат
| Method | Length | Mean | Error | StdDev | Gen0 | Allocated |
|---------------- |------- |-----------:|----------:|----------:|-------:|----------:|
| StringConcat | 100 | 1.234 us | 0.0234 us | 0.0219 us | 5.1230 | 10.54 KB |
| StringBuilder | 100 | 0.089 us | 0.0012 us | 0.0011 us | 0.0610 | 128 B |
| StringCreate | 100 | 0.045 us | 0.0008 us | 0.0007 us | 0.0305 | 64 B |
| StringConcat | 1000 | 89.456 us | 1.2345 us | 1.1547 us | 42.358 | 87.12 KB |
| StringBuilder | 1000 | 0.567 us | 0.0089 us | 0.0083 us | 0.2441 | 512 B |
| StringCreate | 1000 | 0.234 us | 0.0034 us | 0.0032 us | 0.0610 | 128 B |Advanced Configuration
[Config(typeof(CustomConfig))]
public class AdvancedBenchmarks
{
private class CustomConfig : ManualConfig
{
public CustomConfig()
{
// Multiple jobs
AddJob(Job.Default
.WithRuntime(CoreRuntime.Core80)
.WithGcServer(true)
.WithId("ServerGC"));
AddJob(Job.Default
.WithRuntime(CoreRuntime.Core80)
.WithGcServer(false)
.WithId("WorkstationGC"));
// Diagnosers
AddDiagnoser(MemoryDiagnoser.Default);
AddDiagnoser(ThreadingDiagnoser.Default);
AddDiagnoser(EventPipeProfilerProfiler.Default);
// Exporters
AddExporter(MarkdownExporter.GitHub);
AddExporter(CsvMeasurementsExporter.Default);
AddExporter(HtmlExporter.Default);
// Validation
AddValidator(JitOptimizationsValidator.FailOnError);
AddValidator(ExecutionValidator.FailOnError);
}
}
[Benchmark]
public void Benchmark1() { /* ... */ }
}Baseline Comparison
[MemoryDiagnoser]
public class DictionaryBenchmarks
{
private Dictionary<int, string> _dict = null!;
private List<KeyValuePair<int, string>> _list = null!;
[Params(100, 1000, 10000)]
public int Size { get; set; }
[GlobalSetup]
public void Setup()
{
_dict = Enumerable.Range(0, Size)
.ToDictionary(i => i, i => $"Value{i}");
_list = _dict.ToList();
}
[Benchmark(Baseline = true)]
public string DictionaryLookup()
{
return _dict[Size / 2];
}
[Benchmark]
public string ListLookup()
{
return _list.First(x => x.Key == Size / 2).Value;
}
}
// Результат показывает ratio vs baseline:
// | Method | Size | Mean | Ratio |
// |----------------- |------ |---------:|------:|
// | DictionaryLookup | 10000 | 12.34 ns | 1.00 |
// | ListLookup | 10000 | 456.78 ns| 37.02 |Statistical Analysis
// BenchmarkDotNet автоматически рассчитывает:
// - Mean: среднее значение
// - Error: margin of error (99.9% confidence interval)
// - StdDev: standard deviation
// - Median: медиана
// - Min/Max: минимум/максимум
// - P0-P100: percentiles
// Для manual statistical validation:
[SimpleJob(RuntimeMoniker.Net80, launchCount: 5, warmupCount: 3, iterationCount: 10)]
public class StatisticalBenchmarks
{
// launchCount: сколько раз запускать process
// warmupCount: warmup iterations (не измеряются)
// iterationCount: measured iterations
// Больше iterations = более точные результаты
}Load Testing — k6
Что это
Load testing проверяет как система behaves под expected load (например, 1000 requests per second).
k6 Installation
# Windows (winget)
winget install k6
# Или download: https://k6.io/docs/get-started/installation/Basic k6 Script
// load-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';
export const options = {
// Stages: ramp up, steady, ramp down
stages: [
{ duration: '30s', target: 100 }, // Ramp up to 100 users
{ duration: '1m', target: 100 }, // Stay at 100 users
{ duration: '30s', target: 500 }, // Ramp up to 500 users
{ duration: '1m', target: 500 }, // Stay at 500 users
{ duration: '30s', target: 0 }, // Ramp down to 0
],
// Thresholds: fail test if exceeded
thresholds: {
http_req_duration: ['p(95)<500'], // 95% requests < 500ms
http_req_failed: ['rate<0.01'], // < 1% failures
},
};
export default function () {
// Test GET endpoint
const res = http.get('http://localhost:5000/api/users');
check(res, {
'status is 200': (r) => r.status === 200,
'response time < 500ms': (r) => r.timings.duration < 500,
});
sleep(1);
// Test POST endpoint
const payload = JSON.stringify({
name: 'Test User',
email: `test${__VU}-${__ITER}@test.com`
});
const params = { headers: { 'Content-Type': 'application/json' } };
const postRes = http.post('http://localhost:5000/api/users', payload, params);
check(postRes, {
'create status is 201': (r) => r.status === 201,
});
sleep(1);
}Запуск
k6 run load-test.js
# С output в InfluxDB + Grafana
k6 run --out influxdb=http://localhost:8086/k6 load-test.jsРезультат
data_received..............: 1.2 MB 20 kB/s
data_sent..................: 456 kB 7.6 kB/s
http_req_duration..........: avg=123ms min=12ms med=89ms max=890ms p(90)=234ms p(95)=345ms
http_req_failed............: 0.5% ✓ 12 ✗ 2388
http_reqs..................: 2400 40/s
iteration_duration.........: avg=1.2s min=1s med=1.1s max=2.1s
iterations.................: 1200 20/s
vus........................: 500 min=100 max=500
vus_max....................: 500 min=500 max=500Soak Testing — Memory Leak Detection
Что это
Soak testing — long-duration load test (часы/дни) для detection memory leaks, resource exhaustion, degradation over time.
k6 Soak Test
// soak-test.js
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
// Constant load for 24 hours
stages: [
{ duration: '5m', target: 100 }, // Ramp up
{ duration: '23h55m', target: 100 }, // Steady load
],
thresholds: {
http_req_duration: ['p(95)<1000'],
http_req_failed: ['rate<0.05'],
},
};
export default function () {
http.get('http://localhost:5000/api/health');
sleep(0.5);
}Memory Monitoring
// В приложении: monitor memory over time
public class MemoryMonitor : BackgroundService
{
private readonly ILogger<MemoryMonitor> _logger;
public MemoryMonitor(ILogger<MemoryMonitor> logger)
{
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
var memory = GC.GetGCMemoryInfo();
var heapSize = GC.GetTotalMemory(false);
_logger.LogInformation(
"Memory: Heap={HeapSize:N0} bytes, " +
"Gen0={Gen0}, Gen1={Gen1}, Gen2={Gen2}",
heapSize,
memory.GenerationInfo[0].CollectionCount,
memory.GenerationInfo[1].CollectionCount,
memory.GenerationInfo[2].CollectionCount);
await Task.Delay(TimeSpan.FromMinutes(5), stoppingToken);
}
}
}Анализ Memory Leak
Признаки memory leak в soak test:
1. Memory usage постоянно растёт (no plateau)
2. GC collections не освобождают достаточно memory
3. Gen2 collections увеличиваются over time
4. После load removal memory не возвращается к baseline
Инструменты:
- dotnet-gcdump: collect GC dump
- dotnet-dump: full process dump
- dotnet-counters: real-time metrics
- Visual Studio Diagnostic Tools: detailed analysisSpike Testing
Что это
Spike testing — sudden load increase для проверки system behavior under stress.
k6 Spike Test
// spike-test.js
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
stages: [
{ duration: '1m', target: 50 }, // Normal load
{ duration: '30s', target: 1000 }, // SPIKE! 20x increase
{ duration: '1m', target: 1000 }, // Hold spike
{ duration: '30s', target: 50 }, // Return to normal
{ duration: '1m', target: 50 }, // Recovery
],
thresholds: {
http_req_duration: ['p(99)<2000'], // 99% < 2s even during spike
http_req_failed: ['rate<0.1'], // < 10% failures
},
};
export default function () {
http.get('http://localhost:5000/api/users');
sleep(0.1);
}System Behavior Under Spike
Ожидаемое поведение:
1. Response time увеличивается (acceptable)
2. Error rate остаётся низким (< 10%)
3. После spike — recovery к normal performance
4. No data loss или corruption
Нежелательное поведение:
1. Cascading failures (один сервис тянет другие)
2. Memory leak после spike
3. Dead connections не cleanup
4. Database connection pool exhaustionПрактика
Упражнение 1: Comprehensive benchmark suite
// Создайте benchmark suite для:
// 1. String operations (concat, StringBuilder, Span)
// 2. Collection operations (List vs Array vs Span)
// 3. Serialization (JSON, MessagePack, Protobuf)
// 4. Database operations (EF Core vs Dapper vs ADO.NET)
//
// Requirements:
// - Statistical significance (10+ iterations)
// - Multiple runtimes (.NET 8, .NET 9)
// - Memory diagnostics
// - Markdown + CSV exportУпражнение 2: Load test для API endpoint (1k RPS target)
// Создайте k6 load test:
// 1. Ramp up to 1000 RPS over 2 minutes
// 2. Hold for 5 minutes
// 3. Ramp down
// 4. Thresholds: p95 < 200ms, error rate < 1%
// 5. Monitor CPU, memory, response timesУпражнение 3: Soak test 24 часа
# 1. Запустите soak test на 24 часа
# 2. Collect memory dumps каждые 2 часа
# 3. Analyze memory growth pattern
# 4. Identify any leaks
# 5. Report: memory behavior over timeTest Infrastructure
Обзор
Test infrastructure — всё что окружает тесты: CI/CD pipelines, flaky test detection, test data factories, cross-platform compatibility.
CI/CD Integration
Parallel Test Execution
<!-- .csproj: включить parallel execution -->
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<!-- xUnit: parallel by default -->
<xUnitParallelizeTestAssemblies>true</xUnitParallelizeTestAssemblies>
</PropertyGroup>
</Project>// xUnit: отключить parallel для specific tests
// (когда tests share state)
[assembly: CollectionBehavior(DisableTestParallelization = true)]
// Или для specific collection
[CollectionDefinition("Sequential", DisableParallelization = true)]
public class SequentialCollection { }
[Collection("Sequential")]
public class TestsThatShareState
{
// Эти тесты run sequentially
}GitHub Actions CI
# .github/workflows/ci.yml
name: CI
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
dotnet: ['8.0.x']
os: [ubuntu-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ matrix.dotnet }}
- name: Restore
run: dotnet restore
- name: Build
run: dotnet build --no-restore --configuration Release
- name: Unit Tests
run: dotnet test --no-build --configuration Release
--filter "Category=Unit"
--logger "trx;LogFileName=unit-results.trx"
--collect:"XPlat Code Coverage"
- name: Integration Tests
run: dotnet test --no-build --configuration Release
--filter "Category=Integration"
--logger "trx;LogFileName=integration-results.trx"
- name: Upload Test Results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ matrix.os }}
path: '**/*.trx'
- name: Upload Coverage
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-${{ matrix.os }}
path: '**/coverage.cobertura.xml'Shard Strategy (large test suites)
# Разделить тесты на shards для parallel execution
jobs:
test-shard-1:
runs-on: ubuntu-latest
steps:
- name: Run Tests (Shard 1)
run: dotnet test --filter "FullyQualifiedName~NamespaceA"
test-shard-2:
runs-on: ubuntu-latest
steps:
- name: Run Tests (Shard 2)
run: dotnet test --filter "FullyQualifiedName~NamespaceB"
test-shard-3:
runs-on: ubuntu-latest
steps:
- name: Run Tests (Shard 3)
run: dotnet test --filter "FullyQualifiedName~NamespaceC"Optimal Sharding
// Стратегия sharding по времени выполнения
// 1. Collect test durations
// 2. Group tests to equal total duration per shard
// 3. Balance fast + slow tests across shards
// Пример: 3 shards, ~5 minutes each
// Shard 1: 50 fast tests (100ms each) + 10 slow tests (30s each) = 5min
// Shard 2: 45 fast tests + 12 slow tests = 5min
// Shard 3: 55 fast tests + 8 slow tests = 5minFlaky Test Detection
Что это
Flaky tests — тесты которые иногда pass иногда fail без changes в code. Главная причина frustration в CI.
Common Causes
| Причина | Пример | Решение |
|---|---|---|
| Timing issues | await Task.Delay(100) | Use controlled timing |
| Shared state | Static variables, singletons | Isolate tests |
| Non-deterministic order | Dictionary iteration | Sort before assert |
| External dependencies | Network, file system | Mock/fake |
| Random values | new Random() | Seeded random |
| Time-dependent | DateTime.Now | ISystemClock |
Flaky Test Detector
// Run tests multiple times to detect flakiness
public class FlakyTestDetector
{
public async Task<FlakyTestReport> DetectFlakyTests(
Assembly testAssembly, int runCount = 10)
{
var flakyTests = new List<FlakyTestInfo>();
var allTests = GetTestMethods(testAssembly);
foreach (var test in allTests)
{
int passCount = 0;
int failCount = 0;
for (int i = 0; i < runCount; i++)
{
var result = await RunTestAsync(test);
if (result.Passed) passCount++;
else failCount++;
}
if (passCount > 0 && failCount > 0)
{
flakyTests.Add(new FlakyTestInfo
{
TestName = test.Name,
PassRate = (double)passCount / runCount,
FailRate = (double)failCount / runCount,
TotalRuns = runCount
});
}
}
return new FlakyTestReport
{
TotalTests = allTests.Count,
FlakyTests = flakyTests,
FlakyRate = (double)flakyTests.Count / allTests.Count
};
}
}
public record FlakyTestInfo
{
public string TestName { get; init; }
public double PassRate { get; init; }
public double FailRate { get; init; }
public int TotalRuns { get; init; }
}
public record FlakyTestReport
{
public int TotalTests { get; init; }
public List<FlakyTestInfo> FlakyTests { get; init; }
public double FlakyRate { get; init; }
}Automatic Quarantine
# GitHub Actions: quarantine flaky tests
- name: Detect Flaky Tests
run: |
dotnet run --project FlakyDetector -- \
--assembly tests.dll \
--runs 20 \
--output flaky-report.json
- name: Quarantine Flaky Tests
if: steps.detect.outputs.flaky-count > 0
run: |
# Add flaky tests to quarantine list
cat flaky-report.json | jq '.flakyTests[].testName' > quarantined-tests.txt
# Run CI excluding quarantined tests
dotnet test --filter "FullyQualifiedName!~@Quarantine"// Mark flaky tests for quarantine
[Fact]
[Trait("Category", "Quarantine")] // Exclude from CI
public void FlakyTest_NeedsInvestigation()
{
// TODO: Fix this test and remove quarantine
}
// CI filter: --filter "Category!=Quarantine"Elimination Strategies
1. Identify: Run tests 50+ times, find inconsistent ones
2. Isolate: Determine root cause (timing, state, randomness)
3. Fix: Apply appropriate fix (see table above)
4. Verify: Run 100+ times, confirm no flakiness
5. Monitor: Track flaky test rate over time in CITest Data Management
Factories
// Simple factory pattern
public static class UserFactory
{
public static User Create()
{
return new User
{
Id = Guid.NewGuid(),
Name = "Test User",
Email = $"test-{Guid.NewGuid():N}@test.com",
CreatedAt = DateTime.UtcNow,
IsActive = true
};
}
public static User CreateAdmin()
{
var user = Create();
user.Role = "Admin";
return user;
}
public static User CreateInactive()
{
var user = Create();
user.IsActive = false;
return user;
}
public static List<User> CreateMany(int count)
{
return Enumerable.Range(1, count)
.Select(_ => Create())
.ToList();
}
}
// Usage
[Fact]
public void ProcessUser_ValidUser_Success()
{
var user = UserFactory.Create();
var result = _service.ProcessUser(user);
Assert.True(result.Success);
}Builder Pattern
// Fluent builder для complex objects
public class UserBuilder
{
private Guid _id = Guid.NewGuid();
private string _name = "Test User";
private string _email = "test@test.com";
private DateTime _createdAt = DateTime.UtcNow;
private string _role = "User";
private bool _isActive = true;
private List<Order> _orders = new();
public UserBuilder WithId(Guid id) { _id = id; return this; }
public UserBuilder WithName(string name) { _name = name; return this; }
public UserBuilder WithEmail(string email) { _email = email; return this; }
public UserBuilder WithRole(string role) { _role = role; return this; }
public UserBuilder Inactive() { _isActive = false; return this; }
public UserBuilder WithOrders(params Order[] orders)
{
_orders = orders.ToList();
return this;
}
public User Build()
{
return new User
{
Id = _id,
Name = _name,
Email = _email,
CreatedAt = _createdAt,
Role = _role,
IsActive = _isActive,
Orders = _orders
};
}
}
// Usage
[Fact]
public void CalculateDiscount_VipUserWithOrders_AppliesDiscount()
{
var user = new UserBuilder()
.WithRole("VIP")
.WithOrders(
new Order { Amount = 100 },
new Order { Amount = 200 })
.Build();
var discount = _calculator.CalculateDiscount(user);
Assert.Equal(0.15m, discount); // 15% for VIP
}Bogus (Fake Data Generator)
// NuGet: Bogus
using Bogus;
public class DataGenerators
{
private static readonly Faker<User> UserFaker = new Faker<User>()
.RuleFor(u => u.Id, f => Guid.NewGuid())
.RuleFor(u => u.Name, f => f.Name.FullName())
.RuleFor(u => u.Email, f => f.Internet.Email())
.RuleFor(u => u.Phone, f => f.Phone.PhoneNumber())
.RuleFor(u => u.Address, f => f.Address.FullAddress())
.RuleFor(u => u.CreatedAt, f => f.Date.Past(1))
.RuleFor(u => u.IsActive, f => f.Random.Bool(0.9f)); // 90% active
private static readonly Faker<Order> OrderFaker = new Faker<Order>()
.RuleFor(o => o.Id, f => Guid.NewGuid())
.RuleFor(o => o.Amount, f => f.Finance.Amount(10, 1000))
.RuleFor(o => o.Status, f => f.PickRandom<OrderStatus>())
.RuleFor(o => o.CreatedAt, f => f.Date.Recent(30));
public static User GenerateUser() => UserFaker.Generate();
public static List<User> GenerateUsers(int count) => UserFaker.Generate(count);
public static Order GenerateOrder() => OrderFaker.Generate();
public static List<Order> GenerateOrders(int count) => OrderFaker.Generate(count);
}
// Usage
[Fact]
public void ProcessBatch_100Users_AllProcessed()
{
var users = DataGenerators.GenerateUsers(100);
var results = _service.ProcessBatch(users);
Assert.Equal(100, results.Count);
Assert.All(results, r => Assert.True(r.Success));
}Seed Databases
// Seed для integration tests
public static class TestDatabaseSeeder
{
public static async Task SeedAsync(AppDbContext context)
{
if (await context.Users.AnyAsync()) return;
var users = new[]
{
new UserBuilder().WithName("Alice").WithRole("Admin").Build(),
new UserBuilder().WithName("Bob").WithRole("User").Build(),
new UserBuilder().WithName("Charlie").WithRole("User").Build(),
};
await context.Users.AddRangeAsync(users);
var orders = new[]
{
new OrderBuilder().ForUser(users[0]).WithAmount(100).Build(),
new OrderBuilder().ForUser(users[0]).WithAmount(200).Build(),
new OrderBuilder().ForUser(users[1]).WithAmount(50).Build(),
};
await context.Orders.AddRangeAsync(orders);
await context.SaveChangesAsync();
}
}Cross-Platform Testing
Linux Containers vs Windows Hosts
# Test on multiple platforms
jobs:
test-linux:
runs-on: ubuntu-latest
steps:
- run: dotnet test
test-windows:
runs-on: windows-latest
steps:
- run: dotnet test
test-macos:
runs-on: macos-latest
steps:
- run: dotnet testCommon Issues
| Issue | Linux | Windows | Fix |
|---|---|---|---|
| Path separators | / | \ | Use Path.Combine |
| Case sensitivity | Case-sensitive | Case-insensitive | Use consistent casing |
| Line endings | \n | \r\n | Use Environment.NewLine |
| File permissions | Strict | Relaxed | Set explicit permissions |
| Docker socket | /var/run/docker.sock | npipe:// | Use Testcontainers (handles this) |
// Cross-platform compatible code
[Fact]
public void FileOperations_CrossPlatform_Works()
{
// BAD: hardcoded path
// var path = "C:\\temp\\file.txt";
// GOOD: cross-platform
var path = Path.Combine(Path.GetTempPath(), "file.txt");
// BAD: hardcoded line ending
// var content = "line1\r\nline2";
// GOOD: cross-platform
var content = string.Join(Environment.NewLine, "line1", "line2");
}Практика
Упражнение 1: Parallel test execution в CI
# 1. Настройте GitHub Actions workflow
# 2. Разделите тесты на 3 shards
# 3. Каждый shard run parallel
# 4. Collect и aggregate results
# 5. Цель: total test time < 5 minutesУпражнение 2: Flaky test detector
// 1. Создайте FlakyTestDetector
// 2. Run tests 50+ times
// 3. Identify flaky tests
# 4. Quarantine flaky tests
# 5. Fix root causesУпражнение 3: Test data factory pattern
// 1. Создайте builders для всех domain objects
// 2. Реализуйте Bogus generators
// 3. Создайте seeder для integration tests
// 4. Используйте в 10+ testsQuality Gates и Metrics
Обзор
Quality gates — автоматические проверки которые block merge/deployment если метрики не соответствуют threshold. Metrics: code coverage, CRAP score, mutation score, test execution time.
Code Coverage
Types
| Type | Описание | Пример | Когда использовать |
|---|---|---|---|
| Line Coverage | % строк кода executed | 85% | Basic quality gate |
| Branch Coverage | % branches taken | 70% | Better than line |
| Path Coverage | % execution paths | 50% | Complex logic |
| Method Coverage | % methods called | 90% | High-level metric |
Line vs Branch vs Path
public string GetGreeting(int hour, bool isWeekend)
{
if (hour < 12) // Branch 1: hour < 12
return "Good morning";
else if (hour < 18) // Branch 2: hour >= 12 && hour < 18
return "Good afternoon";
else // Branch 3: hour >= 18
return "Good evening";
}
// Line coverage: 4/4 lines = 100%
// Branch coverage: нужно 3 теста для 100%
// Path coverage: 3 paths (hour + weekend combinations)// Тесты для 100% branch coverage
[Theory]
[InlineData(8, false, "Good morning")] // Branch 1
[InlineData(14, false, "Good afternoon")] // Branch 2
[InlineData(20, true, "Good evening")] // Branch 3
public void GetGreeting_AllBranches_Correct(int hour, bool isWeekend, string expected)
{
Assert.Equal(expected, GetGreeting(hour, isWeekend));
}Что измерять (и что нет)
MEASURE:
✓ Business logic
✓ Algorithms
✓ Validation rules
✓ Error handling
✓ Edge cases
DON'T MEASURE:
✗ Generated code (migrations, DTOs)
✗ Simple getters/setters
✗ Configuration classes
✗ Third-party code
✗ UI markup (XAML, Razor)Coverage Configuration
<!-- .csproj: enable coverage collection -->
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<CollectCoverage>true</CollectCoverage>
<CoverletOutputFormat>cobertura</CoverletOutputFormat>
<CoverletOutput>./coverage.cobertura.xml</CoverletOutput>
</PropertyGroup>
</Project>
<!-- Exclude from coverage -->
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<PackageReference Include="coverlet.msbuild" Version="6.0.0" />
<PackageReference Include="coverlet.collector" Version="6.0.0" />
</ItemGroup>
</Project>// Exclude via attributes
[ExcludeFromCodeCoverage]
public class GeneratedDto
{
public int Id { get; set; }
public string Name { get; set; }
}
// Or via runsettings
// runsettings.xml
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="XPlat code coverage">
<Configuration>
<Format>cobertura</Format>
<Exclude>
<ExcludeByFile>**/Migrations/*.cs</ExcludeByFile>
<ExcludeByAttribute>GeneratedCodeAttribute</ExcludeByAttribute>
</Exclude>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>CRAP Score
Что это
CRAP (Change Risk Anti-Patterns) score = комбинация cyclomatic complexity и code coverage. Высокий CRAP = high risk code that needs refactoring или better tests.
Формула
CRAP(m) = comp(m)^2 * (1 - cov(m)/100)^3 + comp(m)
где:
- comp(m) = cyclomatic complexity метода
- cov(m) = code coverage метода (0-100%)Примеры
// CRAP = 1 (perfect): complexity 1, coverage 100%
public int Add(int a, int b) => a + b;
// CRAP = 1^2 * (1 - 1)^3 + 1 = 0 + 1 = 1
// CRAP = 6: complexity 2, coverage 0%
public int Divide(int a, int b)
{
if (b == 0) throw new DivideByZeroException();
return a / b;
}
// CRAP = 2^2 * (1 - 0)^3 + 2 = 4 + 2 = 6
// CRAP = 132: complexity 10, coverage 50%
public string ComplexMethod(int x, string y, bool z)
{
if (x > 0)
{
if (y.Length > 5)
{
if (z) return "A";
else return "B";
}
else
{
if (z) return "C";
else return "D";
}
}
else
{
if (y.Length > 10)
{
if (z) return "E";
else return "F";
}
else
{
if (z) return "G";
else return "H";
}
}
}
// CRAP = 10^2 * (1 - 0.5)^3 + 10 = 100 * 0.125 + 10 = 12.5 + 10 = 22.5CRAP Score Interpretation
| Score | Risk Level | Action |
|---|---|---|
| 1-5 | Low | Acceptable |
| 6-20 | Medium | Consider refactoring |
| 21-50 | High | Refactor soon |
| 50+ | Critical | Refactor immediately |
CRAP Score в CI
# Calculate CRAP score in pipeline
- name: Calculate CRAP Score
run: |
# Using NCrunch or similar tool
dotnet test --collect:"XPlat Code Coverage"
# Generate CRAP report
dotnet run --project CrapCalculator -- \
--coverage coverage.cobertura.xml \
--complexity complexity.xml \
--output crap-report.json
- name: Fail on High CRAP
run: |
# Fail if any method has CRAP > 50
cat crap-report.json | jq '.methods[] | select(.crapScore > 50)'
if [ $? -eq 0 ]; then
echo "Found methods with CRAP score > 50"
exit 1
fiSonarQube Quality Gates
Setup
# GitHub Actions with SonarQube
- name: SonarQube Scan
uses: SonarSource/sonarqube-scan-action@v4
with:
args: >
-Dsonar.projectKey=my-project
-Dsonar.sources=.
-Dsonar.tests=**/*Tests.cs
-Dsonar.coverageReportPaths=coverage.cobertura.xml
-Dsonar.cs.opencover.reportsPaths=**/coverage.opencover.xml
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
- name: SonarQube Quality Gate
uses: sonarsource/sonarqube-quality-gate-action@v1
timeout-minutes: 5
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}Quality Gate Conditions
Default Quality Gate:
✓ Coverage on New Code > 80%
✓ Duplicated Lines on New Code < 3%
✓ Maintainability Rating on New Code = A
✓ Reliability Rating on New Code = A
✓ Security Rating on New Code = A
✓ Technical Debt Ratio on New Code < 5%
Custom Quality Gate:
✓ Coverage > 80%
✓ Mutation Score > 70%
✓ Zero Critical Issues
✓ Zero Blocker Issues
✓ CRAP Score < 50 for all methodsSonarQube Analysis Configuration
<!-- sonar-project.properties -->
sonar.projectKey=dotnet-senior-testing
sonar.projectName=DotNet Senior Testing Module
sonar.sources=src/
sonar.tests=tests/
sonar.cs.opencover.reportsPaths=**/coverage.opencover.xml
sonar.coverageReportPaths=**/coverage.cobertura.xml
sonar.exclusions=**/Migrations/**,**/*.Designer.cs,**/obj/**,**/bin/**
sonar.test.inclusions=**/*Tests.cs,**/*Test.csTest Effectiveness Metrics
Mutation Score
Mutation Score = (Killed Mutants / Total Mutants) * 100
Target: > 70% for critical paths
Target: > 50% for non-critical codeTest Execution Time Trends
# Track test duration over time
- name: Track Test Duration
run: |
dotnet test --logger:"trx;LogFileName=results.trx"
# Parse TRX file for durations
dotnet run --project TestMetrics -- \
--results results.trx \
--output metrics.json
# Upload to dashboard
curl -X POST https://metrics.example.com/api/test-duration \
-H "Authorization: Bearer $TOKEN" \
-d @metrics.jsonTest Health Dashboard
Metrics to track:
1. Test Count (total, by category)
2. Pass Rate (% passing)
3. Flaky Test Rate (% flaky)
4. Coverage Trend (over time)
5. Mutation Score Trend
6. Average Test Duration
7. CI Pipeline Duration
8. CRAP Score DistributionПрактика
Упражнение 1: Quality Gate настройка
# 1. Настройте coverage collection в CI
# 2. Установите threshold: coverage > 80%
# 3. Установите threshold: mutation score > 70%
# 4. Установите threshold: zero critical issues
# 5. Pipeline fails если любой threshold не metУпражнение 2: Dashboard для test health metrics
// 1. Collect metrics после каждого test run
# 2. Store metrics (JSON, database, or API)
# 3. Visualize trends (Grafana, custom dashboard)
# 4. Alert on degradationУпражнение 3: Pre-commit hook с fast tests
#!/bin/bash
# .git/hooks/pre-commit
echo "Running fast tests before commit..."
# Run only unit tests (fast subset)
dotnet test --filter "Category=Unit&ExecutionTime<5s" \
--no-build \
--configuration Release
if [ $? -ne 0 ]; then
echo "Fast tests failed! Commit blocked."
exit 1
fi
# Run coverage check
dotnet test --filter "Category=Unit" \
--collect:"XPlat Code Coverage"
# Check coverage threshold
coverage=$(cat coverage.cobertura.xml | grep -o 'line-rate="[^"]*"' | head -1)
if (( $(echo "$coverage < 0.80" | bc -l) )); then
echo "Coverage below 80%! Commit blocked."
exit 1
fi
echo "All checks passed!"
exit 0Testing Distributed Systems
Обзор
Testing distributed systems — challenges и strategies для microservices, event-driven architectures, и distributed data systems.
Chaos Testing
Что это
Chaos testing — intentional fault injection для проверки system resilience. Inspired by Netflix Chaos Monkey.
Types of Faults
| Fault Type | Example | Impact |
|---|---|---|
| Service Failure | Kill a service process | Test circuit breaker |
| Network Partition | Block network between services | Test timeout/retry |
| Latency Injection | Add 5s delay to responses | Test timeout handling |
| Database Failure | Kill database connection | Test fallback |
| Message Loss | Drop messages from queue | Test idempotency |
| Clock Skew | Desync service clocks | Test time-sensitive logic |
Polly + Chaos Engineering
// NuGet: Polly, Polly.Extensions
// Chaos strategy: inject failures
public class ChaosPolicy
{
public static IAsyncPolicy<T> CreateChaosPolicy<T>(
double failureRate = 0.1, // 10% failures
TimeSpan? latency = null)
{
return Policy<T>
.Handle<Exception>()
.FallbackAsync(async (context, ct) =>
{
// Fallback behavior
return default;
})
.WrapAsync(Policy<T>
.Handle<Exception>()
.RetryAsync(3, onRetry: (exception, retry, context) =>
{
Console.WriteLine($"Retry {retry} after {exception.Message}");
})
)
.WrapAsync(Policy<T>
.Handle<Exception>()
.CircuitBreakerAsync(
handledEventsAllowedBeforeBreaking: 5,
durationOfBreak: TimeSpan.FromSeconds(30),
onBreak: (exception, duration) =>
{
Console.WriteLine($"Circuit broken: {exception.Message}");
},
onReset: () => Console.WriteLine("Circuit reset")
)
);
}
}
// Chaos middleware for ASP.NET Core
public class ChaosMiddleware
{
private readonly RequestDelegate _next;
private readonly double _failureRate;
private readonly TimeSpan? _latency;
private readonly Random _random = new();
public ChaosMiddleware(RequestDelegate next, double failureRate = 0.1, TimeSpan? latency = null)
{
_next = next;
_failureRate = failureRate;
_latency = latency;
}
public async Task InvokeAsync(HttpContext context)
{
// Inject latency
if (_latency.HasValue)
{
await Task.Delay(_latency.Value);
}
// Inject failures
if (_random.NextDouble() < _failureRate)
{
context.Response.StatusCode = 503;
await context.Response.WriteAsync("Service Unavailable (Chaos)");
return;
}
await _next(context);
}
}
// Register in Program.cs (only in test environment)
if (app.Environment.IsEnvironment("ChaosTesting"))
{
app.UseMiddleware<ChaosMiddleware>(failureRate: 0.1, latency: TimeSpan.FromMilliseconds(500));
}Chaos Test Scenarios
public class ChaosTests : IAsyncLifetime
{
private readonly WebApplicationFactory<Program> _factory;
private readonly HttpClient _client;
public ChaosTests()
{
_factory = new WebApplicationFactory<Program>()
.WithWebHostBuilder(builder =>
{
builder.UseEnvironment("ChaosTesting");
});
_client = _factory.CreateClient();
}
public Task InitializeAsync() => Task.CompletedTask;
public Task DisposeAsync() => Task.CompletedTask;
[Fact]
public async Task ServiceFailure_CircuitBreakerOpens_GracefulDegradation()
{
// Simulate 10 consecutive failures
for (int i = 0; i < 10; i++)
{
var response = await _client.GetAsync("/api/users");
// After circuit opens, should return fallback
}
// Verify circuit breaker opened
var circuitState = GetCircuitState();
Assert.Equal(CircuitState.Open, circuitState);
}
[Fact]
public async Task NetworkPartition_RetrySucceeds_EventualConsistency()
{
// Simulate network partition
SimulateNetworkPartition(duration: TimeSpan.FromSeconds(5));
// Request should retry and eventually succeed
var sw = Stopwatch.StartNew();
var response = await _client.GetAsync("/api/orders");
sw.Stop();
Assert.True(response.IsSuccessStatusCode);
Assert.True(sw.Elapsed > TimeSpan.FromSeconds(5)); // Waited for partition to heal
}
[Fact]
public async Task LatencyInjection_TimeoutHandled_NoCascadingFailure()
{
// Inject 10s latency
var response = await _client.GetAsync("/api/slow-endpoint");
// Should timeout after 3s, not hang
Assert.Equal(HttpStatusCode.RequestTimeout, response.StatusCode);
// Other endpoints should still work
var healthyResponse = await _client.GetAsync("/api/health");
Assert.True(healthyResponse.IsSuccessStatusCode);
}
}Contract Testing at Scale
Consumer-Driven Contract Registry
// Pact Broker integration
public class ContractRegistry
{
private readonly HttpClient _pactBrokerClient;
public ContractRegistry(Uri pactBrokerUri, string token)
{
_pactBrokerClient = new HttpClient
{
BaseAddress = pactBrokerUri
};
_pactBrokerClient.DefaultRequestHeaders.Add(
"Authorization", $"Bearer {token}");
}
// Publish contract
public async Task PublishContractAsync(string consumerName, string providerName, string pactJson)
{
var content = new StringContent(pactJson, Encoding.UTF8, "application/json");
var response = await _pactBrokerClient.PutAsync(
$"/pacts/provider/{providerName}/consumer/{consumerName}", content);
response.EnsureSuccessStatusCode();
}
// Verify contract
public async Task<bool> VerifyContractAsync(string providerName, string providerVersion)
{
var response = await _pactBrokerClient.PostAsync(
$"/pacts/provider/{providerName}/verification",
new StringContent(JsonSerializer.Serialize(new { providerVersion })));
var result = await response.Content.ReadFromJsonAsync<VerificationResult>();
return result.Success;
}
// Can I deploy?
public async Task<bool> CanIDeployAsync(string pacticipant, string version)
{
var response = await _pactBrokerClient.GetAsync(
$"/pacticipant/{pacticipant}/version/{version}/can-i-deploy");
return response.IsSuccessStatusCode;
}
}
public record VerificationResult(bool Success, List<string> Errors);Multi-Service Contract Tests
// Service A (Consumer) defines contract with Service B (Provider)
public class ServiceA_Contracts
{
[Fact]
public async Task GetUserFromServiceB_ValidId_ReturnsUser()
{
// Define expected interaction
var pact = new PactBuilder()
.ServiceConsumer("ServiceA")
.HasPactWith("ServiceB")
.UponReceiving("a request for user details")
.Given("user exists")
.WithRequest(HttpMethod.Get, "/api/users/1")
.WillRespond()
.WithStatus(HttpStatusCode.OK)
.WithBody(new { id = 1, name = "Alice", email = "alice@test.com" });
await pact.Build();
// Verify contract
var client = new HttpClient { BaseAddress = pact.MockProviderUri };
var response = await client.GetAsync("/api/users/1");
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
}
}
// Service B (Provider) verifies contract
public class ServiceB_ContractVerification
{
[Fact]
public async Task VerifyServiceAContract()
{
var verifier = new PactVerifier(new PactVerifierConfig
{
PublishVerificationResults = true,
ProviderVersion = "1.0.0"
});
verifier
.Provider("ServiceB")
.WithHttpEndpoint(new Uri("http://localhost:5001"))
.WithPactBroker(new Uri("https://pact-broker.example.com"))
.Verify();
}
}E2E Testing Strategies
Когда стоит
| Сценарий | E2E нужен? | Почему |
|---|---|---|
| Critical user journey | Да | End-to-end verification |
| Payment flow | Да | Money involved |
| Authentication/Authorization | Да | Security critical |
| Simple CRUD | Нет | Covered by integration tests |
| UI styling | Нет | Visual regression better |
| API contract | Нет | Contract testing better |
Когда не стоит
E2E Anti-Patterns:
✗ E2E для каждого use case (too slow)
✗ E2E вместо unit tests (wrong level)
✗ E2E с production data (security risk)
✗ E2E без cleanup (data pollution)
✗ E2E that depend on each other (fragile)E2E Test Example (Playwright)
// NuGet: Microsoft.Playwright.NUnit
using Microsoft.Playwright.NUnit;
[Parallelizable(ParallelScope.Self)]
[TestFixture]
public class E2ETests : PageTest
{
[Test]
public async Task UserRegistration_FullFlow_Success()
{
// Navigate to registration
await Page.GotoAsync("http://localhost:5000/register");
// Fill form
await Page.FillAsync("#name", "Test User");
await Page.FillAsync("#email", "test@example.com");
await Page.FillAsync("#password", "SecurePass123!");
// Submit
await Page.ClickAsync("#submit");
// Verify redirect to dashboard
await Page.WaitForURLAsync("**/dashboard");
await Expect(Page.Locator("#welcome")).ToContainTextAsync("Welcome, Test User");
// Verify email sent (check test email inbox)
var emails = await GetTestEmailsAsync("test@example.com");
Assert.Single(emails);
Assert.Contains("Welcome", emails[0].Subject);
}
private async Task<List<TestEmail>> GetTestEmailsAsync(string email)
{
// Integration with test email service (e.g., MailHog, Ethereal)
using var client = new HttpClient();
var response = await client.GetAsync(
$"http://localhost:8025/api/v2/search?kind=to&query={email}");
var result = await response.Content.ReadFromJsonAsync<EmailSearchResult>();
return result.Items;
}
}Test Environment Management
Ephemeral Environments per PR
# GitHub Actions: create ephemeral environment
name: Deploy Ephemeral Environment
on:
pull_request:
types: [opened, synchronize]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy to Ephemeral Environment
run: |
# Create unique environment name
ENV_NAME="pr-${{ github.event.pull_request.number }}"
# Deploy to Kubernetes namespace
kubectl create namespace $ENV_NAME --dry-run=client -o yaml | kubectl apply -f -
# Deploy application
helm install my-app ./charts/my-app \
--namespace $ENV_NAME \
--set image.tag=${{ github.sha }} \
--set environment=ephemeral
# Get URL
URL=$(kubectl get ingress my-app -n $ENV_NAME -o jsonpath='{.spec.rules[0].host}')
# Comment on PR
gh pr comment ${{ github.event.pull_request.number }} \
--body "Ephemeral environment deployed: https://$URL"
- name: Run E2E Tests
run: |
URL=$(kubectl get ingress my-app -n $ENV_NAME -o jsonpath='{.spec.rules[0].host}')
dotnet test --filter "Category=E2E" \
--environment "BASE_URL=https://$URL"
cleanup:
runs-on: ubuntu-latest
if: github.event.pull_request.merged == true || github.event.action == 'closed'
steps:
- name: Cleanup Ephemeral Environment
run: |
ENV_NAME="pr-${{ github.event.pull_request.number }}"
kubectl delete namespace $ENV_NAME --ignore-not-foundEnvironment Configuration
// appsettings.Ephemeral.json
{
"ConnectionStrings": {
"DefaultConnection": "Server=postgres;Database=testdb;Username=postgres;Password=postgres"
},
"Redis": {
"ConnectionString": "redis:6379"
},
"RabbitMQ": {
"HostName": "rabbitmq",
"Port": 5672
},
"Logging": {
"LogLevel": {
"Default": "Debug"
}
}
}Практика
Упражнение 1: Chaos test для service failure
// 1. Создайте chaos middleware
// 2. Настройте circuit breaker
// 3. Напишите tests для:
// - Service failure → circuit opens → fallback
// - Network partition → retry → eventual success
// - Latency injection → timeout → no cascadeУпражнение 2: Contract test registry
// 1. Настройте Pact Broker (local or cloud)
// 2. Создайте 3+ consumer contracts
// 3. Producer verifies all contracts
// 4. Implement can-i-deploy checkУпражнение 3: Ephemeral environment provisioning
# 1. Создайте GitHub Actions workflow
# 2. Deploy PR to isolated environment
# 3. Run E2E tests against environment
# 4. Cleanup on PR merge/close
# 5. Post environment URL to PR commentsКонтрольная точка модуля 6
Обзор
Финальный проект модуля — создание полной тестовой стратегии для microservice platform с unit tests, integration tests, contract tests, performance benchmarks, и quality gates в CI.
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ API Gateway │
│ (YARP / Ocelot) │
└────────────┬──────────────────────┬──────────────────┬──────────┘
│ │ │
▼ ▼ ▼
┌────────────────┐ ┌────────────────┐ ┌────────────────┐
│ User Service │ │ Order Service │ │ Payment Service │
│ │ │ │ │ │
│ - PostgreSQL │ │ - PostgreSQL │ │ - External API │
│ - Redis Cache │ │ - RabbitMQ │ │ - Retry/Timeout │
└────────────────┘ └────────────────┘ └────────────────┘
│ │ │
└──────────────────────┼──────────────────┘
▼
┌────────────────┐
│ Notification │
│ Service │
│ │
│ - RabbitMQ │
│ - Email/SMS │
└────────────────┘Requirements
Unit Tests (coverage > 85%, mutation score > 75%)
// User Service: Domain logic
public class UserServiceTests
{
[Fact]
public void Register_ValidUser_CreatesUserWithHashedPassword()
{
// Arrange
var mockPasswordHasher = new Mock<IPasswordHasher>();
mockPasswordHasher
.Setup(h => h.Hash(It.IsAny<string>()))
.Returns("hashed-password");
var mockUserRepository = new Mock<IUserRepository>();
mockUserRepository
.Setup(r => r.CreateAsync(It.IsAny<User>()))
.ReturnsAsync((User u) => u);
var service = new UserService(
mockUserRepository.Object, mockPasswordHasher.Object);
// Act
var result = service.Register("test@test.com", "SecurePass123!");
// Assert
Assert.True(result.Success);
mockPasswordHasher.Verify(h => h.Hash("SecurePass123!"), Times.Once);
mockUserRepository.Verify(r => r.CreateAsync(
It.Is<User>(u => u.PasswordHash == "hashed-password")), Times.Once);
}
[Theory]
[InlineData("")]
[InlineData("invalid-email")]
[InlineData("test@")]
[InlineData("@test.com")]
public void Register_InvalidEmail_ReturnsValidationError(string email)
{
var service = new UserService(
Mock.Of<IUserRepository>(), Mock.Of<IPasswordHasher>());
var result = service.Register(email, "SecurePass123!");
Assert.False(result.Success);
Assert.Contains("Invalid email", result.Errors);
}
[Fact]
public void Register_WeakPassword_ReturnsValidationError()
{
var service = new UserService(
Mock.Of<IUserRepository>(), Mock.Of<IPasswordHasher>());
var result = service.Register("test@test.com", "123");
Assert.False(result.Success);
Assert.Contains("Password too weak", result.Errors);
}
}Integration Tests с Testcontainers
public class UserServiceIntegrationTests : IAsyncLifetime
{
private readonly PostgreSqlContainer _postgres;
private readonly RedisContainer _redis;
private AppDbContext _dbContext;
private IConnectionMultiplexer _redisConnection;
public async Task InitializeAsync()
{
_postgres = new PostgreSqlBuilder().Build();
_redis = new RedisBuilder().Build();
await Task.WhenAll(_postgres.StartAsync(), _redis.StartAsync());
var dbOptions = new DbContextOptionsBuilder<AppDbContext>()
.UseNpgsql(_postgres.GetConnectionString())
.Options;
_dbContext = new AppDbContext(dbOptions);
await _dbContext.Database.MigrateAsync();
_redisConnection = await ConnectionMultiplexer.ConnectAsync(
_redis.GetConnectionString());
}
public async Task DisposeAsync()
{
await _dbContext.DisposeAsync();
await _redisConnection.DisposeAsync();
await Task.WhenAll(_postgres.DisposeAsync(), _redis.DisposeAsync());
}
[Fact]
public async Task Register_CreateAndRetrieveUser_Success()
{
var service = new UserService(
new EfUserRepository(_dbContext),
new RealPasswordHasher(),
new RedisCacheService(_redisConnection));
var result = await service.RegisterAsync("test@test.com", "SecurePass123!");
Assert.True(result.Success);
var user = await _dbContext.Users.FindAsync(result.UserId);
Assert.NotNull(user);
Assert.Equal("test@test.com", user.Email);
}
[Fact]
public async Task Login_ValidCredentials_ReturnsToken()
{
// Seed user
await _dbContext.Users.AddAsync(new User
{
Email = "test@test.com",
PasswordHash = new RealPasswordHasher().Hash("SecurePass123!")
});
await _dbContext.SaveChangesAsync();
var service = new UserService(
new EfUserRepository(_dbContext),
new RealPasswordHasher(),
new RedisCacheService(_redisConnection));
var result = await service.LoginAsync("test@test.com", "SecurePass123!");
Assert.True(result.Success);
Assert.NotNull(result.Token);
}
}Contract Tests между сервисами
// Order Service (Consumer) → Payment Service (Provider)
public class OrderPaymentContractTests
{
[Fact]
public async Task ProcessPayment_ValidOrder_PaymentSucceeds()
{
var pact = new PactBuilder()
.ServiceConsumer("OrderService")
.HasPactWith("PaymentService")
.UponReceiving("a request to process payment")
.Given("order exists")
.WithRequest(HttpMethod.Post, "/api/payments")
.WithBody(new { orderId = "ord-123", amount = 99.99m })
.WillRespond()
.WithStatus(HttpStatusCode.OK)
.WithBody(new { transactionId = "txn-456", status = "completed" });
await pact.Build();
var client = new HttpClient { BaseAddress = pact.MockProviderUri };
var response = await client.PostAsJsonAsync("/api/payments",
new { orderId = "ord-123", amount = 99.99m });
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var result = await response.Content.ReadFromJsonAsync<PaymentResult>();
Assert.Equal("txn-456", result.TransactionId);
Assert.Equal("completed", result.Status);
}
}
// Notification Service (Consumer) → RabbitMQ (Provider)
public class NotificationRabbitMqContractTests
{
[Fact]
public async Task SendNotification_PublishesMessageToQueue()
{
var rabbitMq = new RabbitMqBuilder().Build();
await rabbitMq.StartAsync();
var factory = new ConnectionFactory
{
Uri = new Uri(rabbitMq.GetConnectionString())
};
using var connection = await factory.CreateConnectionAsync();
using var channel = await connection.CreateChannelAsync();
await channel.QueueDeclareAsync(
queue: "notifications",
durable: true,
exclusive: false,
autoDelete: false);
var message = new NotificationMessage
{
UserId = "user-123",
Type = "email",
Content = "Welcome!"
};
var body = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(message));
await channel.BasicPublishAsync(
exchange: "",
routingKey: "notifications",
body: body);
// Verify message was published
var result = await channel.BasicGetAsync("notifications", autoAck: true);
Assert.NotNull(result);
var receivedMessage = JsonSerializer.Deserialize<NotificationMessage>(
Encoding.UTF8.GetString(result.Body.ToArray()));
Assert.Equal("user-123", receivedMessage.UserId);
Assert.Equal("email", receivedMessage.Type);
}
}Performance Benchmarks с CI Integration
[MemoryDiagnoser]
[SimpleJob(RuntimeMoniker.Net80)]
public class UserServiceBenchmarks
{
private UserService _service;
private List<User> _users;
[Params(100, 1000, 10000)]
public int UserCount { get; set; }
[GlobalSetup]
public void Setup()
{
_service = new UserService(
new InMemoryUserRepository(),
new RealPasswordHasher(),
new NullCacheService());
_users = Enumerable.Range(1, UserCount)
.Select(i => new User
{
Email = $"user{i}@test.com",
PasswordHash = $"hash{i}"
})
.ToList();
}
[Benchmark]
public async Task<List<User>> GetAllUsers()
{
return await _service.GetAllUsersAsync();
}
[Benchmark]
public async Task<User> GetUserById()
{
return await _service.GetUserByIdAsync(1);
}
[Benchmark]
public async Task<bool> RegisterUser()
{
var result = await _service.RegisterAsync(
$"newuser{Guid.NewGuid():N}@test.com", "SecurePass123!");
return result.Success;
}
}Quality Gate в CI Pipeline
# .github/workflows/quality-gate.yml
name: Quality Gate
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- name: Run Unit Tests
run: dotnet test --filter "Category=Unit" --collect:"XPlat Code Coverage"
- name: Check Coverage
run: |
coverage=$(cat **/coverage.cobertura.xml | grep -o 'line-rate="[^"]*"' | head -1 | cut -d'"' -f2)
if (( $(echo "$coverage < 0.85" | bc -l) )); then
echo "Coverage $coverage < 85% threshold"
exit 1
fi
integration-tests:
runs-on: ubuntu-latest
services:
docker:
image: docker:24.0
options: --privileged
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- name: Run Integration Tests
run: dotnet test --filter "Category=Integration"
mutation-testing:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- name: Install Stryker
run: dotnet tool install -g dotnet-stryker
- name: Run Mutation Testing
run: stryker --threshold-high 80 --threshold-low 60 --threshold-break 40
contract-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- name: Run Contract Tests
run: dotnet test --filter "Category=Contract"
- name: Publish Pact Contracts
run: |
dotnet pact publish --broker-url $PACT_BROKER_URL --broker-token $PACT_BROKER_TOKEN
performance-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- name: Run Benchmarks
run: dotnet run --project Benchmarks --configuration Release
- name: Check Performance Regression
run: |
# Compare with baseline
dotnet run --project PerfRegression -- \
--baseline baseline.json \
--current results.json \
--threshold 10%
quality-gate:
needs: [unit-tests, integration-tests, mutation-testing, contract-tests, performance-tests]
runs-on: ubuntu-latest
if: always()
steps:
- name: Check All Gates Passed
run: |
if [ "${{ needs.unit-tests.result }}" != "success" ]; then
echo "Unit tests failed"
exit 1
fi
if [ "${{ needs.integration-tests.result }}" != "success" ]; then
echo "Integration tests failed"
exit 1
fi
if [ "${{ needs.mutation-testing.result }}" != "success" ]; then
echo "Mutation testing failed"
exit 1
fi
if [ "${{ needs.contract-tests.result }}" != "success" ]; then
echo "Contract tests failed"
exit 1
fi
if [ "${{ needs.performance-tests.result }}" != "success" ]; then
echo "Performance tests failed"
exit 1
fi
echo "All quality gates passed!"Критерии Passing
| Metric | Target | Check |
|---|---|---|
| Unit Test Coverage | > 85% | Coverage report |
| Mutation Score | > 75% | Stryker report |
| Integration Tests | 100% pass | CI pipeline |
| Contract Tests | 100% pass | Pact verification |
| Flaky Tests | 0 in 30 days | CI history |
| Test Suite Duration | < 5 minutes | CI timing |
| Performance Regression | < 10% degradation | Benchmark comparison |
| Critical Issues | 0 | SonarQube |
Deliverables
- Source Code — полный проект с 4 microservices
- Test Suite — unit, integration, contract, performance tests
- CI Pipeline — GitHub Actions workflow с quality gates
- Documentation — test strategy document
- Metrics Dashboard — coverage, mutation score, trends