Modular Monolith with DDD

Full Modular Monolith .NET application with Domain-Driven Design approach.

1. Introduction

1.1 Purpose of this Repository

This is a list of the main goals of this repository:

Showing how you can implement a monolith application in a modular way
Presentation of the full implementation of an application
- This is not another simple application
- This is not another proof of concept (PoC)
- The goal is to present the implementation of an application that would be ready to run in production
Showing the application of best practices and object-oriented programming principles
Presentation of the use of design patterns. When, how and why they can be used
Presentation of some architectural considerations, decisions, approaches
Presentation of the implementation using Domain-Driven Design approach (tactical patterns)
Presentation of the implementation of Unit Tests for Domain Model (Testable Design in mind)

1.2 Out of Scope

This is a list of subjects which are out of scope for this repository:

Business requirements gathering and analysis
System analysis
Domain exploration
Domain distillation
Domain-Driven Design strategic patterns
Architecture evaluation, quality attributes analysis
Integration, system tests
Project management
Infrastructure
Containerization
Software engineering process, CI/CD
Deployment process
Maintenance
Documentation

1.3 Reason

The reason for creating this repository is the lack of something similar. Most sample applications on GitHub have at least one of the following issues:

Very, very simple - few entities and use cases implemented
Not finished (for example there is no authentication, logging, etc..)
Poorly designed (in my opinion)
Poorly implemented (in my opinion)
Not well described
Assumptions and decisions are not clearly explained
Implements "Orders" domain - yes, everyone knows this domain, but something different is needed
Implemented in old technology
Not maintained

To sum up, there are some very good examples, but there are far too few of them. This repository has the task of filling this gap at some level.

1.4 Disclaimer

Software architecture should always be created to resolve specific business problems. Software architecture always supports some quality attributes and at the same time does not support others. A lot of other factors influence your software architecture - your team, opinions, preferences, experiences, technical constraints, time, budget, etc.

Always functional requirements, quality attributes, technical constraints and other factors should be considered before an architectural decision is made.

Because of the above, the architecture and implementation presented in this repository is one of the many ways to solve some problems. Take from this repository as much as you want, use it as you like but remember to always pick the best solution which is appropriate to the problem class you have.

1.5 Give a Star

My primary focus in this project is on quality. Creating a good quality product involves a lot of analysis, research and work. It takes a lot of time. If you like this project, learned something or you are using it in your applications, please give it a star ⭐. This is the best motivation for me to continue this work. Thanks!

1.6 Share It

There are very few really good examples of this type of application. If you think this repository makes a difference and is worth it, please share it with your friends and on social networks. I will be extremely grateful.

2. Domain

2.1 Description

Definition:

Domain - A sphere of knowledge, influence, or activity. The subject area to which the user applies a program is the domain of the software. Domain-Driven Design Reference, Eric Evans

The Meeting Groups domain was selected for the purposes of this project based on the Meetup.com system.

Main reasons for selecting this domain:

It is common, a lot of people use the Meetup site to organize or attend meetings
There is a system for it, so everyone can check this implementation against a working site which supports this domain
It is not complex so it is easy to understand
It is not trivial - there are some business rules and logic and it is not just CRUD operations
You don't need much specific domain knowledge unlike other domains like financing, banking, medical
It is not big so it is easier to implement

Meetings

The main business entities are Member, Meeting Group and Meeting. A Member can create a Meeting Group, be part of a Meeting Group or can attend a Meeting.

A Meeting Group Member can be an Organizer of this group or a normal Member.

Only an Organizer of a Meeting Group can create a new Meeting.

A Meeting has attendees, not attendees (Members which declare they will not attend the Meeting) and Members on the Waitlist.

A Meeting can have an attendee limit. If the limit is reached, Members can only sign up to the Waitlist.

A Meeting Attendee can bring guests to the Meeting. The number of guests allowed is an attribute of the Meeting. Bringing guests can be unallowed.

A Meeting Attendee can have one of two roles: Attendee or Host. A Meeting must have at least one Host. The Host is a special role which grants permission to edit Meeting information or change the attendees list.

Administration

To create a new Meeting Group, a Member needs to propose the group. A Meeting Group Proposal is sent to Administrators. An Administrator can accept or reject a Meeting Group Proposal. If a Meeting Group Proposal is accepted, a Meeting Group is created.

Payments

To be able to organize Meetings, the Meeting Group must be paid for. The Meeting Group Organizer who is the Payer, must pay some fee according to a payment plan.

Additionally, Meeting organizer can set an Event Fee. Each Meeting Attendee is obliged to pay the fee. All guests should be paid by Meeting Attendee too.

Users

Each Administrator, Member and Payer is a User. To be a User, User Registration is required and confirmed.

Each User is assigned one or more User Role.

Each User Role has set of Permissions. A Permission defines whether User can invoke a particular action.

2.2 Conceptual Model

Definition:

Conceptual Model - A conceptual model is a representation of a system, made of the composition of concepts that are used to help people know, understand, or simulate a subject the model represents. Wikipedia - Conceptual model

Conceptual Model

2.3 Event Storming

While a Conceptual Model focuses on structures and relationships between them, behavior and events that occur in our domain are more important.

There are many ways to show behavior and events. One of them is a light technique called Event Storming which is becoming more popular. Below are presented 3 main business processes using this technique: user registration, meeting group creation and meeting organization.

Note: Event Storming is a light, live workshop. One of the possible outputs of this workshop is presented here. Even if you are not doing Event Storming workshops, this type of process presentation can be very valuable to you and your stakeholders.

User Registration process

Meeting Group creation

Meeting organization

3. Architecture

3.1 High Level View

Module descriptions:

API - Very thin ASP.NET MVC Core REST API application. Main responsibilities are:
1. Accept request
2. Authenticate and authorize request (using User Access module)
3. Delegate work to specific module sending Command or Query
4. Return response
User Access - responsible for user authentication, authorization and registration
Meetings - implements Meetings Bounded Context: creating meeting groups, meetings
Administration - implements Administration Bounded Context: implements administrative tasks like meeting group proposal verification
Payments - implements Payments Bounded Context: implements all functionalities associated with payments
In Memory Events Bus - Publish/Subscribe implementation to asynchronously integrate all modules using events (Event Driven Architecture).

Key assumptions:

API contains no application logic
API communicates with Modules using a small interface to send Queries and Commands
Each Module has its own interface which is used by API
Modules communicate each other only asynchronously using Events Bus - direct method calls are not allowed
Each Module has it's own data in a separate schema - shared data is not allowed
- Module data could be moved into separate databases if desired
Modules can only have a dependency on the integration events assembly of other Module (see Module level view)
Each Module has its own Composition Root, which implies that each Module has its own Inversion-of-Control container
API as a host needs to initialize each module and each module has an initialization method
Each Module is highly encapsulated - only required types and members are public, the rest are internal or private

3.2 Module Level View

Each Module has Clean Architecture and consists of the following submodules (assemblies):

Application - the application logic submodule which is responsible for requests processing: use cases, domain events, integration events, internal commands.
Domain - Domain Model in Domain-Driven Design terms implements the applicable Bounded Context
Infrastructure - infrastructural code responsible for module initialization, background processing, data access, communication with Events Bus and other external components or systems
IntegrationEvents - Contracts published to the Events Bus; only this assembly can be called by other modules

Note: Application, Domain and Infrastructure assemblies could be merged into one assembly. Some people like horizontal layering or more decomposition, some don't. Implementing the Domain Model or Infrastructure in separate assembly allows encapsulation using the internal keyword. Sometimes Bounded Context logic is not worth it because it is too simple. As always, be pragmatic and take whatever approach you like.

3.3 API and Module Communication

The API only communicates with Modules in two ways: during module initialization and request processing.

Module initialization

Each module has a static Initialize method which is invoked in the API Startup class. All configuration needed by this module should be provided as arguments to this method. All services are configured during initialization and the Composition Root is created using the Inversion-of-Control Container.

public static void Initialize(
    string connectionString,
    IExecutionContextAccessor executionContextAccessor,
    ILogger logger,
    EmailsConfiguration emailsConfiguration)
{
    var moduleLogger = logger.ForContext("Module", "Meetings");

    ConfigureCompositionRoot(connectionString, executionContextAccessor, moduleLogger, emailsConfiguration);

    QuartzStartup.Initialize(moduleLogger);

    EventsBusStartup.Initialize(moduleLogger);
}

Request processing

Each module has the same interface signature exposed to the API. It contains 3 methods: command with result, command without result and query.

public interface IMeetingsModule
{
    Task<TResult> ExecuteCommandAsync<TResult>(ICommand<TResult> command);

    Task ExecuteCommandAsync(ICommand command);

    Task<TResult> ExecuteQueryAsync<TResult>(IQuery<TResult> query);
}

Note: Some people say that processing a command should not return a result. This is an understandable approach but sometimes impractical, especially when you want to immediately return the ID of a newly created resource. Sometimes the boundary between Command and Query is blurry. One example is AuthenticateCommand - it returns a token but it is not a query because it has a side effect.

3.4 Module Requests Processing via CQRS

Processing of Commands and Queries is separated by applying the architectural style/pattern Command Query Responsibility Segregation (CQRS).

Commands are processed using Write Model which is implemented using DDD tactical patterns:

internal class CreateNewMeetingGroupCommandHandler : ICommandHandler<CreateNewMeetingGroupCommand>
{
    private readonly IMeetingGroupRepository _meetingGroupRepository;
    private readonly IMeetingGroupProposalRepository _meetingGroupProposalRepository;

    internal CreateNewMeetingGroupCommandHandler(
        IMeetingGroupRepository meetingGroupRepository,
        IMeetingGroupProposalRepository meetingGroupProposalRepository)
    {
        _meetingGroupRepository = meetingGroupRepository;
        _meetingGroupProposalRepository = meetingGroupProposalRepository;
    }

    public async Task<Unit> Handle(CreateNewMeetingGroupCommand request, CancellationToken cancellationToken)
    {
        var meetingGroupProposal = await _meetingGroupProposalRepository.GetByIdAsync(request.MeetingGroupProposalId);

        var meetingGroup = meetingGroupProposal.CreateMeetingGroup();

        await _meetingGroupRepository.AddAsync(meetingGroup);

        return Unit.Value;
    }
}

Queries are processed using Read Model which is implemented by executing raw SQL statements on database views:

internal class GetAllMeetingGroupsQueryHandler : IQueryHandler<GetAllMeetingGroupsQuery, List<MeetingGroupDto>>
{
    private readonly ISqlConnectionFactory _sqlConnectionFactory;

    internal GetAllMeetingGroupsQueryHandler(ISqlConnectionFactory sqlConnectionFactory)
    {
        _sqlConnectionFactory = sqlConnectionFactory;
    }

    public async Task<List<MeetingGroupDto>> Handle(GetAllMeetingGroupsQuery request, CancellationToken cancellationToken)
    {
        var connection = _sqlConnectionFactory.GetOpenConnection();

        const string sql = "SELECT " +
                           "[MeetingGroup].[Id], " +
                           "[MeetingGroup].[Name], " +
                           "[MeetingGroup].[Description], " +
                           "[MeetingGroup].[LocationCountryCode], " +
                           "[MeetingGroup].[LocationCity]" +
                           "FROM [meetings].[v_MeetingGroups] AS [MeetingGroup]";
        var meetingGroups = await connection.QueryAsync<MeetingGroupDto>(sql);

        return meetingGroups.AsList();
    }
}

Key advantages:

Solution is appropriate to the problem - reading and writing needs are usually different
Supports Single Responsibility Principle (SRP) - one handler does one thing
Supports Interface Segregation Principle (ISP) - each handler implements interface with exactly one method
Supports Parameter Object pattern - Commands and Queries are objects which are easy to serialize/deserialize
Easy way to apply Decorator pattern to handle cross-cutting concerns
Supports Loose Coupling by use of the Mediator pattern - separates invoker of request from handler of request

Disadvantage:

Mediator pattern introduces extra indirection and is harder to reason about which class handles the request

For more information: Simple CQRS implementation with raw SQL and DDD

3.5 Domain Model Principles and Attributes

The Domain Model, which is the central and most critical part in the system, should be designed with special attention. Here are some key principles and attributes which are applied to Domain Models of each module:

High level of encapsulation

All members are private by default, then internal - only public at the very edge.
High level of PI (Persistence Ignorance)

No dependencies to infrastructure, databases, etc. All classes are POCOs.
Rich in behavior

All business logic is located in the Domain Model. No leaks to the application layer or elsewhere.
Low level of Primitive Obsession

Primitive attributes of Entites grouped together using ValueObjects.
Business language

All classes, methods and other members are named in business language used in this Bounded Context.
Testable

The Domain Model is a critical part of the system so it should be easy to test (Testable Design).

public class MeetingGroup : Entity, IAggregateRoot
{
    public MeetingGroupId Id { get; private set; }

    private string _name;

    private string _description;

    private MeetingGroupLocation _location;

    private MemberId _creatorId;

    private List<MeetingGroupMember> _members;

    private DateTime _createDate;

    private DateTime? _paymentDateTo;

    internal static MeetingGroup CreateBasedOnProposal(
        MeetingGroupProposalId meetingGroupProposalId,
        string name,
        string description,
        MeetingGroupLocation location, MemberId creatorId)
    {
        return new MeetingGroup(meetingGroupProposalId, name, description, location, creatorId);
    }

     public Meeting CreateMeeting(
            string title,
            MeetingTerm term,
            string description,
            MeetingLocation location,
            int? attendeesLimit,
            int guestsLimit,
            Term rsvpTerm,
            MoneyValue eventFee,
            List<MemberId> hostsMembersIds,
            MemberId creatorId)
        {
            this.CheckRule(new MeetingCanBeOrganizedOnlyByPayedGroupRule(_paymentDateTo));

            this.CheckRule(new MeetingHostMustBeAMeetingGroupMemberRule(creatorId, hostsMembersIds, _members));

            return new Meeting(this.Id,
                title,
                term,
                description,
                location,
                attendeesLimit,
                guestsLimit,
                rsvpTerm,
                eventFee,
                hostsMembersIds,
                creatorId);
        }

3.6 Cross-Cutting Concerns

To support Single Responsibility Principle and Don't Repeat Yourself principles, the implementation of cross-cutting concerns is done using the Decorator Pattern. Each Command processor is decorated by 3 decorators: logging, validation and unit of work.

Logging

The Logging decorator logs execution, arguments and processing of each Command. This way each log inside a processor has the log context of the processing command.

internal class LoggingCommandHandlerDecorator<T> : ICommandHandler<T> where T:ICommand
{
    private readonly ILogger _logger;
    private readonly IExecutionContextAccessor _executionContextAccessor;
    private readonly ICommandHandler<T> _decorated;

    public LoggingCommandHandlerDecorator(
        ILogger logger,
        IExecutionContextAccessor executionContextAccessor,
        ICommandHandler<T> decorated)
    {
        _logger = logger;
        _executionContextAccessor = executionContextAccessor;
        _decorated = decorated;
    }
    public async Task<Unit> Handle(T command, CancellationToken cancellationToken)
    {
        if (command is IRecurringCommand)
        {
            return await _decorated.Handle(command, cancellationToken);
        }
        using (
            LogContext.Push(
                new RequestLogEnricher(_executionContextAccessor),
                new CommandLogEnricher(command)))
        {
            try
            {
                this._logger.Information(
                    "Executing command {Command}",
                    command.GetType().Name);

                var result = await _decorated.Handle(command, cancellationToken);

                this._logger.Information("Command {Command} processed successful", command.GetType().Name);

                return result;
            }
            catch (Exception exception)
            {
                this._logger.Error(exception, "Command {Command} processing failed", command.GetType().Name);
                throw;
            }
        }
    }

    private class CommandLogEnricher : ILogEventEnricher
    {
        private readonly ICommand _command;

        public CommandLogEnricher(ICommand command)
        {
            _command = command;
        }
        public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
        {
            logEvent.AddOrUpdateProperty(new LogEventProperty("Context", new ScalarValue($"Command:{_command.Id.ToString()}")));
        }
    }

    private class RequestLogEnricher : ILogEventEnricher
    {
        private readonly IExecutionContextAccessor _executionContextAccessor;
        public RequestLogEnricher(IExecutionContextAccessor executionContextAccessor)
        {
            _executionContextAccessor = executionContextAccessor;
        }
        public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
        {
            if (_executionContextAccessor.IsAvailable)
            {
                logEvent.AddOrUpdateProperty(new LogEventProperty("CorrelationId", new ScalarValue(_executionContextAccessor.CorrelationId)));
            }
        }
    }
}

Validation

The Validation decorator performs Command data validation. It checks rules against Command arguments using the FluentValidation library.

internal class ValidationCommandHandlerDecorator<T> : ICommandHandler<T> where T:ICommand
{
    private readonly IList<IValidator<T>> _validators;
    private readonly ICommandHandler<T> _decorated;

    public ValidationCommandHandlerDecorator(
        IList<IValidator<T>> validators,
        ICommandHandler<T> decorated)
    {
        this._validators = validators;
        _decorated = decorated;
    }

    public Task<Unit> Handle(T command, CancellationToken cancellationToken)
    {
        var errors = _validators
            .Select(v => v.Validate(command))
            .SelectMany(result => result.Errors)
            .Where(error => error != null)
            .ToList();

        if (errors.Any())
        {
            var errorBuilder = new StringBuilder();

            errorBuilder.AppendLine("Invalid command, reason: ");

            foreach (var error in errors)
            {
                errorBuilder.AppendLine(error.ErrorMessage);
            }

            throw new InvalidCommandException(errorBuilder.ToString(), null);
        }

        return _decorated.Handle(command, cancellationToken);
    }
}

Unit Of Work

All Command processing has side effects. To avoid calling commit on every handler, UnitOfWorkCommandHandlerDecorator is used. It additionally marks InternalCommand as processed (if it is Internal Command) and dispatches all Domain Events (as part of Unit Of Work).

public class UnitOfWorkCommandHandlerDecorator<T> : ICommandHandler<T> where T:ICommand
{
    private readonly ICommandHandler<T> _decorated;
    private readonly IUnitOfWork _unitOfWork;
    private readonly MeetingsContext _meetingContext;

    public UnitOfWorkCommandHandlerDecorator(
        ICommandHandler<T> decorated,
        IUnitOfWork unitOfWork,
        MeetingsContext meetingContext)
    {
        _decorated = decorated;
        _unitOfWork = unitOfWork;
        _meetingContext = meetingContext;
    }

    public async Task<Unit> Handle(T command, CancellationToken cancellationToken)
    {
        await this._decorated.Handle(command, cancellationToken);

        if (command is InternalCommandBase)
        {
            var internalCommand =
                await _meetingContext.InternalCommands.FirstOrDefaultAsync(x => x.Id == command.Id,
                    cancellationToken: cancellationToken);

            if (internalCommand != null)
            {
                internalCommand.ProcessedDate = DateTime.UtcNow;
            }
        }

        await this._unitOfWork.CommitAsync(cancellationToken);

        return Unit.Value;
    }
}

3.7 Modules Integration

Integration between modules is strictly asynchronous using Integration Events and the In Memory Event Bus as broker. In this way coupling between modules is minimal and exists only on the structure of Integration Events.

Modules don't share data so it is not possible nor desirable to create a transaction which spans more than one module. To ensure maximum reliability, the Outbox / Inbox pattern is used. This pattern provides accordingly "At-Least-Once delivery" and "At-Least-Once processing".

The Outbox and Inbox is implemented using two SQL tables and a background worker for each module. The background worker is implemented using the Quartz.NET library.

Saving to Outbox:

Processing Outbox:

3.8 Internal Processing

The main principle of this system is that you can change its state only by calling a specific Command.

Commands can be called not only by the API, but by the processing module itself. The main use case which implements this mechanism is data processing in eventual consistency mode when we want to process something in a different process and transaction. This applies, for example, to Inbox processing because we want to do something (calling a Command) based on an Integration Event from the Inbox.

This idea is taken from Alberto's Brandolini's Event Storming picture called "The picture that explains “almost” everything" which shows that every side effect (domain event) is created by invoking a Command on Aggregate. See EventStorming cheat sheet article for more details.

Implementation of internal processing is very similar to implementation of the Outbox and Inbox. One SQL table and one background worker for processing. Each internally processing Command must inherit from InternalCommandBase class:

internal abstract class InternalCommandBase : ICommand
{
    public Guid Id { get; }

    protected InternalCommandBase(Guid id)
    {
        this.Id = id;
    }
}

This is important because the UnitOfWorkCommandHandlerDecorator must mark an internal Command as processed during committing:

public async Task<Unit> Handle(T command, CancellationToken cancellationToken)
{
    await this._decorated.Handle(command, cancellationToken);

    if (command is InternalCommandBase)
    {
        var internalCommand =
            await _meetingContext.InternalCommands.FirstOrDefaultAsync(x => x.Id == command.Id,
                cancellationToken: cancellationToken);

        if (internalCommand != null)
        {
            internalCommand.ProcessedDate = DateTime.UtcNow;
        }
    }

    await this._unitOfWork.CommitAsync(cancellationToken);

    return Unit.Value;
}

3.9 Security

Authentication

Authentication is implemented using JWT Token and Bearer scheme using IdentityServer. For now, only one authentication method is implemented: forms style authentication (username and password) via the OAuth2 Resource Owner Password Grant Type. It requires implementation of the IResourceOwnerPasswordValidator interface:

public class ResourceOwnerPasswordValidator : IResourceOwnerPasswordValidator
{
    private readonly IUserAccessModule _userAccessModule;

    public ResourceOwnerPasswordValidator(IUserAccessModule userAccessModule)
    {
        _userAccessModule = userAccessModule;
    }

    public async Task ValidateAsync(ResourceOwnerPasswordValidationContext context)
    {
        var authenticationResult = await _userAccessModule.ExecuteCommandAsync(new AuthenticateCommand(context.UserName, context.Password));
        if (!authenticationResult.IsAuthenticated)
        {
            context.Result = new GrantValidationResult(
                TokenRequestErrors.InvalidGrant,
                authenticationResult.AuthenticationError);
            return;
        }
        context.Result = new GrantValidationResult(
            authenticationResult.User.Id.ToString(),
            "forms",
            authenticationResult.User.Claims);
    }
}

Authorization

Authorization is achieved by implementing RBAC (Role Based Access Control) using Permissions. Permissions are more granular and a much better way to secure your application than Roles alone. Each User has a set of Roles and each Role contains one or more Permission. The User's set of Permissions is extracted from all Roles the User belongs to. Permissions are always checked on Controller level - never Roles:

[HttpPost]
[Route("")]
[HasPermission(MeetingsPermissions.ProposeMeetingGroup)]
public async Task<IActionResult> ProposeMeetingGroup(ProposeMeetingGroupRequest request)
{
    await _meetingsModule.ExecuteCommandAsync(
        new ProposeMeetingGroupCommand(
            request.Name,
            request.Description,
            request.LocationCity,
            request.LocationCountryCode));

    return Ok();
}

3.10 Unit Tests

Definition:

A unit test is an automated piece of code that invokes the unit of work being tested, and then checks some assumptions about a single end result of that unit. A unit test is almost always written using a unit testing framework. It can be written easily and runs quickly. It’s trustworthy, readable, and maintainable. It’s consistent in its results as long as production code hasn’t changed. Art of Unit Testing 2nd Edition Roy Osherove

Attributes of good unit test

Automated
Maitainable
Runs very fast (in ms)
Consistent, Deterministic (always the same result)
Isolated from other tests
Readable
Can be executed by anyone
Testing public API, not internal behavior (overspecification)
Looks like production code
Treated as production code

Implementation

Unit tests should mainly test business logic (domain model):

Each unit test has 3 standard sections: Arrange, Act and Assert:

1. Arrange

The Arrange section is responsible for preparing the Aggregate for testing the public method that we want to test. This public method is often called (from the unit tests perspective) the SUT (system under test).

Creating an Aggregate ready for testing involves calling one or more other public constructors/methods on the Domain Model. At first it may seem that we are testing too many things at the same time, but this is not true. We need to be one hundred percent sure that the Aggregate is in a state exactly as it will be in production. This can only be ensured when we:

Use only public API of Domain Model
Don't use InternalsVisibleToAttribute class
- This exposes the Domain Model to the Unit Tests library, removing encapsulation so our tests and production code are treated differently and it is a very bad thing
Don't use ConditionalAttribute classes - it reduces readability and increases complexity
Don't create any special constructors/factory methods for tests (even with conditional compilation symbols)
- Special constructor/factory method only for unit tests causes duplication of business logic in the test itself and focuses on state - this kind of approach causes the test to be very sensitive to changes and hard to maintain
Don't remove encapsulation from Domain Model (for example: change keywords from internal/private to public)
Don't make methods protected to inherit from tested class and in this way provide access to internal methods/properties

Isolation of external dependencies

There are 2 main concepts - stubs and mocks:

A stub is a controllable replacement for an existing dependency (or collaborator) in the system. By using a stub, you can test your code without dealing with the dependency directly.

A mock object is a fake object in the system that decides whether the unit test has passed or failed. It does so by verifying whether the object under test called the fake object as expected. There’s usually no more than one mock per test. Art of Unit Testing 2nd Edition Roy Osherove

Good advice: use stubs if you need to, but try to avoid mocks. Mocking causes us to test too many internal things and leads to overspecification.

2. Act

This section is very easy - we execute exactly one public method on aggregate (SUT).

3. Assert

In this section we check expectations. There are only 2 possible outcomes:

Method completed and Domain Event(s) published
Business rule was broken

Simple example:

[Test]
public void NewUserRegistration_WithUniqueLogin_IsSuccessful()
{
    // Arrange
    var usersCounter = Substitute.For<IUsersCounter>();

    // Act
    var userRegistration =
        UserRegistration.RegisterNewUser(
            "login", "password", "test@email",
            "firstName", "lastName", usersCounter);

    // Assert
    var newUserRegisteredDomainEvent = AssertPublishedDomainEvent<NewUserRegisteredDomainEvent>(userRegistration);
    Assert.That(newUserRegisteredDomainEvent.UserRegistrationId, Is.EqualTo(userRegistration.Id));
}

[Test]
public void NewUserRegistration_WithoutUniqueLogin_BreaksUserLoginMustBeUniqueRule()
{
    // Arrange
    var usersCounter = Substitute.For<IUsersCounter>();
    usersCounter.CountUsersWithLogin("login").Returns(x => 1);

    // Assert
    AssertBrokenRule<UserLoginMustBeUniqueRule>(() =>
    {
        // Act
        UserRegistration.RegisterNewUser(
            "login", "password", "test@email",
            "firstName", "lastName", usersCounter);
    });
}

Advanced example:

[Test]
public void AddAttendee_WhenMemberIsAlreadyAttendeeOfMeeting_IsNotPossible()
{
    // Arrange
    var creatorId = new MemberId(Guid.NewGuid());
    var meetingTestData = CreateMeetingTestData(new MeetingTestDataOptions
    {
        CreatorId = creatorId
    });
    var newMemberId = new MemberId(Guid.NewGuid());
    meetingTestData.MeetingGroup.JoinToGroupMember(newMemberId);
    meetingTestData.Meeting.AddAttendee(meetingTestData.MeetingGroup, newMemberId, 0);

    // Assert
    AssertBrokenRule<MemberCannotBeAnAttendeeOfMeetingMoreThanOnceRule>(() =>
    {
        // Act
        meetingTestData.Meeting.AddAttendee(meetingTestData.MeetingGroup, newMemberId, 0);
    });
}

CreateMeetingTestData method is an implementation of SUT Factory described by Mark Seemann which allows keeping common creation logic in one place:

protected MeetingTestData CreateMeetingTestData(MeetingTestDataOptions options)
{
    var proposalMemberId = options.CreatorId ?? new MemberId(Guid.NewGuid());
    var meetingProposal = MeetingGroupProposal.ProposeNew(
        "name", "description",
        new MeetingGroupLocation("Warsaw", "PL"), proposalMemberId);

    meetingProposal.Accept();

    var meetingGroup = meetingProposal.CreateMeetingGroup();

    meetingGroup.UpdatePaymentInfo(DateTime.Now.AddDays(1));

    var meetingTerm = options.MeetingTerm ??
                      new MeetingTerm(DateTime.UtcNow.AddDays(1), DateTime.UtcNow.AddDays(2));

    var rsvpTerm = options.RvspTerm ?? Term.NoTerm;
    var meeting = meetingGroup.CreateMeeting("title",
        meetingTerm,
        "description",
        new MeetingLocation("Name", "Address", "PostalCode", "City"),
        options.AttendeesLimit,
        options.GuestsLimit,
        rsvpTerm,
        MoneyValue.Zero,
        new List<MemberId>(),
        proposalMemberId);

    DomainEventsTestHelper.ClearAllDomainEvents(meetingGroup);

    return new MeetingTestData(meetingGroup, meeting);
}

3.11 Architecture Decision Log

All Architectural Decisions (AD) are documented in the Architecture Decision Log (ADL).

More information about documenting architecture-related decisions in this way : https://github.com/joelparkerhenderson/architecture_decision_record

3.12 Architecture Unit Tests

In some cases it is not possible to enforce the application architecture, design or established conventions using compiler (compile-time). For this reason, code implementations can diverge from the original design and architecture. We want to minimize this behavior, not only by code review.

To do this, unit tests of system architecture, design, major conventions and assumptions have been written. In .NET there is special library for this task: NetArchTest. This library has been written based on the very popular JAVA architecture unit tests library - ArchUnit.

Using this kind of tests we can test proper layering of our application, dependencies, encapsulation, immutability, DDD correct implementation, naming, conventions and so on - everything what we need to test. Example:

More information about architecture unit tests here: https://blogs.oracle.com/javamagazine/unit-test-your-architecture-with-archunit

3.13 Integration Tests

Definition

"Integration Test" term is blurred. It can mean test between classes, modules, services, even systems - see this article (by Martin Fowler).

For this reason, the definition of integration test in this project is as follows:

it verifies how system works in integration with "out-of-process" dependencies - database, messaging system, file system or external API
it tests particular use case
it can be slow (as opposed to Unit Test)

Approach

Do not mock dependencies over which you have full control (like database). Full control dependency means you can always revert all changes (remove side-effects) and no one can notice it. They are not visible to others. See next point, please.
Use "production", normal, real database version. Some use e.g. in memory repository, some use light databases instead "production" version. This is still mocking. Testing makes sense if we have full confidence in testing. You can't trust the test if you know that the infrastructure in the production environment will vary. Be always as close to production environment as possible.
Mock dependencies over which you don't have control. No control dependency means you can't remove side-effects after interaction with this dependency (external API, messaging system, SMTP server etc.). They can be visible to others.

Implementation

Integration test should test exactly one use case. One use case is represented by one Command/Query processing so CommandHandler/QueryHandler in Application layer is perfect starting point for running the Integration Test:

For each test, the following preparation steps must be performed:

Clear database
Prepare mocks
Initialize testing module

[SetUp]
public async Task BeforeEachTest()
{
    const string connectionStringEnvironmentVariable =
        "ASPNETCORE_MyMeetings_IntegrationTests_ConnectionString";
    ConnectionString = Environment.GetEnvironmentVariable(connectionStringEnvironmentVariable, EnvironmentVariableTarget.Machine);
    if (ConnectionString == null)
    {
        throw new ApplicationException(
            $"Define connection string to integration tests database using environment variable: {connectionStringEnvironmentVariable}");
    }

    using (var sqlConnection = new SqlConnection(ConnectionString))
    {
        await ClearDatabase(sqlConnection);
    }

    Logger = Substitute.For<ILogger>();
    EmailSender = Substitute.For<IEmailSender>();
    EventsBus = new EventsBusMock();
    ExecutionContext = new ExecutionContextMock(Guid.NewGuid());
    
    PaymentsStartup.Initialize(
        ConnectionString,
        ExecutionContext,
        Logger,
        EventsBus,
        false);

    PaymentsModule = new PaymentsModule();
}

After preparation, test is performed on clear database. Usually, it is the execution of some (or many) Commands and:
a) running a Query or/and
b) verifying mocks
to check the result.

[TestFixture]
public class MeetingPaymentTests : TestBase
{
    [Test]
    public async Task CreateMeetingPayment_Test()
    {
        PayerId payerId = new PayerId(Guid.NewGuid());
        MeetingId meetingId = new MeetingId(Guid.NewGuid());
        decimal value = 100;
        string currency = "EUR";
        await PaymentsModule.ExecuteCommandAsync(new CreateMeetingPaymentCommand(Guid.NewGuid(),
            payerId, meetingId, value, currency));

        var payment = await PaymentsModule.ExecuteQueryAsync(new GetMeetingPaymentQuery(meetingId.Value, payerId.Value));

        Assert.That(payment.PayerId, Is.EqualTo(payerId.Value));
        Assert.That(payment.MeetingId, Is.EqualTo(meetingId.Value));
        Assert.That(payment.FeeValue, Is.EqualTo(value));
        Assert.That(payment.FeeCurrency, Is.EqualTo(currency));
    }
}

Each Command/Query processing is a separate execution (with different object graph resolution, context, database connection etc.) thanks to Composition Root of each module. This behavior is important and desirable.

3.14 System Integration Testing

Definition

System Integration Testing (SIT) is performed to verify the interactions between the modules of a software system. It involves the overall testing of a complete system of many subsystem components or elements.

Implementation

Implementation of system integration tests is based on approach of integration testing of modules in isolation (invoking commands and queries) described in the previous section.

The problem is that in this case we are dealing with asynchronous communication. Due to asynchrony, our test must wait for the result at certain times.

To correctly implement such tests, the Sampling technique and implementation described in the Growing Object-Oriented Software, Guided by Tests book was used:

An asynchronous test must wait for success and use timeouts to detect failure. This implies that every tested activity must have an observable effect: a test must affect the system so that its observable state becomes different. This sounds obvious but it drives how we think about writing asynchronous tests. If an activity has no observable effect, there is nothing the test can wait for, and therefore no way for the test to synchronize with the system it is testing. There are two ways a test can observe the system: by sampling its observable state or by listening for events that it sends out.

Test below:

Creates Meeting Group Proposal in Meetings module
Waits until Meeting Group Proposal to verification will be available in Administration module with 10 seconds timeout
Accepts Meeting Group Proposal in Administration module
Waits until Meeting Group is created in Meetings module with 15 seconds timeout

public class CreateMeetingGroupTests : TestBase
{
    [Test]
    public async Task CreateMeetingGroupScenario_WhenProposalIsAccepted()
    {
        var meetingGroupId = await MeetingsModule.ExecuteCommandAsync(
            new ProposeMeetingGroupCommand("Name",
            "Description",
            "Location",
            "PL"));

        AssertEventually(
            new GetMeetingGroupProposalFromAdministrationProbe(meetingGroupId, AdministrationModule), 
            10000);

        await AdministrationModule.ExecuteCommandAsync(new AcceptMeetingGroupProposalCommand(meetingGroupId));

        AssertEventually(
            new GetCreatedMeetingGroupFromMeetingsProbe(meetingGroupId, MeetingsModule),
            15000);
    }

    private class GetCreatedMeetingGroupFromMeetingsProbe : IProbe
    {
        private readonly Guid _expectedMeetingGroupId;

        private readonly IMeetingsModule _meetingsModule;

        private List<MeetingGroupDto> _allMeetingGroups;

        public GetCreatedMeetingGroupFromMeetingsProbe(
            Guid expectedMeetingGroupId, 
            IMeetingsModule meetingsModule)
        {
            _expectedMeetingGroupId = expectedMeetingGroupId;
            _meetingsModule = meetingsModule;
        }

        public bool IsSatisfied()
        {
            return _allMeetingGroups != null && 
                   _allMeetingGroups.Any(x => x.Id == _expectedMeetingGroupId);
        }

        public async Task SampleAsync()
        {
            _allMeetingGroups = await _meetingsModule.ExecuteQueryAsync(new GetAllMeetingGroupsQuery());
        }

        public string DescribeFailureTo() 
            => $"Meeting group with ID: {_expectedMeetingGroupId} is not created";
    }

    private class GetMeetingGroupProposalFromAdministrationProbe : IProbe
    {
        private readonly Guid _expectedMeetingGroupProposalId;

        private MeetingGroupProposalDto _meetingGroupProposal;

        private readonly IAdministrationModule _administrationModule;

        public GetMeetingGroupProposalFromAdministrationProbe(Guid expectedMeetingGroupProposalId, IAdministrationModule administrationModule)
        {
            _expectedMeetingGroupProposalId = expectedMeetingGroupProposalId;
            _administrationModule = administrationModule;
        }

        public bool IsSatisfied()
        {
            if (_meetingGroupProposal == null)
            {
                return false;
            }

            if (_meetingGroupProposal.Id == _expectedMeetingGroupProposalId &&
                _meetingGroupProposal.StatusCode == MeetingGroupProposalStatus.ToVerify.Value)
            {
                return true;
            }

            return false;
        }

        public async Task SampleAsync()
        {
            try
            {
                _meetingGroupProposal =
                    await _administrationModule.ExecuteQueryAsync(
                        new GetMeetingGroupProposalQuery(_expectedMeetingGroupProposalId));
            }
            catch
            {
                // ignored
            }
        }

        public string DescribeFailureTo()
            => $"Meeting group proposal with ID: {_expectedMeetingGroupProposalId} to verification not created";
    }
}

Poller class implementation (based on example in the book):

public class Poller
{
    private readonly int _timeoutMillis;

    private readonly int _pollDelayMillis;

    public Poller(int timeoutMillis)
    {
        _timeoutMillis = timeoutMillis;
        _pollDelayMillis = 1000;
    }

    public void Check(IProbe probe)
    {
        var timeout = new Timeout(_timeoutMillis);
        while (!probe.IsSatisfied())
        {
            if (timeout.HasTimedOut())
            {
                throw new AssertErrorException(DescribeFailureOf(probe));
            }

            Thread.Sleep(_pollDelayMillis);
            probe.SampleAsync();
        }
    }

    private static string DescribeFailureOf(IProbe probe)
    {
        return probe.DescribeFailureTo();
    }
}

4. Technology

List of technologies, frameworks and libraries used for implementation:

.NET Core 3.1 (platform). Note for Visual Studio users: VS 2019 is required.
MS SQL Server Express (database)
Entity Framework Core 3.1 (ORM Write Model implementation for DDD)
Autofac (Inversion of Control Container)
IdentityServer4 (Authentication and Authorization)
Serilog (structured logging)
Hellang.Middleware.ProblemDetails (API Problem Details support)
Swashbuckle (Swagger automated documentation)
Dapper (micro ORM for Read Model)
Newtonsoft.Json (serialization/deserialization to/from JSON)
Quartz.NET (background processing)
FluentValidation (data validation)
MediatR (mediator implementation)
Postman (API tests)
NUnit (Testing framework)
NSubstitute (Testing isolation framework)
Visual Paradigm Community Edition (CASE tool for modeling and documentation)
NetArchTest (Architecture Unit Tests library)

5. How to Run

Download and install .NET Core 3.1 SDK
Download and install MS SQL Server Express or other
Create an empty database and run InitializeDatabase.sql script
- 2 test users will be created - check the script for usernames and passwords
Set a database connection string called MeetingsConnectionString in the root of the API project's appsettings.json or use Secrets

Example config setting in appsettings.json for a database called ModularMonolith:
```
{
    "MeetingsConnectionString": "Server=(localdb)\\mssqllocaldb;Database=ModularMonolith;Trusted_Connection=True;"
}
```
Set the Startup Item in your IDE to the API Project, not IIS Express
Once it is running you'll need a token to make API calls. This is done via OAuth2 Resource Owner Password Grant Type. By default IdentityServer is configured with the following:
- client_id = ro.client
- client_secret = secret (this is literally the value - not a statement that this value is secret!)
- scope = myMeetingsAPI openid profile
- grant_type = password
Include the credentials of a test user created in the InitializeDatabase.sql script - for example:
- username = testMember@mail.com
- password = testMemberPass

Example HTTP Request for an Access Token:

POST /connect/token HTTP/1.1
Host: localhost:5000
 
grant_type=password
&username=testMember@mail.com
&password=testMemberPass
&client_id=ro.client
&client_secret=secret

This will fetch an access token for this user to make authorized API requests using the HTTP request header Authorization: Bearer <access_token>

If you use a tool such as Postman to test your API, the token can be fetched and stored within the tool itself and appended to all API calls. Check your tool documentation for instructions.

6. Contribution

This project is still under analysis and development. I assume its maintenance for a long time and I would appreciate your contribution to it. Please let me know by creating an Issue or Pull Request.

7. Roadmap

List of features/tasks/approaches to add:

Name	Priority	Status	Release date
Domain Model Unit Tests	High	Completed	2019-09-10
Architecture Decision Log update	High	Completed	2019-11-09
Integration automated tests	Normal	Completed	2020-02-24
Migration to .NET Core 3.1	Low	Completed	2020-03-04
System Integration Testing	Normal	Completed	2020-03-28
API automated tests	Normal
FrontEnd SPA application	Normal
Meeting comments feature	Low
Notifications feature	Low
Messages feature	Low
More advanced Payments module	Low

NOTE: Please don't hesitate to suggest something else or a change to the existing code. All proposals will be considered.

8. Author

Kamil Grzybek

Blog: https://kamilgrzybek.com

Twitter: https://twitter.com/kamgrzybek

LinkedIn: https://www.linkedin.com/in/kamilgrzybek/

GitHub: https://github.com/kgrzybek

9. License

The project is under MIT license.

10. Inspirations and Recommendations