Merge branch 'develop' of github.com:ls1intum/Hephaestus into develop

docs/Makefile

@@ -4,9 +4,10 @@
 # You can set these variables from the command line, and also
 # from the environment for the first two.
 SPHINXAUTOBUILD = sphinx-autobuild
+
 ALLSPHINXLIVEOPTS = $(ALLSPHINXOPTS) -q \
    --port 0 \
-   --host 0.0.0.0 \
+   --host 127.0.0.1 \
    --open-browser \
    --delay 1 \
    --ignore "*.swp" \

docs/README.md

@@ -2,15 +2,23 @@
 
 ## Writing Documentation
 
-We are using [Sphinx](http://www.sphinx-doc.org/en/stable/) to generate the documentation. The documentation is written in [reStructuredText](http://docutils.sourceforge.net/rst.html) format.
+We are using [Sphinx](http://www.sphinx-doc.org/en/stable/) to generate the documentation. The documentation is written in Markdown format.
 
 ### Installation
 
 Optional: Create a virtual environment
 
 ```bash
 python3 -m venv venv
+```
+
+Activate the virtual environment:
+
+```bash
+# Unix:
 source venv/bin/activate
+# Windows:
+.\venv\Scripts\activate
 ```
 
 Install the required packages

docs/dev/coding_design_guidelines/client.md

@@ -0,0 +1,35 @@
+# Client
+
+This section contains guidelines for designing the client-side of the application. The client is built using modern web technologies and follows best practices for performance, accessibility, and user experience.
+
+## Client-Side Data Handling
+
+When working with data from the server, the client should use the auto-generated TypeScript interfaces from the OpenAPI specification. This ensures type safety and consistency between the client and server.
+
+Example:
+
+```typescript
+// This is auto-generated from the OpenAPI specification
+// Note that the DTO ending is omitted by the generator
+export interface Example {
+    name: string;
+}
+
+// example.component.ts
+@Component({
+    selector: 'app-example',
+    templateUrl: './example.component.html',
+})
+export class ExampleComponent {
+    // type-safe access to the server
+    exampleService = inject(ExampleService);
+
+    // Use Tanstack Query to fetch the actual data
+    // The data now inherits the DTO structure and can be accessed in a type-safe manner
+    example = injectQuery(() => ({
+        queryKey: ['example'],
+        // note that this will be correctly typed as Example
+        queryFn: async () => lastValueFrom(this.exampleService.getExampleByName('example')),
+    }));
+}
+``` 

docs/dev/coding_design_guidelines/database.md

@@ -0,0 +1,37 @@
+# Database
+
+This section contains guidelines to optimize the database schema and queries for performance and maintainability. Hephaestus uses a PostgreSQL database, which is accessed through the JPA (Java Persistence API) with Hibernate.
+
+## Database Queries
+
+When dealing with the database, it is important to optimize the queries to minimize the number of database calls and the amount of data retrieved. Use the `@Query` annotation to write custom queries in the repository classes and follow the following principles:
+
+1. **Only retrieve necessary data**: Filtering and smart data formatting as part of the query can help reduce the amount of data retrieved from the database.
+2. **Use pagination**: When retrieving a large number of records, use pagination to limit the number of records retrieved in a single query.
+3. **Follow the naming conventions**: To avoid writing unnecessary custom queries, make use of the pre-defined keywords in the method name instead. You can find a list of reserved words in the [Spring Data JPA documentation](https://docs.spring.io/spring-data/jpa/reference/jpa/query-methods.html).
+
+## Entity Design
+
+To define a new entity, create a new class with the `@Entity` annotation. The class should have a no-argument constructor and a primary key field annotated with `@Id`. The primary key field should be of type `Long` and should be auto-generated. For example:
+
+```java 
+@Entity
+@Table(name = "example")
+public class ExampleClass {
+
+    @Id
+    private Long id;
+
+    // Other fields and methods
+}
+```
+
+## Entity Fields
+
+- Use `@NonNull` to indicate that a field is required. Generally, this is a very desirable property and should be used unless there is a specific reason to allow the field to be `null`.
+- Use `@Id` for the primary ID key field. Every entity should have such a field.
+- If `String` fields can contain more than 255 characters, use the `@Lob` annotation to indicate that the field should be stored as a large object in the database. This is particularly important whenever fields are filled by an external source (e.g. the Github API).
+
+## Entity Relationships
+
+- Use `@ToString.Exclude` to exclude fields from the `toString`. This is particularly useful for relationships to avoid circular references. 

docs/dev/coding_design_guidelines/index.md

@@ -0,0 +1,47 @@
+# Coding and Design Guidelines
+
+These guidelines are intended to help maintain a consistent coding style and architectural design across the Hephaestus codebase. We distinguish between these main areas:
+
+```{toctree}
+:includehidden:
+:maxdepth: 1
+
+performance
+server
+client
+database
+```
+
+
+## Overview
+
+This document provides an overview of the coding and design guidelines for the Hephaestus project. For detailed guidelines on specific areas, please refer to the linked documents above.
+
+### Purpose
+
+The purpose of these guidelines is to:
+
+1. Ensure consistency across the codebase
+2. Improve code quality and maintainability
+3. Facilitate collaboration between team members
+4. Reduce the learning curve for new developers
+5. Optimize performance and user experience
+
+### How to Use These Guidelines
+
+These guidelines should be used as a reference when developing new features or modifying existing code. They are not meant to be strict rules but rather best practices that should be followed whenever possible.
+
+If you find that a guideline is not applicable in a specific situation, you can deviate from it, but you should document the reason for doing so in the code or in the pull request description.
+
+### Contributing to the Guidelines
+
+These guidelines are not set in stone and can be improved over time. If you have suggestions for improving the guidelines, please open a pull request with your proposed changes.
+
+When proposing changes to the guidelines, please consider the following:
+
+1. The impact on existing code
+2. The benefits of the proposed change
+3. The ease of adoption
+4. The alignment with industry best practices
+
+

docs/dev/coding_design_guidelines/performance.md

@@ -0,0 +1,29 @@
+# Performance Optimization
+
+This section contains guidelines for optimizing the performance of the Hephaestus application. Performance is a critical aspect of the user experience and should be considered at all stages of development.
+
+## General Performance Principles
+
+1. **Minimize network requests**: Reduce the number of HTTP requests by bundling resources, using caching, and optimizing API calls.
+2. **Optimize data transfer**: Only transfer the data that is needed for the current operation.
+3. **Use efficient algorithms**: Choose algorithms with appropriate time and space complexity for the task at hand.
+4. **Implement caching**: Use caching strategies to avoid redundant computations and data fetching.
+5. **Optimize database queries**: Write efficient database queries that retrieve only the necessary data and use appropriate indexes.
+
+## Specific Optimization Techniques
+
+### Frontend Optimizations
+
+- Use lazy loading for components and modules that are not immediately needed
+- Implement virtual scrolling for large lists
+- Optimize images and other assets
+- Use memoization for expensive computations
+- Implement proper state management to avoid unnecessary re-renders
+
+### Backend Optimizations
+
+- Use connection pooling for database connections
+- Implement caching for frequently accessed data
+- Use asynchronous processing for long-running tasks
+- Optimize database queries with proper indexing
+- Use pagination for large data sets 

docs/dev/coding_design_guidelines/server.md

@@ -0,0 +1,169 @@
+# Server
+
+This section contains guidelines for designing the server and API endpoints. Hephaestus uses Spring Boot to create a RESTful API, which is secured using JSON Web Tokens (JWT).
+
+## Folder Structure
+
+The Java server (commonly referred to as `application-server`) can be found under `server/application-server`. This folder contains the following main folders in `src/main`:
+
+- **resources**: Contains the compile-time configuration files for the server. 
+  - Most importantly, this includes the environment variables defined in `application-{env}.yml` files. Variables can be loaded according to the current environment, in development, the `application.yml` and `application-local.yml` files are used.
+  - The **/db**-folder contains the database schema and migrations. Refer to the Liquibase sections for more information.
+- **java**: Contains the Java source code for the server. Folders are organized by feature/functionality.
+  - **/config**: Contains the configuration classes for the server and external services.
+  - **/core**: Core elements of the server, such as exception and utility classes.
+  - **/gitprovider**: Database access and representation of the Github API and potential other git providers.
+  - **/intelligenceservice**: OpenAPI auto-generated code for the intelligence service. Used to communicate with the intelligence service and get type-safe responses.
+  - **/leaderboard**: Logic related to the leaderboard and correlated gamification features.
+  - **/mentor**: Database access and entities related to the mentor feature.
+  - **/meta**: General meta-data about the server accessible for other components and the client to consume. Mainly contains non-feature specific information, such as environment variables required for the client.
+  - **/syncing**: Logic related to the syncing of data between the server and the git providers.
+  - **/workspaces**: Classes related to workspace access and management.
+
+## Optionals
+
+Throughout the Java codebase, use `Optionals` instead of `null` to represent values that may be absent. This helps to avoid `NullPointerExceptions` and makes the code more readable. In many cases, Spring Boot returns them automatically. For example:
+
+```java
+// Finding a database row via the repository class
+@Repository
+public interface ExampleRepository extends JpaRepository<ExampleClass, Long> {
+
+    Optional<ExampleClass> findByName(String name);
+
+    // ...
+}
+
+// Returning optional values in a service class
+@Service
+public class ExampleService {
+
+    public Optional<ExampleClass> getExampleByName(String name) {
+        Optional<ExampleClass> example = exampleRepository.findByName(name);
+        if (example.isEmpty()) {
+            return Optional.empty();
+        }
+        ExampleClass exampleClass = example.get();
+        // Perform some operations...
+        return exampleClass;
+    }
+}
+```
+
+## Data Transfer Objects (DTOs)
+
+When designing API endpoints, use Data Transfer Objects (DTOs) to define the structure of the data exchanged between the client and the server. DTOs help to decouple the internal entity structure from the external API structure and provide a clear contract for the data exchanged.
+
+The client should receive data in the form of DTOs, which should be typed via the generated OpenAPI specification and modules (see [Client](./client.md#client-side-data-handling) version). Example:
+
+```java
+public record ExampleDTO(String name) {
+    public static ExampleDTO fromExampleClass(ExampleClass example) {
+        return new ExampleDTO(example.getName());
+    }
+}
+
+@RestController
+@RequestMapping("/api/example")
+public class ExampleController {
+
+    @GetMapping("/{id}")
+    public ResponseEntity<ExampleDTO> getExampleByName(@PathVariable String name) {
+        Optional<ExampleClass> example = exampleService.getExampleByName(name);
+        if (example.isEmpty()) {
+            return ResponseEntity.notFound().build();
+        }
+        ExampleDTO exampleDTO = ExampleDTO.fromEntity(example.get());
+        return ResponseEntity.ok(exampleDTO);
+    }
+
+}
+
+// Alternatively you can return DTOs directly from the repository. 
+// This is generally only useful for extremely specific requests.
+@Repository
+public interface ExampleRepository extends JpaRepository<ExampleClass, Long> {
+
+    @Query("""
+        SELECT new com.example.ExampleDTO(e.name) 
+        FROM ExampleClass e 
+        WHERE e.name = :name
+        """)
+    Optional<ExampleDTO> findByName(String name);
+
+    // ...
+}
+```
+
+**General Guidelines for DTOs:**
+
+- **Immutable Java Records**: Use Java's `record` syntax to guarantee immutability. While Java records preclude inheritance, resulting in potential duplication, this is considered acceptable in the context of DTOs to ensure data integrity and simplicity.
+- **Primitive data types and composition**: DTOs should strictly encapsulate primitive data types, their corresponding wrapper classes, enums, or compositions of other DTOs. This exclusion of entity objects from DTOs ensures that data remains decoupled from the database entities, facilitating a cleaner and more secure data transfer mechanism.
+- **Minimum necessary data**: Adhere to the principle of including only the minimal data required by the client within DTOs. This practice reduces the overall data footprint, enhances performance, and mitigates the risk of inadvertently exposing unnecessary or sensitive data.
+- **Single responsibility principle**: Each DTO should be dedicated to a specific task or subset of data. Avoid the temptation to reuse DTOs across different data payloads unless the data is identical. This approach maintains clarity and purpose within the data transfer objects.
+- **Simplicity over complexity**: Refrain from embedding methods or business logic within DTOs. Their role is to serve as straightforward data carriers without additional functionalities that could complicate their structure or purpose.
+
+## Dependency Injection
+
+- We prefer field injection using `@Autowired` over constructor injection. This avoids bloated constructors with many parameters and makes dependency injection more explicit through method parameters.
+- This means that it is important to manually check for and resolve circular dependencies. In general, the less dependencies a class has, the easier it is to reason about its behavior and the more maintainable it is.
+
+## Logging
+
+- Initialize a new logger of type `Logger` for each class.
+- Prefer structured logging (e.g. using `{}` placeholders) over traditional logging.
+- Keep logging at the DEBUG level for development purposes and minimize the amount of logging in production. Preferably only log errors.
+- If you need to customize the logging level for a specific class or external dependency, you can do so by setting the logging level in the `application-local.yml` file (see example below).
+
+Example for structured logging:
+
+```java
+// ExampleService.java
+@Service
+public class ExampleService {
+    // Make sure to use the class name as the logger name
+    private static final Logger logger = LoggerFactory.getLogger(ExampleService.class);
+
+    public void doSomething() {
+        logger.info("User {} logged in", user.getName());
+    }
+}
+```
+
+Example for custom logging levels:
+
+```yaml
+# application-local.yml
+logging:
+  level:
+    # Log all errors from the Github API
+    org.kohsuke.github.GithubClient: ERROR
+    # Log all errors from the Slack API
+    com.slack.api: ERROR
+```
+
+## Environment Variables
+
+In Spring Boot, environment variables are found in the `application-{env}.yml` files. Which files are loaded depends on the current environment, which can be set via the `SPRING_PROFILES_ACTIVE` execution parameter or via CLI arguments.
+
+The following `env` profiles are important:
+
+- **local**: Used for local development. This file is not committed to the repository and should contain all your secrets, such as API keys or tokens. You can use the `application.yml` file as a template and fill in the missing values.
+- **specs**: Used for the staging environment, such as Github Action executions. This file usually does not need to be touched.
+- **prod**: Used for the production environment. This file is committed to the repository and contains the production-relevant default values. These are overwritten by the actual environment variables of the deployment server.
+
+Per default, Spring Boot will load the `application.yml` file first and then the `application-{env}.yml` file. This means that values can be overwritten using more specific profiles. Omit variables to avoid this behavior.
+
+On UNIX-based systems, environment variables can also be set via the `export` command. This is generally only recommended for testing purposes as it makes it hard for other developers to reproduce your environment. For example:
+
+```bash
+# Set the active profile to local
+export SPRING_PROFILES_ACTIVE="local"
+# Set the Github API token
+export github.auth.token="..."
+```
+
+## Best Practices
+
+- **Avoid Transactional**: Use the `@Transactional` annotation sparingly. It is useful for operations that span multiple database queries, but is very performance-intensive and generally can be solved differently. Good read: https://codete.com/blog/5-common-spring-transactional-pitfalls/
+- **Format using Prettier**: We use Prettier as code formatter. The npm setup of the main Hephaestus folder can be used to format the Java code. The commands `npm run format:java:check` and `npm run format:java:write` can be used to check and format the Java code, respectively. Make sure to run these scripts regularly to avoid formatting issues. 

docs/dev/guidelines.rst

@@ -0,0 +1,10 @@
+.. toctree::
+   :caption: Coding and design guidelines
+   :includehidden:
+   :maxdepth: 3
+
+   coding_design_guidelines/database
+   coding_design_guidelines/performance
+   coding_design_guidelines/server
+   coding_design_guidelines/client
+   coding_design_guidelines/database

docs/index.md

@@ -37,6 +37,7 @@ user/getting_started
 dev/setup_guide
 dev/getting_started/index
 dev/system_design/index
+dev/coding_design_guidelines/index
 ```
 
 ```{toctree}