Skip to content

Instantly share code, notes, and snippets.

@ptzagk

ptzagk/EntraID Synchronization System.md

Last active April 16, 2025 17:07
Show Gist options
  • Save ptzagk/17ed3434051259f13979ec85180c15ee to your computer and use it in GitHub Desktop.
Save ptzagk/17ed3434051259f13979ec85180c15ee to your computer and use it in GitHub Desktop.
Download ZIP
EntraID Synchronization System
Raw
EntraID Synchronization System.md

EntraID Synchronization System

This document provides an overview of the EntraID synchronization process and its comprehensive logging system.

Table of Contents

Overview

The EntraID Synchronization System is designed to synchronize user accounts and role assignments between Microsoft EntraID (formerly Azure AD) and the application's database. The system operates in simulation mode, printing actions to the console instead of performing actual database operations, while maintaining detailed logs of all activities.

Components

1. EntraIDSyncManager

The EntraIDSyncManager class is the main component responsible for managing the synchronization process. It simulates database operations by printing actions to the console with [SIMULATION] tags.

Key features:

  • User creation simulation
  • User deactivation simulation
  • Role assignment simulation
  • Role removal simulation

2. SyncLogger

The SyncLogger class provides comprehensive logging functionality for the synchronization process. It interacts with the database to record all sync activities, allowing for detailed tracking and auditing.

Key features:

  • Sync session logging
  • User action logging
  • Role action logging
  • Detailed statistics and reporting

3. Database Structure

The logging system uses the following database tables:

  • SyncLog: Records synchronization sessions
  • SyncUserActionLog: Records user-related actions (creation, deactivation)
  • SyncRoleActionLog: Records role-related actions (assignment, removal)

Synchronization Process

The synchronization process follows these steps:

  1. Initialization:

    • Create a sync log entry in the database
    • Connect to EntraID and retrieve groups and users
    • Retrieve user domains and roles from the database

  2. Comparison:

    • Identify users in EntraID but not in the database
    • Identify users in the database but not in EntraID
    • Identify role discrepancies between EntraID and the database

  3. Synchronization:

    • Simulate creation of users that exist in EntraID but not in the database
    • Simulate deactivation of users that exist in the database but not in EntraID
    • Simulate role assignments and removals to resolve discrepancies

  4. Completion:

    • Update the sync log status
    • Generate statistics about the synchronization
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   EntraID API   │     │ EntraIDSyncMgr  │     │    Database     │
└────────┬────────┘     └────────┬────────┘     └────────┬────────┘
         │                       │                       │
         │  Fetch Groups/Users   │                       │
         │◄──────────────────────┤                       │
         │                       │                       │
         │  Return Data          │                       │
         ├──────────────────────►│                       │
         │                       │                       │
         │                       │  Create Sync Log      │
         │                       ├──────────────────────►│
         │                       │                       │
         │                       │  Get Users/Roles      │
         │                       ├──────────────────────►│
         │                       │                       │
         │                       │  Compare & Identify   │
         │                       │  Discrepancies        │
         │                       │                       │
         │                       │  Simulate Actions     │
         │                       │  & Log Activities     │
         │                       ├──────────────────────►│
         │                       │                       │
         │                       │  Update Sync Status   │
         │                       ├──────────────────────►│
         │                       │                       │

Logging System

Database Setup

The logging system requires several database tables and stored procedures, which are defined in SyncLoggerDbSetup.sql.

Tables

  1. SyncLog:

    • Records each synchronization session
    • Tracks start time, end time, status, and any errors

  2. SyncUserActionLog:

    • Records user-related actions (creation, deactivation)
    • Links to the parent sync log
    • Tracks action type, username, domain, status, and any errors

  3. SyncRoleActionLog:

    • Records role-related actions (assignment, removal)
    • Links to the parent sync log
    • Tracks action type, username, role name, status, and any errors

Stored Procedures

All stored procedures follow a consistent naming convention with the usp_Sync prefix:

  1. usp_SyncCreateLog:

    • Creates a new synchronization log entry

  2. usp_SyncUpdateLogStatus:

    • Updates the status of a synchronization log entry

  3. usp_SyncLogUserAction:

    • Logs a user action during synchronization

  4. usp_SyncUpdateUserActionStatus:

    • Updates the status of a user action log

  5. usp_SyncLogRoleAction:

    • Logs a role action during synchronization

  6. usp_SyncUpdateRoleActionStatus:

    • Updates the status of a role action log

  7. usp_SyncGetRecentLogs:

    • Gets recent sync logs

  8. usp_SyncGetLogDetail:

    • Gets detailed information about a specific sync log

  9. usp_SyncGetStats:

    • Gets statistics about synchronization activities

Log Flow

  1. Start Synchronization:

    • Create a sync log entry with status "Started"

  2. Log User Actions:

    • Before simulating a user action, create a user action log with status "Pending"
    • After simulating the action, update the status to "Completed" or "Failed"

  3. Log Role Actions:

    • Before simulating a role action, create a role action log with status "Pending"
    • After simulating the action, update the status to "Completed" or "Failed"

  4. Complete Synchronization:

    • Update the sync log status to "Completed" or "Failed"

Simulation Mode

The system operates in simulation mode, which means:

  1. No actual changes are made to the database
  2. All actions are printed to the console with [SIMULATION] tags
  3. The logging system still records all actions as if they were performed

This allows for testing and verification of the synchronization logic without affecting the production database.

Configuration

Environment Variables

The EntraID synchronization process uses the following environment variables:

  • DATABASE: Set to "mssql" to use SQL Server
  • ASPNETCORE_ENVIRONMENT: Development, Staging, or Production
  • TEST_DEFAULT_AZURE_CREDENTIAL: Set to "true" to test the database connection

EntraID Configuration

EntraID configuration is stored in the database Config table with the following entries:

  • AzureAd:ClientId: The client ID for the EntraID application
  • AzureAd:ClientSecret: The client secret for the EntraID application
  • AzureAd:TenantId: The tenant ID for the EntraID directory
  • ApplicationPrefix: The prefix used for mapping EntraID groups to application roles

Group to Role Mapping

EntraID groups are mapped to application roles using the following convention:

  1. EntraID groups that start with the ApplicationPrefix are considered for mapping
  2. The role name is derived by removing the prefix from the group name
  3. For example, if ApplicationPrefix is "App_", an EntraID group named "App_Admin" would map to the "Admin" role

Usage

Command-Line Execution

The synchronization process is executed by the SyncEntraID method in the ProgramSync.cs file. It can be triggered using the following command:

cd /path/to/auth-api
dotnet run -- sync

Scheduled Execution

To schedule the synchronization process to run automatically, you can use a task scheduler like cron (Linux) or Task Scheduler (Windows).

Example cron job (runs daily at 2 AM):

0 2 * * * cd /path/to/auth-api && dotnet run -- sync > /path/to/logs/sync_$(date +\%Y\%m\%d).log 2>&1

Monitoring and Reporting

Sync Statistics

The system provides built-in statistics through the SyncLogger.GetSyncStatsAsync method, which returns:

  • Total number of synchronizations
  • Number of successful synchronizations
  • Number of failed synchronizations
  • Count of user actions by type
  • Count of role actions by type

Recent Logs

You can view recent synchronization logs using the SyncLogger.GetRecentSyncLogsAsync method, which returns:

  • Sync log ID
  • Start time
  • End time
  • Status
  • Started by
  • Error message (if any)

Detailed Log Information

For detailed information about a specific synchronization, use the SyncLogger.GetSyncLogDetailAsync method, which returns:

  • Sync log details
  • User actions performed during the sync
  • Role actions performed during the sync

Troubleshooting

Common Issues

  1. Connection Issues:

    • Ensure the database connection string is correct
    • Verify that the EntraID credentials are valid
    • Check network connectivity to both the database and EntraID

  2. Authentication Issues:

    • Ensure the EntraID application has the necessary permissions
    • Verify that the client secret has not expired
    • Check if the tenant ID is correct

  3. Mapping Issues:

    • Ensure EntraID groups follow the naming convention with the correct prefix
    • Verify that the roles exist in the database
    • Check if the user domains are correctly configured

  4. Logging Issues:

    • Ensure the database tables and stored procedures are properly created
    • Check if the user running the process has the necessary database permissions
    • Verify that the log4net configuration is correct

Debugging

For detailed debugging, enable debug logging by setting the appropriate log level in the log4net configuration file.

Security Considerations

Credentials

  • Store EntraID credentials securely using environment variables or a secure vault
  • Avoid hardcoding credentials in the source code
  • Rotate client secrets regularly

Permissions

  • Use the principle of least privilege for both database and EntraID access
  • Ensure the EntraID application has only the necessary permissions
  • Restrict database access to only the required tables and procedures

Audit Logging

  • Review sync logs regularly for unauthorized or unexpected activities
  • Implement alerts for failed synchronization attempts
  • Consider integrating with a security information and event management (SIEM) system

Performance Considerations

Database Optimization

  • Create appropriate indexes on the logging tables
  • Consider archiving old log entries to maintain performance
  • Use parameterized queries to avoid SQL injection and improve query plan caching

Batch Processing

  • Process users and roles in batches to avoid memory issues with large datasets
  • Consider implementing pagination for large EntraID directories

Scheduling

  • Schedule synchronization during off-peak hours to minimize impact on system performance
  • Consider incremental synchronization for large directories

Future Enhancements

  1. Live Mode:

    • Implement a mode that performs actual database operations instead of simulation
    • Add a configuration option to switch between simulation and live modes

  2. Incremental Synchronization:

    • Implement delta synchronization to only process changes since the last sync
    • Use EntraID change tracking features to efficiently identify changes

  3. Web Interface:

    • Develop a web interface for monitoring and managing synchronization
    • Implement visualizations for sync statistics and trends

  4. Multi-Tenant Support:

    • Extend the system to support multiple EntraID tenants
    • Implement tenant-specific configuration and logging

Version History

Version Date Description
0.8.0 2025-04-16 Initial implementation with simulation mode
0.7.0 2025-04-10 Beta version with basic EntraID synchronization
0.5.0 2025-04-01 Alpha version with logging system implementation

Glossary

  • EntraID: Microsoft's cloud-based identity and access management service (formerly Azure AD)
  • Synchronization: The process of aligning user accounts and roles between EntraID and the database
  • Simulation Mode: A mode where actions are logged but not actually performed in the database
  • User Domain: A domain prefix used in usernames (e.g., "DOMAIN\username")
  • Auth Domain: The email domain associated with EntraID accounts (e.g., "example.com" in "[email protected]")
  • Role Discrepancy: A difference between the roles assigned in EntraID and the database
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment