Skip to content

Instantly share code, notes, and snippets.

@TheNetJedi
Created April 7, 2025 10:13
Show Gist options
  • Save TheNetJedi/0791b9222dc9825f73b1bfb8e9ed5ed4 to your computer and use it in GitHub Desktop.
Save TheNetJedi/0791b9222dc9825f73b1bfb8e9ed5ed4 to your computer and use it in GitHub Desktop.
This script fetches users from `https://gitlab.com` or self-hosted GitLab using the provided PAT and saves the relevant user details (ID, username, name, email, state, creation/activity dates, creator, URL, admin status, 2FA status) to `gitlab_com_users.csv`.
"""
GitLab Users to CSV Script
Purpose: Fetches user data from a GitLab instance API and saves it into a CSV file.
Prerequisites:
* Python 3 installed.
* `requests` library installed (`pip install requests`).
Usage:
1. Provide GitLab URL (Required):
Specify your GitLab instance URL using the `--gitlab-url` argument.
```bash
--gitlab-url https://your.gitlab.instance.com
```
2. Provide Personal Access Token (PAT) (Required):
You need a GitLab PAT with the `api` scope. Provide it in one of these ways:
* Command-line Argument (Recommended): Use the `--pat` argument.
This overrides the environment variable if both are set.
```bash
--pat 'YOUR_GITLAB_PAT'
```
* Environment Variable: Set the `GITLAB_PAT` environment variable.
```bash
# Linux/macOS
export GITLAB_PAT='YOUR_GITLAB_PAT'
# Windows (Command Prompt)
set GITLAB_PAT=YOUR_GITLAB_PAT
# Windows (PowerShell)
$env:GITLAB_PAT='YOUR_GITLAB_PAT'
```
3. Specify Output File (Optional):
By default, the output is saved to `gitlab_users.csv`. Change this with `--output-csv`.
```bash
--output-csv my_gitlab_users.csv
```
Example Command:
```bash
python gitlab_users_to_csv.py --gitlab-url https://gitlab.com --pat 'your_secret_token' --output-csv gitlab_com_users.csv
```
This command fetches users from `https://gitlab.com` using the provided PAT and saves the
relevant user details (ID, username, name, email, state, creation/activity dates, creator,
URL, admin status, 2FA status) to `gitlab_com_users.csv`.
"""
import requests
import csv
import os
import argparse
import sys
def fetch_gitlab_users(gitlab_url, pat):
"""Fetches all users from the GitLab API, handling pagination."""
users = []
headers = {"PRIVATE-TOKEN": pat}
# Ensure the URL ends with a slash
if not gitlab_url.endswith('/'):
gitlab_url += '/'
api_url = f"{gitlab_url}api/v4/users?per_page=100" # Request 100 users per page
print(f"Fetching users from {api_url}...")
while api_url:
try:
response = requests.get(api_url, headers=headers, timeout=30)
response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)
except requests.exceptions.RequestException as e:
print(f"Error fetching users: {e}", file=sys.stderr)
sys.exit(1)
current_page_users = response.json()
if not current_page_users:
break # No more users
users.extend(current_page_users)
print(f"Fetched {len(users)} users so far...")
# Check for the next page link in headers
if 'next' in response.links:
api_url = response.links['next']['url']
else:
api_url = None # No more pages
print(f"Finished fetching. Total users: {len(users)}")
return users
def write_users_to_csv(users, filename):
"""Writes a list of user dictionaries to a CSV file."""
if not users:
print("No users to write to CSV.")
return
# Define the desired fields in a logical order
fieldnames = ['id', 'username', 'name', 'email', 'state', 'created_at', 'last_activity_on', 'created_by', 'web_url', 'is_admin', 'two_factor_enabled']
# Filter users data to include only specified fieldnames and handle missing keys
filtered_users = []
for user in users:
user_data = {}
for field in fieldnames:
if field == 'created_by':
created_by_obj = user.get('created_by')
if isinstance(created_by_obj, dict):
name = created_by_obj.get('name', '')
username = created_by_obj.get('username', '')
# Format the string nicely
if name and username:
user_data[field] = f"{name} ({username})"
elif name:
user_data[field] = name
elif username:
user_data[field] = username
else:
user_data[field] = '' # Or perhaps fallback to ID if needed
else:
user_data[field] = '' # Not present or not a dict
else:
user_data[field] = user.get(field, '')
filtered_users.append(user_data)
print(f"Writing users to {filename}...")
try:
with open(filename, 'w', newline='', encoding='utf-8') as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(filtered_users)
print(f"Successfully wrote {len(filtered_users)} users to {filename}")
except IOError as e:
print(f"Error writing to CSV file {filename}: {e}", file=sys.stderr)
sys.exit(1)
def main():
parser = argparse.ArgumentParser(description="Fetch GitLab users and save to CSV.")
parser.add_argument("--gitlab-url", required=True, help="Your GitLab instance URL (e.g., https://gitlab.com)")
parser.add_argument("--output-csv", default="gitlab_users.csv", help="Output CSV file name (default: gitlab_users.csv)")
parser.add_argument("--pat", help="Your GitLab Personal Access Token (PAT). Overrides GITLAB_PAT environment variable.")
args = parser.parse_args()
gitlab_pat = args.pat or os.environ.get("GITLAB_PAT")
if not gitlab_pat:
print("Error: GitLab PAT not provided.", file=sys.stderr)
print("Please provide the PAT using the --pat argument or set the GITLAB_PAT environment variable.", file=sys.stderr)
sys.exit(1)
users = fetch_gitlab_users(args.gitlab_url, gitlab_pat)
write_users_to_csv(users, args.output_csv)
if __name__ == "__main__":
main()
@TheNetJedi
Copy link
Author

GitLab Users to CSV Script (gitlab_users_to_csv.py)

Purpose: Fetches user data from a GitLab instance API and saves it into a CSV file.

Prerequisites:

  • Python 3 installed.
  • requests library installed (pip install requests).

Usage:

  1. Provide GitLab URL (Required):
    Specify your GitLab instance URL using the --gitlab-url argument.

    --gitlab-url https://your.gitlab.instance.com
  2. Provide Personal Access Token (PAT) (Required):
    You need a GitLab PAT with the api scope. Provide it in one of these ways:

    • Command-line Argument (Recommended): Use the --pat argument. This overrides the environment variable if both are set.
      --pat 'YOUR_GITLAB_PAT'
    • Environment Variable: Set the GITLAB_PAT environment variable.
      # Linux/macOS
      export GITLAB_PAT='YOUR_GITLAB_PAT'
      # Windows (Command Prompt)
      set GITLAB_PAT=YOUR_GITLAB_PAT
      # Windows (PowerShell)
      $env:GITLAB_PAT='YOUR_GITLAB_PAT'
  3. Specify Output File (Optional):
    By default, the output is saved to gitlab_users.csv. Change this with --output-csv.

    --output-csv my_gitlab_users.csv

Example Command:

python gitlab_users_to_csv.py --gitlab-url https://gitlab.com --pat 'your_secret_token' --output-csv gitlab_com_users.csv

This command fetches users from https://gitlab.com using the provided PAT and saves the relevant user details (ID, username, name, email, state, creation/activity dates, creator, URL, admin status, 2FA status) to gitlab_com_users.csv.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment