riga

Event ID based splitting of 2024 MC samples into 2024, 2025 and 2026 parts

The snippets below demonstrate how to split MC events originally intended for the 2024 data-taking period into orthogonal parts that can be used to simulate the 2024, 2025 and 2026 data-taking periods. The splitting is done event-by-event and only depends on the event ID. The underlying algorithm is deterministic and splits events into three parts with frequencies that reflect the relative recorded luminosities in 2024, 2025 and 2026.

The unbiased behavior is strictly necessary to avoid that algorithms based on event IDs that are used downstream in analyses see a bias towards specific event IDs in any of the three years. For instance, event splitting for ML training / validation purposes, usually done via (e.g.) event % 2 == 0 or alike, should remain statistically unaffected. If there was a bias in the event ID distribution, this would lead to skewed training and validation datasets, which is not desirable.

riga

Event ID based splitting of 2024 MC samples into 2024, 2025 and 2026 parts

The snippets below demonstrate how to split MC events originally intended for the 2024 data-taking period into orthogonal parts that can be used to simulate the 2024, 2025 and 2026 data-taking periods. The splitting is done event-by-event and only depends on the event ID. The underlying algorithm is

• fast,
• deterministic (also across different working groups for synchronization purposes),
• unbiased towards the event ID, and
• splits events into three parts with frequencies that reflect the relative recorded luminosities in 2024, 2025 and 2026.

riga

/*
Interface for accessing Numpy's random number generators and configurable
bit generators via its C API. Usage example in main.cpp below.

Author: Marcel Rieger
*/

#include <string>

#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION

riga

# coding: utf-8

"""
Pseudo code for generating deterministic event and jet seeds,
to be used in random number generators with consistent output.

The seeds use the entire uint64 space and are proven to be
uniformly distributed using bijective hashing.

It requires a list of the first 200 primes plus some event

riga

riga

#!/usr/bin/env python
# coding: utf-8

# python imports
import os
import cherrypy


class Server(object):

riga

universe = vanilla
executable = job.sh
log = /dev/null
should_transfer_files = YES
transfer_input_files = job.sh
when_to_transfer_output = ON_EXIT
output = out.txt
error = err.txt
notification = Never
initialdir = /afs/cern.ch/user/m/mrieger/htctest

riga

# coding: utf-8

"""
Setup via
> pip install torch torchvision
"""

from __future__ import annotations

from contextlib import contextmanager

riga

# coding: utf-8


import law


class LongRunning(law.Task):

    def output(self):
        return law.LocalFileTarget("$PWD/data/long_running.json")

riga

# coding: utf-8

from __future__ import annotations

import os
from fnmatch import fnmatch
from multiprocessing import Pool as ProcessPool
from typing import Any

from tqdm import tqdm