-
Purpose:
Develop a Node.js application that runs once per day, retrieves events from a PostgreSQL database older than a configurable threshold (default one year), groups them by their UTC-created date, and exports each group as a CSV file. Each CSV is named in the formatevents_YYYY-MM-DD.csv
and stored in an Azure Data Lake under the foldertitan-pulse-event-archive
. After successful upload, the corresponding events are deleted from the database in one transaction. -
Deployment Environment:
The application will run in a Docker container (using a base imagenode:20-alpine3.18
) deployed as a Kubernetes CronJob scheduled daily at 02:00 AM UTC in an Azure Kubernetes Cluster.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def encode_intro_segment_core( | |
video_file_path: str, | |
duration: float = 300.0, # 5 minutes | |
video_height: int = 120, # Low resolution | |
video_crf: int = 28, # Quality setting | |
video_fps: int = 5, # Reduced frame rate | |
) -> Optional[str]: | |
"""Extract and encode the first 5 minutes of the video.""" | |
# Build optimized ffmpeg command |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Configurable quality settings | |
video_height = int(os.environ.get("VIDEO_HEIGHT", "120")) # Lower = faster | |
video_crf = int(os.environ.get("VIDEO_CRF", "28")) # Higher = smaller files | |
video_fps = int(os.environ.get("VIDEO_FPS", "5")) # Lower = fewer frames | |
# These settings reduce file sizes by 95%+ while maintaining detection accuracy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MOVIE_ANALYSIS_TOOLS = [ | |
get_video_duration, # Determine total video length | |
encode_intro_segment, # Create optimized intro clip | |
encode_outro_segment, # Create optimized outro clip | |
detect_intro_end_time, # AI analysis of intro segment | |
detect_outro_start_time, # AI analysis of outro segment | |
save_analysis_results, # Structured result storage | |
] | |
# The agent orchestrates these tools automatically |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"intro_end_time": "01:10", | |
"outro_start_time": "09:50", | |
"total_cost": 0.29665749999999996, | |
"analysis_timestamp": "2025-05-25T12:49:02.741502", | |
"video_file": { | |
"name": "tears_of_steel.mp4", | |
"path": "/Users/patrickkalkman/projects/st/credit-scout/tests/test_data/sample_video.mp4", | |
"size_bytes": 185765954 | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def detect_film_start(self, video_file: types.File) -> Dict[str, float]: | |
"""Detect when the main film content begins.""" | |
prompt = """Analyze this video clip from the beginning of a film. | |
Identify the exact timestamp (in MM:SS format) where the first scene | |
of the main, continuous narrative body of the film begins. | |
This point must occur after the full conclusion of ALL of the following: | |
- All studio logos and distributor cards | |
- The main title card of the film itself |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def encode_outro_segment_core( | |
video_file_path: str, | |
duration: float = 600.0, # 10 minutes | |
# ... other parameters | |
) -> Optional[str]: | |
"""Extract the final 10 minutes using ffmpeg's -sseof flag.""" | |
cmd = [ | |
"ffmpeg", "-y", | |
"-sseof", f"-{duration}", # Start from end minus duration |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
result | start_position | date | circuit | name | constructor | |
---|---|---|---|---|---|---|
1.0 | 1.0 | 2008-03-16 | Albert Park Grand Prix Circuit | Lewis Hamilton | McLaren | |
2.0 | 5.0 | 2008-03-16 | Albert Park Grand Prix Circuit | Nick Heidfeld | BMW Sauber | |
3.0 | 7.0 | 2008-03-16 | Albert Park Grand Prix Circuit | Nico Rosberg | Williams | |
4.0 | 12.0 | 2008-03-16 | Albert Park Grand Prix Circuit | Fernando Alonso | Renault | |
5.0 | 3.0 | 2008-03-16 | Albert Park Grand Prix Circuit | Heikki Kovalainen | McLaren |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Column Name | Data Type | Constraints | |
---|---|---|---|
id | INTEGER | UNIQUE | |
scan_id | INTEGER | UNIQUE | |
created | TEXT | ||
updated | TEXT | ||
progress | INTEGER | ||
high_alerts | INTEGER | ||
medium_alerts | INTEGER | ||
low_alerts | INTEGER | ||
info_alerts | INTEGER |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
resources: | |
requests: | |
cpu: 1500m | |
memory: 1024Mi | |
limits: | |
cpu: 8000m | |
memory: 2524Mi |
NewerOlder