Skip to content

Instantly share code, notes, and snippets.

@donal56
Last active October 21, 2023 05:23
Show Gist options
  • Save donal56/dc37300469cec5aa15d3deb0f12b3ad2 to your computer and use it in GitHub Desktop.
Save donal56/dc37300469cec5aa15d3deb0f12b3ad2 to your computer and use it in GitHub Desktop.
(agumation-utilities) Descarga videos del aguringu FC. Es necesario agregar los enlaces en links.txt y las credenciales del sitio junto con la api key de deepl en properties.js. Youtube-dl debe estar instalado y disponible en el path.
const exec = require('child_process').exec;
const deepl = require('deepl-node');
const puppeteer = require('puppeteer');
const URL = require('url').URL;
const fs = require('fs');
const properties = require('./properties.js');
// Datos del club
const baseUrl = "https://aguri-onishi.com";
const loginUrl = baseUrl + "/signin";
const user = properties.aguringu.user;
const pass = properties.aguringu.pass;
// Datos de DeepL
const apiKey = properties.deepl.apiKey;
const translator = new deepl.Translator(apiKey);
// Otras constantes
const streamingRegex = /{"akfire_interconnect_quic":{.*?"url":"(https:\/\/.*?\/master\.json\?.*?)"/m;
// Conf de descarga
const file = fs.readFileSync('links.txt', 'utf-8');
const urlsToDownload = file.split(/\r?\n/);
const timeout = 20_000;
// Formatos de consola
const styleReset = "\x1b[0m";
const stylePrimary = "\x1b[42m%s\x1b[37m";
const styleSuccess = "\x1b[4m%s\x1b[32m";
const styleError = "\x1b[4m%s\x1b[31m";
const styleMuted = "\x1b[2m%s\x1b[4m";
(async () => {
console.info(stylePrimary, "===RUNNING===", styleReset);
// Inicializar
const browser = await puppeteer.launch();
const page = await browser.newPage();
// Abrir sesión
await page.goto(loginUrl, {timeout: timeout});
await page.waitForSelector('#jsLoadingAnimation', {hidden: true});
await page.click("#user_login");
await page.keyboard.type(user);
await page.click("#user_password");
await page.keyboard.type(pass);
await Promise.all([
page.click(".signin-form + div input[type=submit]"),
page.waitForNavigation({waitUntil: "networkidle2", timeout: timeout})
]);
console.info("");
console.info(stylePrimary, "===LOGGED IN===", styleReset);
console.info("");
// Descargar cada video
for(index in urlsToDownload) {
// Generar título del video
await page.goto(urlsToDownload[index], {timeout: timeout});
const untranslatedTitle = await page.$eval(".title", el => el.textContent.trim().replace("_", " "));
const translationJob = await translator.translateText(untranslatedTitle, "JA", "EN-US");
const translation = translationJob.text.replace("\"", "'")
const date = await page.$eval(".time", el => el.textContent.replace(".", "_"));
// Generar enlace de descarga
const frameUrl = await page.$eval("iframe#player", el => el.src);
const command = "yt-dlp --referer " + baseUrl + " -o \"" + translation + " " + date + "\" \"" + frameUrl + "\"";
// Ejecutar descarga
const progress = (Number(index) + 1) + "/" + urlsToDownload.length;
await exec(command,
function(error, stdout, stderr) {
if(error !== null) {
console.err(styleError, " DOWNLOAD " + progress + " ERROR: " + error, styleReset);
}
else {
if(stdout.trim().endsWith("has already been downloaded")) {
console.info(styleMuted, " DOWNLOAD " + progress + " SKIPPED", styleReset);
}
else {
console.info(styleSuccess, " DOWNLOAD " + progress + " COMPLETED", styleReset);
}
}
}
);
}
await browser.close();
console.info("");
console.info(stylePrimary, "===DONE===", styleReset);
})();
//await page.screenshot({path: "ss_" + index + ".png"});
@donal56
Copy link
Author

donal56 commented Sep 6, 2023

Para poder descargar videos privados o embedidos de vimeo:

  1. Inspeccionar la red y buscar la petición .json que desencadena el stream. La solicitud es iniciada por el iframe del reproductor, el mismo URL esta dentro del codigo del iframe. P/E: https://xxxx.xxx/master.json?base64_init=1&query_string_ranges=1
  2. Remover el parámetro base64_init y cambiar la extensión a m3u8. P/E: https://xxxx.xxx/master.m3u8?query_string_ranges=1
  3. Si es necesario pasar las cookies con --cookies-from-browser, o la web origen con --referer. Opcionalmente --hls-prefer-native si se quiere usar el cliente de descarga HLS nativo en lugar de ffmpeg.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment