Copy a whole Telegram channel to a local searchable log (Telethon user-account stack, Dockerized)

GIST-telegram-channel-copy.md

Copying a whole Telegram channel to a local, searchable log

A small Dockerized Telethon stack that logs into Telegram as your user account, finds a channel, and dumps its entire history to JSON — then turns that into a grep-able TSV, a timeline rollup, and a chunked transcript. Used to archive the Tevm Smithereans channel back to message #1.

Why a user account, not a bot

This trips people up: a Telegram bot cannot read a channel's history. Even after you add a bot to a channel, the Bot API only delivers messages posted after it joined, and only with privacy mode configured. To copy a channel from the beginning you log in as a user account that is a member of the channel — that account can page back through the full history. Telethon (MTProto) does exactly this. The "add our bot to the channel" step is irrelevant to the copy; it's the user session that does the work.

Use an account that's actually a member of the channel, and respect the channel's norms — this reads the same messages you can already scroll to by hand.

---

0. One-time: get API credentials

1. Go to https://my.telegram.org → API development tools.
2. Create an app (any name). Copy the api_id (int) and api_hash (hex string).
3. Note the phone number of the account you'll log in as (international format, e.g. +15551234567).

These are user-app credentials, distinct from a BotFather token.

1. Files

tg-logs/
  .env                 # secrets (gitignored)
  Dockerfile
  docker-compose.yml
  login.py             # interactive sign-in -> writes data/session.session
  find_channels.py     # list/filter your dialogs to get the channel id
  dump_messages.py     # the copy: all messages -> data/channel_<id>.json
  make_searchable.py    # json -> grep-friendly TSV
  timeline.py          # json -> day/week rollup with senders, keywords, links
  data/                # session + output (gitignored)

.env (your own values from my.telegram.org — never commit this):

TELEGRAM_API_ID=YOUR_API_ID
TELEGRAM_API_HASH=YOUR_API_HASH
TELEGRAM_PHONE=+15551234567

.gitignore — never commit these:

.env
data/session.session

The session.session file is a live login to your account. Treat it like a password.

Dockerfile:

FROM python:3.12-slim
RUN pip install telethon
WORKDIR /app
COPY *.py .
ENTRYPOINT ["python"]

docker-compose.yml:

services:
  tg:
    build: .
    env_file: .env
    volumes:
      - ./data:/data        # session + JSON output persist on the host
    entrypoint: ["python"]

Because the scripts are COPYd into the image, add --build when you change one: docker compose run --rm --build tg <script.py> ...

2. The scripts

login.py — sends a login code to the account, then signs in (handles 2FA). Writes data/session.session:

from telethon import TelegramClient
import os, sys
client = TelegramClient('/data/session', int(os.environ['TELEGRAM_API_ID']), os.environ['TELEGRAM_API_HASH'])
async def main():
    code, password = (sys.argv[1] if len(sys.argv) > 1 else None), (sys.argv[2] if len(sys.argv) > 2 else None)
    await client.connect()
    if await client.is_user_authorized():
        print("Already logged in as", (await client.get_me()).first_name); return
    await client.send_code_request(os.environ['TELEGRAM_PHONE'])
    if not code:
        print("Code sent! Run: login.py <code> [2fa_password]"); return
    try:
        await client.sign_in(os.environ['TELEGRAM_PHONE'], code)
    except Exception as e:
        if "SessionPasswordNeeded" in str(type(e)):
            await client.sign_in(password=password)     # 2FA
        else: raise
    print("Logged in as", (await client.get_me()).first_name)
client.loop.run_until_complete(main())

find_channels.py — lists every dialog (channel/group/user); optional substring filter. Gives you the numeric channel id:

from telethon import TelegramClient
import os, sys
client = TelegramClient('/data/session', int(os.environ['TELEGRAM_API_ID']), os.environ['TELEGRAM_API_HASH'])
async def main():
    query = sys.argv[1].lower() if len(sys.argv) > 1 else ''
    await client.connect()
    async for d in client.iter_dialogs():
        if query and query not in (d.title or '').lower(): continue
        kind = 'channel' if d.is_channel else 'group' if d.is_group else 'user'
        print(f"{d.id:>16}  {kind:<8}  {d.title}")
client.loop.run_until_complete(main())

dump_messages.py — the copy. Pages through the whole channel newest→oldest, reverses to chronological, writes JSON. get_sender() resolves display names (this is the slow part — Telethon round-trips per new sender):

from telethon import TelegramClient
import os, sys, json
client = TelegramClient('/data/session', int(os.environ['TELEGRAM_API_ID']), os.environ['TELEGRAM_API_HASH'])
async def main():
    channel_id = int(sys.argv[1])
    limit = None if (len(sys.argv) > 2 and sys.argv[2] == 'all') else int(sys.argv[2]) if len(sys.argv) > 2 else 100
    await client.connect()
    messages = []
    async for msg in client.iter_messages(channel_id, limit=limit):
        sender = await msg.get_sender()
        name = getattr(sender, 'first_name', '') or getattr(sender, 'title', 'Unknown')
        messages.append({
            'id': msg.id, 'date': msg.date.isoformat(), 'sender': name,
            'text': msg.text or '[media]',
            'reply_to': msg.reply_to.reply_to_msg_id if msg.reply_to else None,
            'topic_id': (getattr(msg.reply_to, 'reply_to_top_id', None) or
                         (msg.reply_to.reply_to_msg_id if getattr(msg.reply_to, 'forum_topic', False) else None)) if msg.reply_to else None,
        })
    messages.reverse()
    out = f"/data/channel_{channel_id}.json"
    json.dump(messages, open(out, 'w'), indent=2)
    print(f"Dumped {len(messages)} messages to {out}")
client.loop.run_until_complete(main())

make_searchable.py — JSON → one line per message, tab-separated (date \t sender \t text [re:id]), perfect for grep:

import json, sys
data = json.load(open(sys.argv[1]))
out = open(sys.argv[2], 'w') if len(sys.argv) > 2 else sys.stdout
for m in data:
    date = m['date'][:16].replace('T', ' ')
    text = (m['text'] or '[media]').replace('\n', ' ⏎ ')[:200]
    reply = f" [re:{m['reply_to']}]" if m.get('reply_to') else ''
    print(f"{date}\t{m['sender']}\t{text}{reply}", file=out)

(timeline.py is longer — it buckets by day, collapses quiet stretches into weeks, and prints top senders + keywords + links per period. Optional; grab it from the repo.)

3. Run it

cd tg-logs

# --- one-time login (writes data/session.session) ---
docker compose run --rm tg login.py                 # sends the code to the account
docker compose run --rm tg login.py 12345           # enter the code Telegram texts you
docker compose run --rm tg login.py 12345 my2fapass # only if the account has 2FA

# --- find the channel id ---
docker compose run --rm tg find_channels.py smither
#   -1002009589709  channel   Tevm Smithereans

# --- copy the ENTIRE channel ---
docker compose run --rm tg dump_messages.py -1002009589709 all
#   Dumped 75074 messages to /data/channel_-1002009589709.json
#   (a busy multi-year channel takes ~10-15 min — get_sender() per message is the bottleneck)

# --- make it usable ---
docker compose run --rm tg make_searchable.py /data/channel_-1002009589709.json /data/searchable.tsv
docker compose run --rm tg timeline.py        /data/channel_-1002009589709.json "Channel name" > data/timeline.md

Then it's just files:

grep -i "time travel"  data/searchable.tsv
grep -i "changelog"    data/searchable.tsv | grep fucory

4. Slicing / transcripts (plain Python on the JSON)

Everything downstream is host-side Python on channel_<id>.json — no Telegram calls. E.g. a date-bounded chunked transcript:

import json
from datetime import datetime, timedelta
from collections import defaultdict
d = [m for m in json.load(open('data/channel_-1002009589709.json')) if m['date'] >= '2026-05-18']
weeks = defaultdict(list)
for m in d:
    dt = datetime.fromisoformat(m['date'][:10]); monday = dt - timedelta(days=dt.weekday())
    weeks[monday.strftime('%Y-%m-%d')].append(m)
for i, (w, msgs) in enumerate(sorted(weeks.items()), 1):
    with open(f'week{i:02d}_{w}.md', 'w') as f:
        day = None
        for m in msgs:
            if m['date'][:10] != day: day = m['date'][:10]; f.write(f'\n## {day}\n\n')
            re = f' (re:{m["reply_to"]})' if m.get('reply_to') else ''
            f.write(f'[{m["date"][11:16]}] {m["sender"]}{re}: {m["text"]}\n')

5. Gotchas

• [media] ≠ "has media". dump_messages.py writes [media] whenever a message has no text. Some of those genuinely have no retrievable media (deleted/expired). To actually pull images/video, fetch by id and download_media():

  for msg in await client.get_messages(channel_id, ids=[77385, 78053]):
      if msg.media: await msg.download_media(file=f"/data/media/msg{msg.id}")

  If msg.media is None, it's gone — don't assume the dump lost it.
• Forum topics. Topic-organized supergroups carry reply_to.forum_topic; topic_id is captured so you can split by topic later. (list_topics.py enumerates them.)
• Speed. iter_messages is fast; the per-message get_sender() is what costs minutes. Telethon caches entities, so repeat senders are cheap — the cost is roughly the number of distinct senders, plus paging.
• Secrets. .env and data/session.session stay out of git. The session is a full login.

---

TODO: turn this into a Smithers workflow

This whole thing is a linear pipeline with one human gate (the login code) — a natural Smithers workflow: <Approval> for the login code → dump_messages task → fan-out make_searchable / timeline / transcript tasks → emit the artifacts. Someone should build it; this gist is the manual version.