Why I built it
The bug that made me give up on existing forwarders.
I needed to mirror a few Telegram channels for backup and cross-posting. The first three open-source forwarders I tried (tgcf, telemirror, and a couple of smaller cloners) all worked for a few hours and then started posting duplicates to the destination.
The cause turned out to be the same in every tool. When more than one runner (a scheduler tick + a manual "run now" click, say) operates on the same pair, both load the full state dict from disk, each holds a stale snapshot, and the slower writer clobbers the faster one's watermark. Next cycle, the lagging key looks fresh again — and the tool happily re-forwards the same messages it forwarded an hour ago.
The fix is mechanical: don't write the whole dict, write only one key, under a lock, with read-modify-write. Refuse to write a value lower than what's on disk unless explicitly told to.
async def save_pair_watermark(name, last_msg_id, updated_at,
*, allow_regression=False):
async with _save_lock:
state = json.load(open(STATE_PATH)) if STATE_PATH.exists() else {}
if not allow_regression:
current = int(state.get(name, {}).get("last_msg_id", 0))
if last_msg_id < current:
return # refuse to roll backwards
state[name] = {"last_msg_id": last_msg_id, "updated_at": updated_at}
# atomic .tmp + replace
tmp = STATE_PATH.with_suffix(".tmp")
json.dump(state, open(tmp, "w"))
tmp.replace(STATE_PATH)That's the whole patch. It killed the duplicates permanently and made me realise the bigger forwarders had this bug too — they just hadn't hit a load high enough to expose it. Once I had a tool that didn't lie about its watermark, I kept adding the other things I wished existed: native batch forward, OOM-safe streaming for huge channels, live edit/delete propagation, a kill switch.
Then I open-sourced it. MIT, single Docker container, runs on a $5 VPS. If you've been chasing duplicate posts in your mirror channels, try it.