NGC group history sync (specification and workflow)

NGC group history sync

specification and workflow

• trigger: tox_group_peer_join_cb()

each time a peer joins we ask that peer to send all non-private group chat messages
from 130 minutes ago up to now.

to make sure that peer is actually really online (tox has a bit of an issue there), wait n seconds before sending ngch_request

n = 9 + random(0 .. 7)

• ngch_request packet data:

what	Length in bytes	Contents
magic	6	0x667788113435
version	1	0x01
pkt id	1	0x01
sync delta	1	how many minutes back from now() to get messages allowed values from 5 to 130 minutes (both inclusive)

• trigger: tox_group_custom_private_packet_cb() and it's the ngch_request packet

the tox client needs to get authorization from the user that it's ok to
share group chat history with the requesting peer

then we send all non-private group chats messages from 130 minutes ago up to now
sorted by oldest (first) to newest (last)
each message is sent as a seperate lossless custom packet

wait n milliseconds seconds before sending the next message.

n = 300 + random(0 .. 300)

• ngch_syncmsg packet data:

MAX_GC_CUSTOM_PACKET_SIZE 40000

header_size = 6 + 1 + 1 + 4 + 32 + 4 + 25 = 73

max sync message text bytes = MAX_GC_CUSTOM_PACKET_SIZE - header_size = 39927

what	Length in bytes	Contents
magic	6	0x667788113435
version	1	0x01
pkt id	1	0x02
msg id	4	4 bytes message id for this group message
sender	32	32 bytes pubkey of the sender in the ngc group
timestamp	4	date/time when the message was received in unix timestamp format
name	25	sender name 25 bytes (cut off if longer, or right padded with 0x0 bytes)
message	39927	message text

• trigger: tox_group_custom_private_packet_cb() and it's the ngch_syncmsg packet

the tox client needs to sort these messages into the group chat view
since the dates can be in the past and newer messages can already be present in the current uservisible view.

• duplicate message check:

if a message with that msg id for this group chat and this sender within the last 300 days already exists,
then assume it is a duplicate and keep the current message and discard the incoming sync message.

• cutoff UTF-8

since name could have been cut at the maximum byte limit, the tox client needs to check for cut off UTF-8 and handle this accordingly.

mini review:

• seconds for timestamp is a bit too imprecise. multiple messages per second is very humanly possible and common
• looks like sync delta is an optional field...