Telegram Bot

GistID: 9d5356c1ec9e7543e08684caa71006b1

Goals

Written in Rust
Unit Testable
Easy to add commands and keyboards
- Encoded as TOML
Keep a history
- A simple web interface with statistics and most recent commands and their status
- This would be helpful in overall command success, debugging
Run on any *nix platform

Rust

Crates and References

HTTP → https://github.com/hyperium/hyper
JSON → https://crates.io/crates/json
TOML → https://crates.io/crates/toml
Running shell commands → https://doc.rust-lang.org/std/process/struct.Command.html
- Simpler shelling out → https://github.com/oconnor663/duct.rs
Expanding arguments → https://crates.io/crates/shellexpand
- Maybe just using strings that can be expanded using format!? Can it expand using collections?
CLI Argument Prasing → https://crates.io/crates/clap
Regex → https://doc.rust-lang.org/regex/regex/index.html
SQLite → https://crates.io/crates/rust-sqlite
Web Framework - Iron → https://github.com/iron/iron
- It seems worthwhile to check out Rocket. Looks like it would be way simpler than Iron, which is just the basic structure. Rocket provides a higher level interface.
Log (facade + log4rs) → https://crates.io/crates/log https://crates.io/crates/log4rs

Configuration and Commands

An example configuration will be distributed with the package.
The configuration will be read at each startup.
Changes made to the configuration file will not be applied until the daemon is restarted.

Registering Bots and Owner

Before using the bot, the bot's token and the owner telegram id must be registered. On the first run at the command line, the user will be prompted for the token and owner id. A TOML configuration file will be used. Storing this information in a TOML file allows for simple alteration later on, if desired.

Commands

The ~~JSON~~ TOML configuration file will also be used to define commands and the responses.

I was originally thinking about using JSON for the configuration file. However, upon closer examination of TOML, it seemed more appropriate for manual editing by the user.

I really want these commands to be easy to create and expand and modify. Therefore, it seems a better fit for this project.

Command Structure in TOML

[[command]]
command = "echo" 	# Used to identify if the attempted command (the leading "/" will be programmatically accounted for)
regex = "(.*)"		# Used to identify parameters, separated from "command" by whitespace (\s+)
exec = "echo $1" 	# "$1" will be parsed to the first capture group of the regex
parameters = [ "String to echo" ] # Description of parameters

[[command]]
command = "hello"
regex = "(\\w+) (\\w+)"
exec = "echo Hello $1 $2, good to see you"
parameters = [
	"First Name",
	"Last Name"
]

Keyboard Structure in TOML

Further experimenting will be needed to give a good structure for the keyboards.

Keyboards will also contain a regex key-value pair that will be used to match it with a command.

Responses

A section of the TOML config file will also be used to specify the default responses for certain actions, such as unknown command or given input not matching the regex of a command.

[responses]
unknown = "Sorry, I don't know the $1 command" # $1 will be the command name
invalid = "Your input didn't match for the $1 command. Try Again."

Example Configuration File

[config]
ownerid = "123456789"
token = "23894yakjsdhfh8349h0s9ueg09adf013j9asj9"

[responses]
unknown = "Sorry, I don't know the $1 command" # $1 will be the command name
invalid = "Your input didn't match for the $1 command. Try Again."

[[command]]
command = "echo" 	# Used to identify if the attempted command (the leading "/" will be programmatically accounted for)
regex = "(.*)"		# Used to identify parameters, separated from "command" by whitespace (\s+)
exec = "echo $1" 	# "$1" will be parsed to the first capture group of the regex
parameters = [ "String to echo" ] # Description of parameters

[[command]]
command = "hello"
regex = "(\\w+) (\\w+)"
exec = "echo Hello $1 $2, good to see you"
parameters = [
	"First Name",
	"Last Name"
]

# ...

[[keyboard]]
# TBD

History

Keep track of commands in SQLite database

History Table

id	historyid	command	username	userid (telegram)	status	tstamp
3	44	/start	dbluhm	1234	0	2017-07-22 16:22:10
2	43	/blah	dbluhm	1234	1	2017-07-22 16:16:32
1	42	/asdf	dbluhm	1234	2	2017-07-22 16:10:03

Columns:

Column	Description
id	ID
hisotryid	Telegram API history id
command	attempted command
username	telegram username, hopefully easily identifiable
userid	telegram userid
status	statusid
tstamp	time command was received

Status Table

ID	Description
0	run
1	failed
2	invalid
3	skipped/dumped (daemon not running when sent)

Statistics and recent commands web service

I'm interested in creating a very (very) simple web service to report statistics and the most recent commands.

Note: SQLite is capable of having selects run at the same time as writing to the DB. There should be no issues with running both this and the bot simultaneously.

Details

Startup
- First time run
  - Copy example configuration and prompt for owner and token
- Configuration parsing
  - Command, Keyboard, Response Parsing
    - Generating regex for command matching
    - Matching inputs against parameter regex
    - Formatting the parameters
- Dumping all pending commands/messages
- Obtaining first offset
Long poll loop

Thoughts on Command parsing

Command parsing poses an interesting challenge. Compiling a bunch of regex from a string with each command submission would be incredibly inefficient, especially if certain commands are repeated frequently. Instead, a smarter way to do it would be to only compile regex on first use and then just keep it in memory for maximum speed. It means that more RAM will be consumed by the daemon but the usage will likely be negligible (though, admittedly, usage may be more pronounced on Raspberry Pi which is ultimately the target platform).

From the Rust regex crate documentation:

It is an anti-pattern to compile the same regular expression in a loop since compilation is typically expensive. (It takes anywhere from a few microseconds to a few milliseconds depending on the size of the regex.) Not only is compilation itself expensive, but this also prevents optimizations that reuse allocations internally to the matching engines.

In Rust, it can sometimes be a pain to pass regular expressions around if they're used from inside a helper function. Instead, we recommend using the lazy_static crate to ensure that regular expressions are compiled exactly once.

The following example is given:

#[macro_use] extern crate lazy_static;
extern crate regex;

use regex::Regex;

fn some_helper_function(text: &str) -> bool {
    lazy_static! {
        static ref RE: Regex = Regex::new("...").unwrap();
    }
    RE.is_match(text)
}

fn main() {}

`RegexSet`

Additionally, usage of a RegexSet seems like a good idea:

use regex::RegexSet;

let set = RegexSet::new(&[
    r"\w+",
    r"\d+",
    r"\pL+",
    r"foo",
    r"bar",
    r"barfoo",
    r"foobar",
]).unwrap();

// Iterate over and collect all of the matches.
let matches: Vec<_> = set.matches("foobar").into_iter().collect();
assert_eq!(matches, vec![0, 2, 3, 4, 6]);

// You can also test whether a particular regex matched:
let matches = set.matches("foobar");
assert!(!matches.matched(5));
assert!(matches.matched(6));

dbluhm/telebot.md