Skip to content

Instantly share code, notes, and snippets.

@tushuhei
Last active November 8, 2018 00:56
Show Gist options
  • Save tushuhei/5ac67442a8e5e4c78a2ad41805864fd9 to your computer and use it in GitHub Desktop.
Save tushuhei/5ac67442a8e5e4c78a2ad41805864fd9 to your computer and use it in GitHub Desktop.
Word segmentation with pure V8 engine.
let segment = (text) => {
let it = Intl.v8BreakIterator('ja');
it.adoptText(text);
let result = [];
let curr, next;
while (true) {
curr = it.current();
next = it.next();
if (next == -1) break;
result.push(text.slice(curr, next));
}
return result;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment