#nlp #chinese #segmenation

jieba-rs

The Jieba Chinese Word Segmentation Implemented in Rust

41 releases

0.7.0 Apr 16, 2024
0.6.8 Jul 16, 2023
0.6.7 Oct 3, 2022
0.6.6 Nov 9, 2021
0.2.3 Jul 6, 2018

#73 in Text processing

Download history 3199/week @ 2024-01-26 2940/week @ 2024-02-02 2956/week @ 2024-02-09 2995/week @ 2024-02-16 2654/week @ 2024-02-23 2696/week @ 2024-03-01 2872/week @ 2024-03-08 3612/week @ 2024-03-15 6190/week @ 2024-03-22 6097/week @ 2024-03-29 5344/week @ 2024-04-05 4704/week @ 2024-04-12 5264/week @ 2024-04-19 4971/week @ 2024-04-26 5127/week @ 2024-05-03 5355/week @ 2024-05-10

21,371 downloads per month
Used in 18 crates (13 directly)

MIT license

4.5MB
2K SLoC

jieba-rs

GitHub Actions codecov Crates.io docs.rs

🚀 Help me to become a full-time open-source developer by sponsoring me on GitHub

The Jieba Chinese Word Segmentation Implemented in Rust

Installation

Add it to your Cargo.toml:

[dependencies]
jieba-rs = "0.6"

then you are good to go. If you are using Rust 2015 you have to extern crate jieba_rs to your crate root as well.

Example

use jieba_rs::Jieba;

fn main() {
    let jieba = Jieba::new();
    let words = jieba.cut("我们中出了一个叛徒", false);
    assert_eq!(words, vec!["我们", "", "", "", "一个", "叛徒"]);
}

Enabling Additional Features

  • default-dict feature enables embedded dictionary, this features is enabled by default
  • tfidf feature enables TF-IDF keywords extractor
  • textrank feature enables TextRank keywords extractor
[dependencies]
jieba-rs = { version = "0.6", features = ["tfidf", "textrank"] }

Run benchmark

cargo bench --all-features

Benchmark: Compare with cppjieba

jieba-rs bindings

License

This work is released under the MIT license. A copy of the license is provided in the LICENSE file.

Dependencies

~3.5–5MB
~91K SLoC