Skip to content

Commit

Permalink
change rasa version to release 1.0
Browse files Browse the repository at this point in the history
  • Loading branch information
colin.gao committed Jun 19, 2019
1 parent 891016a commit ea510e2
Show file tree
Hide file tree
Showing 372 changed files with 276 additions and 39,217 deletions.
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,7 @@ __pycache__
data/*.dat
*.log
stories.md
comparison_models/*
comparison_models/*
failed_stories.md
story_confmat.md
test_env
63 changes: 9 additions & 54 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,58 +1,13 @@
help:
@echo " train-nlu"
@echo " Train the natural language understanding using Rasa NLU."
@echo " train-core"
@echo " Train a dialogue model using Rasa core."
@echo " run"
@echo " Runs the bot on the command line."
@echo " run-api"
@echo " Runs the bot as a server."
train:
rasa train --domain domain.yml --data data --config config.yml --out models

run:
make run-actions&
make run-core
rasa run actions --actions actions.actions &
rasa run --endpoints configs/endpoints.yml --enable-api -m models --debug

run-api:
make run-actions&
make run-http
run-cmdline:
kidx run actions --actions actions.actions &
kidx shell --endpoints configs/endpoints.yml -m models --debug

run-actions:
python -m rasa_core_sdk.endpoint --actions actions

run-core:
python -m rasa_core.run --nlu models/nlu/default/current --core models/dialogue --endpoints endpoints.yml

run-online:
python -m rasa_core.train --online -o models/dialogue -d mobile_domain.yml -s data/mobile_edit_story.md --endpoints endpoints.yml

run-http:
python -m rasa_core.run --enable_api -d models/dialogue -u models/nlu/default/current --endpoints endpoints.yml --auth_token gaoquan

run-nlu-server:
python -m rasa_nlu.server -c configs/nlu_model_config.yml --path models/nlu/

evaluate:
python -m rasa_core.evaluate -d models/dialogue -s data/mobile_edit_story.md

visualize:
python -m rasa_core.visualize -s data/mobile_edit_story.md -d mobile_domain.yml -o story_graph.png

train-nlu:
python bot.py train-nlu

train-core:
python bot.py train-dialogue

train-nlu-gao:
python bot.py train-nlu-gao

run-nlu-gao-server:
python -m rasa_nlu_gao.server -c configs/config_embedding_bilstm.yml --path models/nlu_gao/

compare-policy:
python -m rasa_core.train compare -c policy/attention_policy.yml policy/keras_policy.yml policy/embed_policy.yml \
-d mobile_domain.yml -s data/mobile_edit_story.md -o comparison_models/ --runs 3 --percentages \
0 10 20 30 50 60 70 80 90 100

evaluate-policy:
python -m rasa_core.evaluate compare -s data/mobile_edit_story.md --core comparison_models/ -o comparison_results/
run-graph:
rasa visualize --domain domain.yml --stories data/core --config config.yml --nlu data/nlu
75 changes: 23 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,75 +12,46 @@
- [rasa对话系统踩坑记(十)](https://www.jianshu.com/p/debcf0041fcb)

## Introduction
这个demo是用rasa-nlu完成slot filling和intent classify,用rasa-core完成DM(dialogue management)和NLG(natural language generate)。demo完成的对话主要有办理套餐和查询话费和流量,其他意图做了default回答。demo是参考了[_rasa_chatbot](https://github.com/zqhZY/_rasa_chatbot)_rasa_chatbot存在的问题,一是版本更新不及时,API没有做相应的更改;二是没有自定义的component。
由于rasa社区最近版本更新很频繁,以前很多方法被弃用,而且还增加了好多新的功能,这个demo比较全面和及时。
```
rasa-nlu: 0.13.4
rasa-core: 0.12.3
rasa-core-sdk: 0.12.1
```
五月份rasa官方发布了release版本,做了比较大的改动。介于此,[rasa_chatbot_cn](https://github.com/GaoQ1/rasa_chatbot_cn)这个demo也做出相对应的更新,更新到master分支上。之前基于`0.13`的版本在0.13.x分支上,你可以自由切换。

## Command
### train nlu model
```
python bot.py train-nlu
```
**total_word_feature_extractor.dat**可去https://pan.baidu.com/s/1-ma0ndXBWL0rnbUqCAcL-w ,密码:lhi4 下载。nlu_model_config.yml中的pipeline可自定义,这里由于数据量较少,用了开源的方法和词向量。如果你的rasa_dataset_training.json上数据足够多,可以尝试使用nlu_embedding_config.yml配置来训练nlu model.


### train dialogue
### install packages
```
python bot.py train-dialogue
pip install -r requirements.txt
```
下载依赖package

### train dialogue in online mod
### train model
```
python -m rasa_core_sdk.endpoint --actions actions &
python -m rasa_core.train --online -o models/dialogue -d mobile_domain.yml -s data/mobile_story.md --endpoints endpoints.yml
make train
```
训练nlu和core模型,新版本中会将模型自动打包成zip文件

### test dialogue
### run model
```
python -m rasa_core_sdk.endpoint --actions actions &
python -m rasa_core.run --nlu models/nlu/default/current --core models/dialogue --endpoints endpoints.yml
make run
```

### provide dialogue service
### test in cmdline
```
python -m rasa_core_sdk.endpoint --actions actions &
python -m rasa_core.train --online -o models/dialogue -d mobile_domain.yml -s data/mobile_story.md --endpoints endpoints.yml
make run-cmdline
```
可以在命令行中测试

### compare policy
### test by http server
`http://localhost:5005/webhooks/rest/webhook` post请求,请求参数例如:
```
python -m rasa_core.train compare -c keras_policy.yml embed_policy.yml -d mobile_domain.yml -s data/mobile_edit_story.md -o comparison_models/ --runs 3 --percentages 0 25 50 70
{
"sender": "0001",
"message": "你好"
}
```

### evaluate policy
```
python -m rasa_core.evaluate compare -s data/mobile_edit_story.md --core comparison_models/ -o comparison_results/
```

## Some tips
### rename and count story
utils/re_story.py 是用来对mobile_story.md里面的故事进行重命名和重新计数
### auto generate rasa_dataset_training.json
data/rasa_dataset_training.json 是通过一些规则自动生成的,节省很多人力。仓库是[chatito_gen_nlu_data](https://github.com/GaoQ1/chatito_gen_nlu_data)
具体用法可参考[官方文档](https://rodrigopivi.github.io/Chatito/)

可以使用postman去请求调用

## Some magical functions
最近自己开源了基于rasa-nlu的[rasa-nlu-gao](https://github.com/GaoQ1/rasa_nlu_gq)新增了N多个个自定义组件,具体用法和说明请参考[rasa对话系统踩坑记](https://www.jianshu.com/u/4b912e917c2e)。那下面就是介绍如何使用了。
### 首先需要下载rasa-nlu-gao
```
pip install rasa-nlu-gao
```
### 训练模型
```
python bot.py train-nlu-gao
```
### 测试使用模型
之前在[rasa-nlu-gao](https://github.com/GaoQ1/rasa_nlu_gq)增加了若干个自定义组件。而在release版本中可以直接将组建在外部调用,比如这里我举个之前的`JiebaPsegExtractor component`的栗子,直接将该组建放在*components*下面,在*config.yml*中:
```
python -m rasa_nlu_gao.server -c config_embedding_bilstm.yml --path models/nlu_gao/
- name: "components.extractors.jieba_pseg_extractor.JiebaPsegExtractor"
part_of_speech: ["nr"]
```
后续[rasa-nlu-gao](https://github.com/GaoQ1/rasa_nlu_gq)会持续更新,也欢迎贡献
这样就ok了,后续我会考虑将rasa-nlu-gao重新修改下
File renamed without changes.
142 changes: 0 additions & 142 deletions bot.py

This file was deleted.

6 changes: 0 additions & 6 deletions comparison_models/num_stories.json

This file was deleted.

38 changes: 0 additions & 38 deletions comparison_models/run_1/AttentionPolicy1/domain.json

This file was deleted.

Loading

0 comments on commit ea510e2

Please sign in to comment.