INSTALL.md

Install instructions

This provides instructions for seting up the software on a freshly installed debian 9 system. It will most likely work on any recent ubuntu system too, though there may be some hickup with the python versions.

Installing Debian

This assumes a standard install of debian was made using the smallcd AMD64 debian image. It was tested selecting only the base system with the standard system utilities (which contain python) and no gui. This guide assumes during setup a user named rasa was created, though this shouldn't be too hard to adapt to.

Hypervisor specific steps

Hyper-V

Nothing to do, works out of the box. Tested using Hyper-V Quick Create accepting the defaults.

KVM

Not tested.

VirtualBox

Works.

Installing sudo

Though not required we'll make rasa a sudoer for convenience reasons.

First log in as root and run

apt-get install sudo

Next we'll make the rasa user a sudoer

usermod -aG sudo rasa

All done here. exit and log in as rasa.

Script based installation

This provides instructions for installing with the help of an bash script, if you want to install manually skip ahead.

Installing git

First, we'll install git and a few essentials we'll need along the way.

sudo apt-get update && \
sudo apt-get dist-upgrade -y && \
sudo apt-get install -y --no-install-recommends gcc git build-essential python-dev -y

To clone the project via git run:

git clone https://git.informatik.uni-leipzig.de/text-mining-chatbot/wiki-rasa.git
cd wiki-rasa

Now, run the installer script, it will take care of installing miniconda, spacy and also R.

./install.sh

Finally we'll need to install R packages. We have to do this in an interactive R shell as R will ask wheather to use a personal library. From an R shell run the following:

install.packages(readLines("processing/packages.list"))

To install the wikiproc package navigate to the processing directory and run:

R CMD build wikiproc
R CMD INSTALL wikiproc_<version>.tar.gz

That's it. You should be good to go and run the master script now.

Manual installation

Just to make sure we update the system and also install some stuff we'll need.

sudo apt-get update && sudo apt-get dist-upgrade -y && sudo apt-get install gcc git build-essential python-dev -y

Next, install miniconda:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x84_64.sh

Defaults are fine here.

Log out and back in.

Now we create an environment for spacy and install it:

conda create -n spcy python=3
conda activate spcy
pip install spacy
python -m spacy download en
conda deactivate

Installing R

We need to add the cran repository to sources.list as the r packages in the debian repositories are somewhat out of date.

For that we'll need a few packages

sudo apt install dirmngr --install-recommends
sudo apt install software-properties-common apt-transport-https -y

Now we'll add the key for the cran ppa and add the ppa

sudo apt-key adv --keyserver keys.gnupg.net --recv-key 'E19F5F87128899B192B1A2C2AD5F960A256A04AF'
sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/debian stretch-cran35/'

Finally we may install R

sudo apt-get update
sudo apt-get install r-base-dev

While we're at it, we install a few more things we need for some R packages and also git.

sudo apt-get install libcurl4-openssl-dev libssl-dev libxml2-dev git -y

Cloning the project

Run:

git clone https://git.informatik.uni-leipzig.de/text-mining-chatbot/wiki-rasa.git
cd wiki-rasa

Installing R Packages

This needs to be done from an Interactive R console as R will ask wheather to use an personal library the first time installing packages. To do this, open R and type the following:

install.packages(readLines("packages.list"))

This will install all the packages required. When asked if you want to use a personal library say yes and accept the defaults.

To install the wikiproc package navigate to the processing directory and run:

R CMD build wikiproc
R CMD INSTALL wikiproc_<version>.tar.gz

That's it. You should be good to go and run the master script now.

Bot Setup

In order to setup and run the Rasa Bot we recommend to use a conda environment again with Python 3.6.7

conda create -n rasa_env python=3.6.7
source activate rasa_env

You need to install Rasa Core and Rasa NLU to run the Bot

pip install rasa_nlu
pip install rasa_core

Install the pipeline

pip install sklearn_crfsuite
pip install spacy

python -m spacy download en_core_web_md
python -m spacy link en_core_web_md en

Now you can train and run the bot

cd rasa/
make train

make run

Run in Debug Mode for more logging

make run-debug