Write pipe list in config
Move this part of main.py
pl = Pipeline()
# pl.add_pipe(QueryExpansion(config))
# pl.add_pipe(ChatNoirPipeLine(config))
pl.add_pipe(LoadTopicPipe())
pl.add_pipe(DocumentCleaningPipeline())
# pl.add_pipe(SaveTopicPipe())
pl.add_pipe(Remerging(config))
to the config.yaml and load the pipes dynamically by a list of names:
pipeline:
- "query_expansion"
- "chat_noir"
# ...
- "remerging"