This post aims to define a general purpose data processing architecture to run experiments, the idea is to have a modular system to test new tools and learn how to best take advantage of them. Also, this is something focused on my own learning and attempt to document, and share if useful, anything that I…