A Smooth Sea Never Made a Skillful Sailor

Today, we're going to explore CockroachDB from the Data Science perspective again. We will continue to use Jupyter notebook but instead of Python, we're going to use the R language. I was inspired to write this post based on an article written by my colleague. I will build on that article by introducing Jupyter Notebook to the mix.

I came across an interesting scenario working with one of our customers. They are using a common data integration tool to load hundreds of tables into CockroachDB simultaneously. They reported an issue that their loads fail intermittently due to an unrecognized error. As a debug exercise I set out to write a script to import data from an http endpoint into CRDB in parallel. Disclosure: I do not claim to be an expert in CRDB, Python or anything else for that matter. This is an exercise in answering a why not? question more so than anything educational. I wrote a Python script to execute an import job and need to make sure it executes in parallel to achieve the concurrency scenario I've originally set out to do. I'm new to Python multiprocessing and a short Google search returned a couple of options. Using built-in multiprocess, asyncio module and using Ray. Advantage to using multiprocess and asyncio is that they're built-in Python modules. Since I was rushing thr...