Cosima Meyer

bit.ly/wids2023-slides

Who am I?

👩🏼‍🎓 PhD in Politicial Science @ University of Mannheim

👩🏼‍💻 Data Science @ IBM

🦸🏼‍♀️ Co-organizer of the joint PyLadies 💜 R-Ladies event series

I have an academic background and studied civil wars around the world. During this time, I always enjoyed discovering and, most importantly, understanding methodological approaches. I still remember my first attempt of uncovering survival analysis where I took a pen and paper to calculate the steps by hand and to (at least to some degree) also understand the underlying stats.

After I got my PhD, I then transitioned to industry and work now as a data scientist. I have already seen a few different settings and worked with different envrionemnts and also languages. Picking two, Python and R, each of them has their own benefits and advantages and you can definitely use them together. We see the symbiosis of languages also with Java and Scala but we can also have it for R and Python.

And this inspired me to also bring the communities together - and that’s why I initiated a series of R-Ladies and PyLadies where we invite speakers who operate with both languages and want to showcase how beautifully they work together and how easily you can transition between them. And that’s also one of the goals of this workshop today: I’d love to give you a short introduction how you can use both languages (also together).

What we have ahead of us today

For this, we will first look at one data science workflow and then dive into data access and data wrangling with Python and data visualization with R.

You will get Jupyter notebooks that contains the full code. You can either use it to run the code yourself or watch the screen and follow along while I’ll explain the code to you.

A data science workflow

But first let’s dig into how a data science workflow can look like

A data science workflow

can perfectly accompany more than just one language

As mentioned before, we know from practice that programming languages often go hand-in-hand. Java and Scala may be one example - but I am convinced that also Python and R have both their own advantages (and disadvantages) and are often used in a great symbiosis.

At the beginning of every data science workflow is usually a question that you want to answer. It could also be “Can I build this product?” or “How to classify XYZ?” It is a bit like in academia where you have a research question at the beginning of every paper.

Based on this question, you then gather data and access them - that means you usally load the data.

Now a circular flow starts. You usually inspect the data first, do some wrangling where needed. The degree of wrangling can differ, depending on what you want to analyze.

Once the data is ready (or “clean”), you can also perform some statistical analyses and also machine learning - whatever serves your purpose best.

And then (or also in-between), you have some data visualization. It can be the final outcome or also a validation step in-between to better understand your data.

And in the end, you want to communicate your results :)

We capture three steps in the process and start with accessing the data using Python…

… then move on to do some data wrangling in Python using the library pandas

And finally plot a visualization in R.

While the process might seem to be linear

It is often not and, what we’ll also see today, the end is often just the beginning of another iteration.

Data access

Let’s start with some basics in data accessing

Data access

• Python 🐍
• R 🔵

# Load library
import pandas as pd

# Load data
url = "https://bit.ly/url-data"

health_df = pd.read_csv(url)

# Load data
url = "https://bit.ly/url-data"

health_data <- read.csv(url)

Here’s the code that you’ll often use to access the data.

👩🏼‍💻

And now we’ll see how it works hands-on with some code and real data

Data Wrangling

Data wrangling: Common first steps

• Python 🐍
• R 🔵

# Print first lines
health_df.head()

# Extract the size
health_df.shape

# Extract info
health_df.info()

# Print first lines
head(health_data)

# Extract the size
dim(health_data)

# Extract info
str(health_data)
summary(health_data)

Data wrangling: The logic of pandas

👩🏼‍💻

Visualization

👩🏼‍💻

What else?

If you are curious to learn more about how nicely R and Python can work together, have a look at our R-Ladies 💜 PyLadies event series:

✨ Kickoff (including input talks on working in bilingual teams and using Mlflow for MLOps)

👩‍🎨 Visualization with Plotnine

🤖 Automating Workflows Using GitHub Actions and Quarto

🧠 autoML with H2O

… to be continued!

cosimameyer.com

cosimameyer

cosimameyer

@cosima_meyer@mas.to