Data are becoming the new raw material of business
The Economist

A Study Of Reddit Politics

This article was written for The Data Incubator by Jay Kaiser, a Fellow of our 2018 Winter cohort in Washington, DC who landed a job with our hiring partner, ZeniMax Online Studios, as a Big Data Engineer.


The Question

The 2016 Presidential Election was, in a single word, weird. So much happened during the months leading up to November that it became difficult to keep track with what who said when and why. However, the finale of the election that culminated with Republican candidate Donald J. Trump winning the majority of the Electoral College and hence becoming the 45th President of the United States was an outcome which at the time I had thought impossible, if solely due to the aforementioned eccentric series of events that had circulated around Trump for a majority of his candidacy.

Following the election, the prominent question that could not leave my mind was a simple one: how? How had the American people changed so much in only a couple of years to allow an outsider hit by a number of black marks during the election to be elected to the highest position in the United States government? How did so many pollsters and political scientists fail to predict this outcome? How can we best analyze the campaigns of each candidate, now given hindsight and knowledge of the eventual outcome? In an attempt to answer each of these, I have turned to a perhaps unlikely source.

Continue reading

SQLite vs Pandas: Performance Benchmarks

This technical article was written for The Data Incubator by Paul Paczuski, a Fellow of our 2016 Spring cohort in New York City who landed a job with our hiring partner, Genentech as a Clinical Data Scientist.

As a data scientist, we all know that unglamorous data manipulation is 90% of the work. Two of the most common data manipulation tools are SQL and pandas. In this blog, we’ll compare the performance of pandas and SQLite, a simple form of SQL favored by Data Scientists.

Let’s find out the tasks at which each of these excel. Below, we compare Python’s pandas to sqlite for some common data analysis operations: sort, select, load, join, filter, and group by.

Continue reading

Calculating the Perfect Algorithm: Alumni Spotlight on Sumanth Swaminathan

At The Data Incubator we run a free eight-week Data Science Fellowship Program to help our Fellows land industry jobs. We love Fellows with diverse academic backgrounds that go beyond what companies traditionally think of when hiring Data Scientists. Sumanth was a Fellow in our Winter 2016 cohort who landed a job with one of our hiring partners, Revon.


Tell us about your background. How did it set you up to be a great Data Scientist?

I did my bachelors degree in Chemical Engineering at the University of Delaware and my PhD in Applied Mathematics at Northwestern University. After some postdoctoral work between Northwestern and Oxford University, I went into industry as a quantitative consultant for W.L. Gore & Associates. For the past 4 years, I have spent most of my time delivering technology solutions at W.L. Gore, teaching mathematics at the University of Delaware, and performing and teaching Indian Classical Music.

On the question of what makes a strong data scientist, I think that the better practitioners in the field tend to be hypothesis driven, strong critical thinkers with hard skills in statistics, programming, mathematics, and hardware. Hence, my background in engineering and mathematics, my consulting experience, and my years of teaching probably contributed the most to my success.


What do you think you got out of The Data Incubator?

1. I learned an incredible amount of new problem solving methods, concepts and technologies
2. I joined a large community of practicing and aspiring data scientists (the fellows admitted into this program were really accomplished and came from all educational backgrounds)
3. I was approached and interviewed by numerous companies of all sizes.
4. I received professional advice from hiring managers, computer scientists, and strong mathematical talent.

Continue reading