Matt Harrison - Idiomatic Pandas | SciPy 2023

Unlock the full potential of pandas and simplify your data manipulation workflows with this talk on idiomatic pandas usage.

Key takeaways
  • Chaining operations in pandas can be confusing and may not be necessary.
  • The pipe method can be used to simplify code and make it easier to read.
  • When working with missing values, it’s important to understand why they are missing and how to handle them.
  • Aggregations are powerful and can be used to summarize data, but the syntax can be tricky.
  • The groupby method can be used to group data by one or more columns.
  • The describe method can be used to get summary statistics for a dataset.
  • The loc method can be used to select rows and columns from a dataframe.
  • The set_index method can be used to set the index of a dataframe.
  • The unstack method can be used to unpivot a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The query method can be used to query a dataframe using a SQL-like syntax.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The fillna method can be used to fill missing values in a dataframe.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used to get the count of unique values in a column.
  • The astype method can be used to convert the data type of a column.
  • The dtypes attribute can be used to get the data type of a column.
  • The shape attribute can be used to get the shape of a dataframe.
  • The info method can be used to get information about a dataframe.
  • The describe method can be used to get summary statistics for a dataframe.
  • The head method can be used to get the first few rows of a dataframe.
  • The tail method can be used to get the last few rows of a dataframe.
  • The sample method can be used to sample a dataframe.
  • The dropna method can be used to drop rows with missing values.
  • The fillna method can be used to fill missing values in a dataframe.
  • The pivot_table method can be used to create a pivot table.
  • The groupby method can be used to group data by one or more columns.
  • The agg method can be used to apply aggregations to a dataframe.
  • The value_counts method can be used

More talks

Matt Harrison - An Introduction to Pandas 2, Polars, and DuckDB | PyData Global 2023

Data of an Unusual Size A practical guide to analysis and interactive visualization of massive datas

Effective Pandas I Matt Harrison I PyData Salt Lake City Meetup

Mathieu Cayssol & Chris Lo- We rewrote tsfresh in Polars and why you should too | PyData Global 2023

Shaurya Agarwal - All Them Data Engines: Data Munging with Python circa 2023 | PyData Global 2023

Hossein Mortazavi - How to use Pandas Efficiently | PyData Yerevan 2022

Ashwin Srinath - cudf.pandas: The Zero Code Change GPU Accelerator for Pandas | PyData Global 2023

Miki Tebeka - IPython: The Productivity Booster