Conversion for data manipulation R-Python conversion guide

Main concepts

File management The table below summarizes useful commands to make sure the working directory is correctly set:

Chaining Successive operations can be chained in R using the %>% symbol in order to provide better legibility. It is done as follows:

# df gets some_operation_1, then some_operation_2, ..., then some_operation_n
df %>%
  some_operation_1(params_1) %>%
  some_operation_2(params_1) %>%
             ...             %>%
  some_operation_n(params_1)

The corresponding Python command is as follows:

# df gets some_operation_1, then some_operation_2, ..., then some_operation_n
(df
 .some_operation_1(params_1)
 .some_operation_2(params_2)
 .      ...
 .some_operation_n(params_n))

Exploring the data The table below summarizes the main functions used to get a complete overview of the data:

Data types The table below sums up the main data types that can be contained in columns:

Data preprocessing

Filtering We can filter rows according to some conditions as follows:

Changing columns The table below summarizes the main column operations:

Conditional column A column can take different values with respect to a particular set of conditions with the following R command:

case_when(condition_1 ~ value_1,  # If condition_1 then value_1
          condition_2 ~ value_2,  # If condition_2 then value_2
                ...
          TRUE ~ value_n)         # Otherwise, value_n

The corresponding Python command is as follows:

np.select(
  [condition_1, ..., condition_n],  # If condition_1, ..., condition_n
  [value_1, ..., value_n],          # Then value_1, ..., value_n respectively
  default=default_value             # Otherwise, default_value
)

Mathematical operations The table below sums up the main mathematical operations that can be performed on columns:

Datetime conversion Fields containing datetime values are converted from string to datetime as follows:

where format is a string describing the structure of the field and using the commands summarized in the table below:

Data frame transformation

Merging data frames We can merge two data frames by a given field as follows:

Concatenation The table below summarizes the different ways data frames can be concatenated:

Common transformations The common data frame transformations are summarized in the table below:

Row operations The following actions are used to make operations on rows of the data frame:

Aggregations

Grouping data A data frame can be aggregated with respect to given columns as follows:

The R command is as follows:

df %>%                                                 # Original ungrouped data frame
 group_by(col_1, ..., col_n) %>%                       # Group by some columns
 summarize(agg_metric = some_aggregation(some_cols))   # Aggregation step

and the Python command is as follows:

(df                                                    # Original ungrouped data frame
 .groupby(['col_1', ..., 'col_n'])                     # Group by some columns
 .agg({'col': builtin_agg}))                           # Aggregation step

where builtin_agg is among the following:

Window functions

Definition A window function computes a metric over groups and has the following structure:

The R command is as follows:

df %>%                                       # Original ungrouped data frame
 group_by(col_1, ..., col_n) %>%             # Group by some columns
 mutate(win_metric = window_function(col))   # Window function

and the Python command is as follows:

(df
 .assign(win_metric = lambda x:
           x.groupby(['col_1', ..., 'col_n'])['col'].window_function(params)))

Remark: applying a window function will not change the initial number of rows of the data frame.

Row numbering The table below summarizes the main commands that rank each row across specified groups, ordered by a specific field:

Values The following window functions allow to keep track of specific types of values with respect to the group: