Pandas is a Python language package, which is used for data processing. This is a very common basic programming library when we use Python language for machine learning programming. This article is an introductory tutorial to it. Pandas provide fast, flexible and expressive data structures with the goal of making the work of “relational” or “marking” data simple and intuitive. It is intended to be a high-level building block for actual data analysis in Python.

Pandas Introduction

Pandas is suitable for many different types of data, including:

Since this is a Python language package, you need to have a Python language environment on your machine. In this regard, please search for your own method on the Internet.
For instructions on how to get pandas, please refer to the official website: pandas Installation.

In general, we can pipperform the installation by:

sudo pip3 install pandas

Or install pandas through conda :

conda install pandas

Currently, the latest version of pandas is v0.22.0 .

You can find the source code and test data of this article on Github at: pandas tutorial , readers can go for it.

In addition, pandas and NumPy often used together, the source code in this article will be used NumPy. It is recommended that readers have some familiarity with NumPy before learning Python pandas.

Core data structure

Pandas is the core Seriesand DataFrametwo data structures.

The comparison of these two types of data structures is as follows:

Name Dimensions Instructions
Series 1dimensional An array of isomorphic types with labels
DataFrame 2dimensional Table structure, with tags, variable size, and can contain heterogeneous data columns

A DataFrame can be thought of as a Container of Series, ie a DataFrame can contain several Series.

“Note: Before the 0.20.0 release, there was a three-dimensional data structure named Panel. This is also the reason for the pandas library name: pan (el)- da (ta)- s . However, this data structure has been abandoned because it is rarely used.”

Series

Since Series is a one-dimensional structure of data, we can create this data directly through an array, like this:

# data_structure.py
import pandas as pd
import numpy as np
series1 = pd.Series([1, 2, 3, 4])
print("series1:\n{}\n".format(series1))
The output of this code is as follows:
series1:
0    1
1    2
2    3
3    4
dtype: int64

This output is described as follows:

We can print the data and indexes in the Series separately:

# data_structure.py
print("series1.values: {}\n".format(series1.values))
print("series1.index: {}\n".format(series1.index))
The two lines of code output are as follows:
series1.values: [1 2 3 4]
series1.index: RangeIndex(start=0, stop=4, step=1)

If you do not specify (as above), the index is in the form of [1, N-1]. However, we can also specify the index when creating the Series. The index does not necessarily need to be an integer and can be any type of data, such as a string. For example, we map seven notes in seven letters. The purpose of the index is to use it to obtain the corresponding data, such as the following:

# data_structure.py
series2 = pd.Series([1, 2, 3, 4, 5, 6, 7],
    index=["C", "D", "E", "F", "G", "A", "B"])
print("series2:\n{}\n".format(series2))
print("E is {}\n".format(series2["E"]))
The output of this code is as follows:
series2:
C    1
D    2
E    3
F    4
G    5
A    6
B    7
dtype: int64
E is 3

DataFrame

Let’s take a look at the creation of the DataFrame. We can create a 4×4 matrix through the NumPy interface to create a DataFrame like this:

# data_structure.py
df1 = pd.DataFrame(np.arange(16).reshape(4,4))
print("df1:\n{}\n".format(df1))
The output of this code is as follows:
df1:
    0   1   2   3
0   0   1   2   3
1   4   5   6   7
2   8   9  10  11
3  12  13  14  15

From this output, we can see that the default index and column names are of the form [0, N-1].

We can specify the column name and index when creating the DataFrame, like this:

# data_structure.py
df2 = pd.DataFrame(np.arange(16).reshape(4,4),
    columns=["column1", "column2", "column3", "column4"],
    index=["a", "b", "c", "d"])
print("df2:\n{}\n".format(df2))
The output of this code is as follows:
df2:
   column1  column2  column3  column4
a        0        1        2        3
b        4        5        6        7
c        8        9   	10   	11
d   	12   	13   	14   	15

We can also specify column data directly to create a DataFrame:

# data_structure.py
df3 = pd.DataFrame({"note" : ["C", "D", "E", "F", "G", "A", "B"],
    "weekday": ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]})
print("df3:\n{}\n".format(df3))
The output of this code is as follows:
df3:
  note weekday
0    C 	Mon
1    D 	Tue
2    E 	Wed
3    F 	Thu
4    G 	Fri
5    A 	Sat
6    B 	Sun

Please note:

E.g:

# data_structure.py
noteSeries = pd.Series(["C", "D", "E", "F", "G", "A", "B"],
    index=[1, 2, 3, 4, 5, 6, 7])
weekdaySeries = pd.Series(["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"],
    index=[1, 2, 3, 4, 5, 6, 7])
df4 = pd.DataFrame([noteSeries, weekdaySeries])
print("df4:\n{}\n".format(df4))

The output of df4 is as follows:

df4:
 	1    2    3    4    5    6    7
0    C    D    E    F    G    A    B
1  Mon  Tue  Wed  Thu  Fri  Sat  Sun

We can add or remove column data to DataFrame in the following form:

# data_structure.py
df3["No."] = pd.Series([1, 2, 3, 4, 5, 6, 7])
print("df3:\n{}\n".format(df3))
del df3["weekday"]
print("df3:\n{}\n".format(df3))
The output of this code is as follows:
df3:
  note weekday  No.
0    C 	Mon    1
1    D 	Tue    2
2    E 	Wed    3
3    F 	Thu    4
4    G 	Fri    5
5    A 	Sat    6
6    B 	Sun    7
df3:
  note  No.
0    C    1
1    D    2
2    E    3
3    F    4
4    G    5
5    A    6
6    B    7

Index object and data access

The Pandas Index object contains metadata describing the axis. When creating a Series or DataFrame, the array or sequence of tags is converted to Index. You can get the Index object of the DataFrame column and row in the following way:

# data_structure.py
print("df3.columns\n{}\n".format(df3.columns))
print("df3.index\n{}\n".format(df3.index))
The two lines of code output are as follows:
df3.columns
Index(['note', 'No.'], dtype='object')
df3.index
RangeIndex(start=0, stop=7, step=1)

Please note:

DataFrame provides the following two operators to access the data:

For example:

# data_structure.py
print("Note C, D is:\n{}\n".format(df3.loc[[0, 1], "note"]))
print("Note C, D is:\n{}\n".format(df3.iloc[[0, 1], 0]))
Note C, D is:
0    C
1    D
Name: note, dtype: object
 
Note C, D is:
0    C
1    D
Name: note, dtype: object

The first line of code accesses elements whose row indexes are 0 and 1 and whose column index is “note”. The second line of code accesses the row indices 0 and 1 (for df3, the row index and row subscript are exactly the same, so here are 0 and 1, but they are different meanings), the column subscript Is an element of 0.

Conclusion

This article is a first introductory tutorial for pandas, so here we covered the most basic operations. You can read next article in the series here. We hope that you understood the tutorial well and if you have any queries, please drop your comment in below comment box. We will get back to you as soon as possible.

Note: I learnt this pandas tutorial from this great resources Pandas – Powerful Python Data Analysis Toolkit and Python Data Analysis by J.Metz. It is a great tutorial and I highly recommend to read it if you are more interested in Pandas and Python data analysis!