For machine learning applications it is generally not enough to use only base functionality in the python programming language. We will often rely heavily on different software packages for scientific computing. The most important of these is NumPy, which will be used in most exercises of this course. NumPy provides an array construct that can be used to store and perform efficient operations on data.

This section will provide a brief introduction to NumPy, covering most of the functionality that you will need in the course. There are also a few mandatory exercises where you will test the things you have learned. If you already feel familiar with NumPy you can skip directly to these exercises. This section can also act as a reference that you can go back to later in the course to remind yourself about NumPy specifics. Throughout the course you will likely also need to make use of the NumPy documentation to check details of the different functions used.

At the center of the NumPy package are arrays. NumPy arrays are a way to store a collection of numbers in a structured way. A one-dimensional array mathematically corresponds to a vector and a two-dimensional array corresponds to a matrix. Some examples are shown below:

In NumPy, we can refer to the dimensions of an array using axes. While a one-dimensional array has only one axis, a two-dimensional array has two axes, where axis 0 runs along the rows and axis 1 along the columns. We can also create arrays of higher dimension than two. Sometimes the word tensor is used interchangeably to refer to multi-dimensional arrays, although the formal definitions differ. A three-dimensional array has three axes, as illustrated in the figure below.

There are two key properties of NumPy arrays: their shape and data type. The shape describes the size of the array along each axis, for example the number of rows and columns in a two-dimensional array. It can be accessed by calling .shape on any array defined in your code. The data type describes what type of data is stored in each entry of the array. Examples of data types are integers (int64), floating point numbers (float64) and booleans (bool). The data type of an array can be accessed using .dtype. In the example code below we create some arrays of different dimensionality and inspect their shape and datatype:

For a two-dimensional NumPy array, we access an element by using its row and column indices. For example, if we want to access the element at row $i$ and column $j$ in an array X, we would access it by X[i, j]. We use : to access a full row or column. Remember that python uses zero-based indexing, meaning that the first element along each axis is located at index 0.

There are also more advanced types of array indexing that can sometimes be useful. An array can be indexed along an axis by using a second array containing integers. This means that we pick out those entries from the first array that are listed in the second, indexing array.

Another useful way of indexing arrays is with the help of boolean masks, where we index an array using a boolean array as a mask. Positions in the boolean array that are True will be included in the final array and positions where the boolean array is False will be excluded.

Now you should have gained some understanding of what NumPy arrays are and how we can index them. In order to test your knowledge, please complete the exercise below:

We can generally perform the same operations with arrays as with numbers. An operation between two arrays A and B is said to be element-wise if it is performed separately for every entry in the arrays. Let $\mathbf{A}$ and $\mathbf{B}$ be two-dimensional. The element-wise operation $\mathbf{A} + \mathbf{B}$ would for every pair $i,j$ sum the element at position $(i, j)$ in array $\mathbf{A}$ with the element in the same position in array $\mathbf{B}$ according to $(\mathbf{A}+\mathbf{B})_{ij}=\mathbf{A}_{ij}+\mathbf{B}_{ij}$. In the example below, both $\mathbf{A}$ and $\mathbf{B}$ have shape $2\times 2$.

Some operations can be done element-wise on a single array. We can for example add or multiply single numbers with an entire array and the operation will be applied to each entry of the array.

In general, element-wise operations on multiple arrays will not work unless the arrays $\mathbf{A}$ and $\mathbf{B}$ have the same shape. However, in some cases when the shapes are not the same, NumPy can broadcast the smaller array to match the size of the larger array. An example for when $\mathbf{A}$ is of size $2\times 2$ and $\mathbf{B}$ of size $1\times 2$ is shown below.

Notice that the first element of $\mathbf{B}$ is added to all of the elements in the first column of $\mathbf{A}$ and that the second element of $\mathbf{B}$ is added to all of the elements in the second column of $\mathbf{A}$.

We are also interested in functions that operate on multiple entries in an array. The sum function can be used to sum entries of an array. Per default it will sum all entries, but using the axisargument we can specify to sum along a specific axis. For example, to sum all rows in a two-dimensional array we would use sum with axis=1, as the rows run along axis 1. The mean function works similarly to the sum function, but returns the mean of the entries in place of the sum.

In this exercise you will get to practice using axis-wise operations on arrays.

We will now have a look at matrix multiplication in NumPy. Matrix multiplications are integral to many machine learning models, so it is a good idea to be familiar with how we can do them in code. Consider the following two matrices

We now want to compute the matrix multiplication $\mathbf{A}\mathbf{B}$ using python and NumPy. Recall that matrix multiplication (in this case for $2 \times 2$ matrices) is performed according to:

so if we insert numerical values:

Now in python, we might first attempt to do something like the following:

But recall from before that this is the element-wise multiplication of arrays (an operation sometimes referred to as the Hadamard product). This is not the matrix multiplication that we were after.

Instead NumPy uses the @ operator to represent matrix multiplication. This is also the case for most other python libraries that work with matrices. If we change * for @ in our code, we can see that we get a proper matrix multiplication:

While the matrix multiplication has its own specific syntax with @, many other useful linear algebra operations can be found in the NumPy linalg submodule.

In this third and final exercise you will get to try out different ways to multiply arrays.

Now you have become acquainted with the most important parts of the NumPy package. You should now be ready to use NumPy for some actual machine learning! Remember that you can always go back to this section later on if you want a reminder about some NumPy specifics.