Lists

Lists#

Questions:#

  • How can I store multiple values in a single variable?

Learning Objectives:#

  • Explain why programs need collections of values

  • Write programs that create lists, index them, slice them, and modify them through assignment and method calls

A list stores many values in a single structure#

  • We have seen how to store values by assigning them to variables

  • With data sets of any reasonable size, it would rapidly become inefficient and confusing to store each data point with a separate variable name

  • For example, here are average life expectancies for Canada over 100 years, from the Gapminder data set:

life_exp_1900 = 48.1
life_exp_1920 = 56.6
life_exp_1940 = 64.0
life_exp_1960 = 71.0
life_exp_1980 = 75.2
life_exp_2000 = 79.2

  • While you can store the data this way, it requires a lot of variable names. As well, although there is structure in the data (they reflect an ordered sequence of dates), this ordering is not reflected in how the data are stored.

  • We can use a list to store many values together

  • Lists are defined by one or more items contained within square brackets [], with individual values (list items) separated by commas ,:

life_exp = [48.1, 56.6, 64.0, 71.0, 75.2, 79.2]
print('Life expectancies:', life_exp)

Life expectancies: [48.1, 56.6, 64.0, 71.0, 75.2, 79.2]

Like strings, we can use the len() function to get the length of a list

print('length:', len(life_exp))

length: 6

Here, length refers to the number of items in the list

Lists may contain values of different types#

A single list may contain virtually any Python data type, including integers, floats, strings, and even other lists!

personal_info = ['First Name', 'Costanza', 
                'Last Name', 'Raghnall', 
                'Age', 42, 
                'Height', 1.65, 
                'Children', ['Apoorva', 'Columbanus']
               ]

that Python allows us to input a list over multiple lines. Jupyter nicely indents the additional lines to help us see that they are continuations of the same list.

(Data generated by https://www.behindthename.com)

Indexing is similar for lists and strings#

We previously saw how we could get single characters from a character string, using indexes in square brackets:

element = 'carbon'
print('zeroth character:', element[0])
print('third character:', element[3])

zeroth character: c
third character: b

Use an item’s index to fetch it from a list#

Indexing works for lists as it does for strings:

print('zeroth item of life_exp:', life_exp[0])
print('fourth item of life_exp:', life_exp[4])

zeroth item of life_exp: 48.1
fourth item of life_exp: 75.2

Slicing is similar for lists and strings#

Recall that we can slice a string to get a range of values, e.g.,

print('First three items:', element[0:3])

We can do the same thing with lists:

print('First three items in life_exp:', life_exp[0:4])
print('Last item in life_exp:', life_exp[-1])

First three items in life_exp: [48.1, 56.6, 64.0, 71.0]
Last item in life_exp: 79.2

Indexing can be used to replace a value in a list#

life_exp[0] = 48.3
print(life_exp)

[48.3, 56.6, 64.0, 71.0, 75.2, 79.2]

Appending items to a list lengthens it#

We can use the .append() method to add an item to the end of a list

Here we will add the life expectancy for 2020 in Canada to extend our list:

print(life_exp)
life_exp.append(82.3)
print(life_exp)

[48.3, 56.6, 64.0, 71.0, 75.2, 79.2]
[48.3, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3]

Methods#

  • .append() is a list method

  • Methods are like functions, but tied to a particular type of object. In other words, the fact that there is a list .append() method does not mean that we can use .append() with other Python objects that aren’t lists.

  • Use object_name.method_name() to call methods.

  • We will meet other methods of lists as we go along.

  • Use help(list) for a preview of the methods available for objects of type list

help(list)

Help on class list in module builtins:

class list(object)
 |  list(iterable=(), /)
 |
 |  Built-in mutable sequence.
 |
 |  If no argument is given, the constructor creates a new empty list.
 |  The argument must be an iterable if specified.
 |
 |  Methods defined here:
 |
 |  __add__(self, value, /)
 |      Return self+value.
 |
 |  __contains__(self, key, /)
 |      Return bool(key in self).
 |
 |  __delitem__(self, key, /)
 |      Delete self[key].
 |
 |  __eq__(self, value, /)
 |      Return self==value.
 |
 |  __ge__(self, value, /)
 |      Return self>=value.
 |
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |
 |  __getitem__(self, index, /)
 |      Return self[index].
 |
 |  __gt__(self, value, /)
 |      Return self>value.
 |
 |  __iadd__(self, value, /)
 |      Implement self+=value.
 |
 |  __imul__(self, value, /)
 |      Implement self*=value.
 |
 |  __init__(self, /, *args, **kwargs)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |
 |  __iter__(self, /)
 |      Implement iter(self).
 |
 |  __le__(self, value, /)
 |      Return self<=value.
 |
 |  __len__(self, /)
 |      Return len(self).
 |
 |  __lt__(self, value, /)
 |      Return self<value.
 |
 |  __mul__(self, value, /)
 |      Return self*value.
 |
 |  __ne__(self, value, /)
 |      Return self!=value.
 |
 |  __repr__(self, /)
 |      Return repr(self).
 |
 |  __reversed__(self, /)
 |      Return a reverse iterator over the list.
 |
 |  __rmul__(self, value, /)
 |      Return value*self.
 |
 |  __setitem__(self, key, value, /)
 |      Set self[key] to value.
 |
 |  __sizeof__(self, /)
 |      Return the size of the list in memory, in bytes.
 |
 |  append(self, object, /)
 |      Append object to the end of the list.
 |
 |  clear(self, /)
 |      Remove all items from list.
 |
 |  copy(self, /)
 |      Return a shallow copy of the list.
 |
 |  count(self, value, /)
 |      Return number of occurrences of value.
 |
 |  extend(self, iterable, /)
 |      Extend list by appending elements from the iterable.
 |
 |  index(self, value, start=0, stop=9223372036854775807, /)
 |      Return first index of value.
 |
 |      Raises ValueError if the value is not present.
 |
 |  insert(self, index, object, /)
 |      Insert object before index.
 |
 |  pop(self, index=-1, /)
 |      Remove and return item at index (default last).
 |
 |      Raises IndexError if list is empty or index is out of range.
 |
 |  remove(self, value, /)
 |      Remove first occurrence of value.
 |
 |      Raises ValueError if the value is not present.
 |
 |  reverse(self, /)
 |      Reverse *IN PLACE*.
 |
 |  sort(self, /, *, key=None, reverse=False)
 |      Sort the list in ascending order and return None.
 |
 |      The sort is in-place (i.e. the list itself is modified) and stable (i.e. the
 |      order of two equal elements is maintained).
 |
 |      If a key function is given, apply it once to each list item and sort them,
 |      ascending or descending, according to their function values.
 |
 |      The reverse flag can be set to sort in descending order.
 |
 |  ----------------------------------------------------------------------
 |  Class methods defined here:
 |
 |  __class_getitem__(...)
 |      See PEP 585
 |
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |
 |  __new__(*args, **kwargs)
 |      Create and return a new object.  See help(type) for accurate signature.
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |
 |  __hash__ = None

Many methods modify a data object in place#

Note in the above example of .append() that we didn’t need an assignment statement (i.e., a variable name and the = operator) to modify life_exp. That is, we didn’t write:

life_exp = life_exp.append(82.3)

We wrote:

life_exp.append(82.3)

In fact, the list .append() method modifies the list in place and returns None, so if we tried:

life_exp = life_exp.append(82.3)
print(life_exp)

The resulting value of life_exp would be:

None

which is definitely not what we want!

Extending a list#

  • the .append() method only allows you to add a single item to the end of a list

  • see what happens if you try to append a list to an existing list:

life_exp_2000s = [85.1, 87.3, 89.5, 91.6]
life_exp.append(life_exp_2000s)
print(life_exp)

[48.3, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3, [85.1, 87.3, 89.5, 91.6]]

The result is what we call a nested list, where the appended list is stored as a single list item — of type list — in the original list. This is a consequence of the fact that lists can contain multiple types.

In this case, what we want is a flat list, a single list of floats.

The .extend() method will do this. It is similar to append, but it allows you to combine two lists. For example:

life_exp = [48.1, 56.6, 64.0, 71.0, 75.2, 79.2]
life_exp_2000s = [82.3, 85.1, 87.3, 89.5, 91.6]

life_exp.extend(life_exp_2000s)
print(life_exp)

[48.1, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3, 85.1, 87.3, 89.5, 91.6]

Insert a new list entry at a specific position#

The result of help(list) above includes a method .insert(), which takes two arguments: a value to insert, and the index (position) in the list that you want the element inserted before. For example, to add the value 46.9 to the start of our life_exp list, we would use:

life_exp.insert(0, 46.9)
print(life_exp)

Removing items from a list#

There are two list methods that you can use to remove an element from a list:

  • .remove() will remove the first instance of a specific value you give it as an argument

  • .pop() will remove an element at a specific position (index) that you give as an argument

In the example below, we extend one list with another, but then we notice that the last value in the first list, is duplicated as the first value in the second list. As well, the last value in the second list is duplicated. So we need to remove one copy of each repeated value.

life_exp = [48.1, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3]
life_exp_2000s = [82.3, 85.1, 87.3, 89.5, 91.6, 91.6]
life_exp.extend(life_exp_2000s)
print('With duplicated values:', life_exp)

With duplicated values: [48.1, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3, 82.3, 85.1, 87.3, 89.5, 91.6, 91.6]

We will remove the value that’s duplicated in the two lists, 82.3, using .remove():

life_exp.remove(82.3)
print('Removed duplicate value:', life_exp)

Removed duplicate value: [48.1, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3, 85.1, 87.3, 89.5, 91.6, 91.6]

We will remove the value that’s duplicated at the end using .pop():

life_exp.pop(-1)
print('Removed duplicate value:', life_exp)

Removed duplicate value: [48.1, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3, 85.1, 87.3, 89.5, 91.6]

An empty list contains no values#

We can assign [] to a variable name, to represent a list that doesn’t contain any values.

This can be a good starting point if we want to create a list by appending values as we calculate them

The example below is trivial, but in later lessons we will see better uses for empty lists

x = []
y = 1
x.append(y)
x.append(y + 7)
print(x)

[1, 8]

Difference between strings and lists#

  • Both strings and lists are called collections in Python, so they can be indexed

  • Although indexing works similarly for string and lists, the two data types are different in other ways

  • Importantly, strings are immutable — they cannot be changed after they have been created

    • Python considers the string to be a single value with parts, not a collection of values

  • In contrast, lists are mutable — as we saw above, they can be changed such as by:

    • using indexing to change a single list element

    • appending items to a list with .append() or .extend()

    • deleting items from a list with del

So because we defined element above as a string, we cannot change one letter in the string:

element[0] = 'C'

element[0] = 'C'

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[16], line 1
----> 1 element[0] = 'C'

TypeError: 'str' object does not support item assignment

Likewise, you cannot .append() to a string, nor can you del a string element.

Indexing beyond the end of the collection is an error.#

Python reports an IndexError if we attempt to access a value that doesn’t exist.

  • This is a kind of runtime error, as discussed in the section on built-ins

  • It’s not a syntax error, because it cannot be detected as the code is parsed before being run; this is because the index might be calculated based on data

print('99th element of life_exp is:', life_exp[99])

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[17], line 1
----> 1 print('99th element of life_exp is:', life_exp[99])

IndexError: list index out of range

Exercises#

Fill in the Blanks#

Fill in the blanks so that the program below produces the output shown.

values = ____
values.____(1)
values.____(3)
values.____(5)
print('first time:', values)
values = values[____]
print('second time:', values)

Output:

first time: [1, 3, 5]

second time: [3, 5]

How Large is a Slice?#

Imagine we have a list like life_exp above. If a and b are both non-negative integers (e.g., 2 and 5), how long is the list life_exp[a:b]?

Click the plus to show the answer

Solution

Indexing works similarly for lists as we saw previously for strings: it returns from the first item specified in the range up to, but not including, the last item in the range. For example, values[2:5] has the 3 elements life_exp[2], life_exp[3], and life_exp[4]. Put another way, the list life_exp[a:b] has b - a elements.

Note that the expression will only work if b is less than the total length of the list life_exp.

From Strings to Lists and Back#

What does the list() function do when applied to a string?

For example:

list('tin')

And what does the .join() method do?

E.g.:

''.join(['g', 'o', 'l', 'd'])

What happens if you put a character inside the quote marks for .join(), like this?

'-'.join(['g', 'o', 'l', 'd'])

Click the plus to show the answer

Solution

  1. list('some string') converts a string into a list containing all of its characters.

  2. join returns a string that is the concatenation of each string element in the list and adds the separator between each element in the list. This results in x-y-z. The separator between the elements is the string that provides this method.

Indexing from the end#

What does the following program print?

life_exp = [48.1, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3]
print(life_exp[-1])

82.3

  1. How does Python interpret a negative index?

  2. If a list or string has N elements, what is the most negative index that can safely be used with it, and what location does that index represent?

  3. What does del life_exp[-1] do?

  4. How can you display all elements but the last one in a list (or string)? (Hint: you will need to combine slicing and negative indexing.)

Click the plus to show the answer

The program prints 82.3, which is the last element in the list.

  1. Python interprets a negative index as starting from the end (as opposed to starting from the beginning). The last element is -1.

  2. The last index that can safely be used with a list of N elements is element -N, which represents the first element.

  3. del life_exp[-1] removes the last element from the list.

  4. life_exp[:-1]

  1. If we write a slice as start:stop:step, what does step do?

  2. What expression would select all of the even-numbered items from a collection?

Click the plus to show the answer

  1. step is the step size of the slice

  2. The slice 1::2 selects all even-numbered items from a collection: it starts with element 1 (which is the second element, since indexing starts at 0), goes on until the end (since no end is given), and uses a step size of 2 (i.e., selects every second element).

Slice Bounds#

What does the following program print?

life_exp = [48.1, 56.6, 64.0, 71.0, 75.2, 79.2, 82.3]
print(life_exp[0:20])
print(life_exp[-1:3])

Even though we saw above that we get an error if we try to index a single element that is. outside the range of items in the list (e.g., print(life_exp[20]) would fail), we can use a value outside of the elements in a list when specifying a range.

# Program B
letters = list('gold')
letters_s = letters.sort()
print('letters is', letters, 'and letters_s is', letters_s)

Click the plus to show the answer

Solution

This example highlights the importance of understanding how individual functions and methods work.

Program A prints

letters is ['g', 'o', 'l', 'd'] and letters_s is ['d', 'g', 'l', 'o']

Program B prints

letters is ['d', 'g', 'l', 'o'] and letters_s is None

sorted() is a function returns a sorted copy of the input list letters; the original list letters remains unchanged. In contrast, .sort() is a method sorts the list letters in-place (it modifies its input) and does not return anything (hence None).

Copying (or Not)#

What do these two programs print? In simple terms, explain the difference between x = y and x = y[:].

# Program A
y = list('gold')
x = y      
x[0] = 'D'
print('x is', x, 'and y is', y)

# Program B
y = list('gold')
x = y[:]     
x[0] = 'D'
print('x is', x, 'and y is', y)

Click the plus to show the answer

Solution

Program A prints

x is ['D', 'o', 'l', 'd'] and y is ['D', 'o', 'l', 'd']

Program B prints

x is ['D', 'o', 'l', 'd'] and y is ['g', 'o', 'l', 'd']

x = y makes x a reference to the list y in memory; the variables x and y both point towards the same object. So when we change x, we also change y.

In contrast, x = y[:] creates a new list object x containing all elements from the list y; x and y are different objects, and so changing one has no effect on the other.

The fact that some operations create copies while others make new pointers to existing objects, is an important lesson to learn in Python. Different functions and methods work in different ways; the important thing for now is to be aware that both results could happen.

Although using x = y[:] works to create a copy of a list, it’s not the clearest, most explicit, or “pythonic” way to copy a list. A better approach is to use the list method .copy(). This is preferable because it’s easier to understand what the code is doing:

y = list('gold')
x = y.copy()   
x[0] = 'D'
print('x is', x, 'and y is', y)

Prints:

x is ['D', 'o', 'l', 'd'] and y is ['g', 'o', 'l', 'd']

Summary of Key Points:#

  • A list stores many values in a single structure

  • Lists may contain values of different types

  • The empty list contains no values

  • Use an item’s index to fetch it from a list

  • Lists’ values can be replaced by assigning to them

  • Appending items to a list lengthens it

  • if .append() is used to add a collection of items (such as a list), they are added as a single new list item

  • The list method .extend() can be used with a collection of items (such as another list) to create a flat list

  • The list methods .remove() and .pop() can be used to delete list items, using either their value or index, respectively

  • Character strings can be indexed like lists

  • Character strings are immutable, whereas lists are mutable (changable)

  • Indexing beyond the end of the collection is an error, but slicing can include indexes that do not occur in the list

This lesson is adapted from the Software Carpentry Plotting and Programming in Python workshop.