python - Are lists thread-safe?

ID : 20414

viewed : 29

Tags : pythonmultithreadinglistpython-3.xpython-multithreadingpython

Top 5 Answer for python - Are lists thread-safe?

vote vote

95

Lists themselves are thread-safe. In CPython the GIL protects against concurrent accesses to them, and other implementations take care to use a fine-grained lock or a synchronized datatype for their list implementations. However, while lists themselves can't go corrupt by attempts to concurrently access, the lists's data is not protected. For example:

L[0] += 1 

is not guaranteed to actually increase L[0] by one if another thread does the same thing, because += is not an atomic operation. (Very, very few operations in Python are actually atomic, because most of them can cause arbitrary Python code to be called.) You should use Queues because if you just use an unprotected list, you may get or delete the wrong item because of race conditions.

vote vote

80

To clarify a point in Thomas' excellent answer, it should be mentioned that append() is thread safe.

This is because there is no concern that data being read will be in the same place once we go to write to it. The append() operation does not read data, it only writes data to the list.

vote vote

80

Here's a comprehensive yet non-exhaustive list of examples of list operations and whether or not they are thread safe. Hoping to get an answer regarding the obj in a_list language construct here.

vote vote

67

I recently had this case where I needed to append to a list continuously in one thread, loop through the items and check if the item was ready, it was an AsyncResult in my case and remove it from the list only if it was ready. I could not find any examples that demonstrated my problem clearly Here is an example demonstrating adding to list in one thread continuously and removing from the same list in another thread continuously The flawed version runs easily on smaller numbers but keep the numbers big enough and run a few times and you will see the error

The FLAWED version

import threading import time  # Change this number as you please, bigger numbers will get the error quickly count = 1000 l = []  def add():     for i in range(count):         l.append(i)         time.sleep(0.0001)  def remove():     for i in range(count):         l.remove(i)         time.sleep(0.0001)   t1 = threading.Thread(target=add) t2 = threading.Thread(target=remove) t1.start() t2.start() t1.join() t2.join()  print(l) 

Output when ERROR

Exception in thread Thread-63: Traceback (most recent call last):   File "/Users/zup/.pyenv/versions/3.6.8/lib/python3.6/threading.py", line 916, in _bootstrap_inner     self.run()   File "/Users/zup/.pyenv/versions/3.6.8/lib/python3.6/threading.py", line 864, in run     self._target(*self._args, **self._kwargs)   File "<ipython-input-30-ecfbac1c776f>", line 13, in remove     l.remove(i) ValueError: list.remove(x): x not in list 

Version that uses locks

import threading import time count = 1000 l = [] lock = threading.RLock() def add():     with lock:         for i in range(count):             l.append(i)             time.sleep(0.0001)  def remove():     with lock:         for i in range(count):             l.remove(i)             time.sleep(0.0001)   t1 = threading.Thread(target=add) t2 = threading.Thread(target=remove) t1.start() t2.start() t1.join() t2.join()  print(l) 

Output

[] # Empty list 

Conclusion

As mentioned in the earlier answers while the act of appending or popping elements from the list itself is thread safe, what is not thread safe is when you append in one thread and pop in another

vote vote

53

Top 3 video Explaining python - Are lists thread-safe?

Related QUESTION?