tqdm with multiprocessing, threads and asyncio

The one-liner: process_map and thread_map

The fastest way to get a working bar is from tqdm.contrib.concurrent import process_map, thread_map. Both functions act like list(map(fn, *iterables)) running on a concurrent.futures ProcessPoolExecutor or ThreadPoolExecutor, returning a list in your original input order while drawing a single bar (tqdm.auto, with smoothing=0 by default).

import time
from tqdm.contrib.concurrent import process_map, thread_map

def crunch(n):
    return sum(i * i for i in range(n))

def wait(seconds):
    time.sleep(seconds)
    return seconds

if __name__ == "__main__":
    totals = process_map(crunch, [200_000] * 200, max_workers=4)
    waited = thread_map(wait, [0.05] * 200, max_workers=16)

If you don't pass max_workers, ProcessPoolExecutor defaults to your machine's CPU count, while ThreadPoolExecutor uses min(32, CPU count + 4). Windows and macOS start worker processes with spawn, and since Python 3.14 Linux and other POSIX systems use forkserver. Both import your main module again, so the pool call must sit inside an if __name__ == "__main__": block.

In tqdm 4.70.0 (released 2026-07-27), tqdm.contrib.concurrent was rewritten. That release added several pass-through options and a new helper:

• mp_context and max_tasks_per_child for process pools
• thread_name_prefix for thread pools
• timeout and buffersize (note that buffersize requires Python 3.14)
• interpreter_map for Python 3.14's InterpreterPoolExecutor

chunksize: speed versus an honest bar

By default, process_map hands items to worker processes with chunksize=1. When your shortest iterable has more than 1,000 items and you haven't set chunksize yourself, tqdm prints a warning like this one for 5,000 items:

TqdmWarning: Iterable length 5000 > 1000 but `chunksize` is not set. This may seriously degrade multiprocess performance. Set `chunksize=1` or more.

There's a catch in tqdm 4.70.0 and 4.70.1. Because of a regression in those versions (tracked in open tqdm issue #1791), setting chunksize above 1 makes process_map count chunks instead of individual items. When we ran 100 items with chunksize=10, the bar stopped at 10/100 even though all 100 results came back fine.

Version 4.69.1 counts items properly. Until issue #1791 is fixed, you have two workarounds whenever you need a chunksize above 1:

• Pin "tqdm<4.70" in your dependencies.
• Skip process_map and use Pool.imap with your own tqdm bar.

If you're using thread_map, none of this applies. thread_map ignores chunksize because ThreadPoolExecutor.map ignores it.

Pool.imap with tqdm

When you use multiprocessing.Pool directly, Pool.imap hands results back one at a time in input order and Pool.imap_unordered hands them back as they finish, so wrapping the call in tqdm(pool.imap(fn, items, chunksize=16), total=len(items)) gives you a bar that moves while the pool works and avoids the 4.70.x process_map bug.

from multiprocessing import Pool
from tqdm import tqdm

def crunch(n):
    return sum(i * i for i in range(n))

if __name__ == "__main__":
    items = [200_000] * 2_000
    with Pool(4) as pool:
        results = list(tqdm(pool.imap(crunch, items, chunksize=16), total=len(items)))

You have to pass total=len(items) because the iterator returned by imap has no len(), which otherwise leaves tqdm without a percentage or ETA. Use imap when you need results back in input order, and imap_unordered when you want results yielded in completion order as soon as each one finishes.

Why pool.map shows nothing until the end

pool.map(fn, items) blocks until every single result is ready and only then returns the full list. If you write tqdm(pool.map(fn, items)), tqdm only gets the list after the work is done: in our test the bar first appeared after 1.0 seconds, when the job had already finished, and filled instantly.

Wrapping the input iterable instead (pool.map(fn, tqdm(items))) doesn't work either. pool.map reads the whole input before it hands out tasks, so that bar reached 100% at 0.0 seconds while the workers ran for another second.

Pick an approach

Here is how the patterns compare on live progress, result order and the work they suit:

Use threads (thread_map) for I/O-bound work like downloads and API calls. Use processes (process_map or Pool.imap) for CPU-bound Python code, because of the GIL.

Bars inside worker processes and asyncio

Sometimes you don't want one summary bar for the pool, you want a separate bar inside each worker process. To keep workers from overwriting each other's terminal lines, give each worker bar its own position=n and share tqdm's lock across the pool by calling tqdm.set_lock(RLock()) in the parent and passing Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),)), as shown in tqdm's README.

from multiprocessing import Pool, RLock, freeze_support
from time import sleep
from tqdm import tqdm

def worker(n):
    for _ in tqdm(range(50), desc=f"worker {n}", position=n, leave=False):
        sleep(0.01)
    return n

if __name__ == "__main__":
    freeze_support()
    tqdm.set_lock(RLock())
    with Pool(4, initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),)) as pool:
        pool.map(worker, range(4))

If your concurrency is built on asyncio instead of threads or processes, import from tqdm.asyncio. You can await tqdm.gather(*coros) (which supports return_exceptions since tqdm 4.69.0) or iterate over tqdm.as_completed(coros).

import asyncio
from tqdm.asyncio import tqdm

async def fetch(i):
    await asyncio.sleep(0.05)
    return i

async def main():
    return await tqdm.gather(*(fetch(i) for i in range(100)), desc="requests")

results = asyncio.run(main())

Watch a worker pool from another machine

Runpeek's Python package, tqdm-live, makes your existing tqdm bar viewable in any browser without changing how it looks in your terminal. Swap from tqdm import tqdm for from tqdm_live import tqdm, or call tqdm_live.patch() once in the parent process before creating any bar, and each bar prints a runpeek ▶ https://runpeek.app/b/<uuid> link you can open on your phone. We verified this with process_map, thread_map and tqdm.asyncio gather bars; bars created inside child processes aren't streamed. Links expire 30 minutes after the last update, with no account needed.