1591 lines
56 KiB
Plaintext
1591 lines
56 KiB
Plaintext
Metadata-Version: 2.1
|
||
Name: tqdm
|
||
Version: 4.66.2
|
||
Summary: Fast, Extensible Progress Meter
|
||
Maintainer-email: tqdm developers <devs@tqdm.ml>
|
||
License: MPL-2.0 AND MIT
|
||
Project-URL: homepage, https://tqdm.github.io
|
||
Project-URL: repository, https://github.com/tqdm/tqdm
|
||
Project-URL: changelog, https://tqdm.github.io/releases
|
||
Project-URL: wiki, https://github.com/tqdm/tqdm/wiki
|
||
Keywords: progressbar,progressmeter,progress,bar,meter,rate,eta,console,terminal,time
|
||
Classifier: Development Status :: 5 - Production/Stable
|
||
Classifier: Environment :: Console
|
||
Classifier: Environment :: MacOS X
|
||
Classifier: Environment :: Other Environment
|
||
Classifier: Environment :: Win32 (MS Windows)
|
||
Classifier: Environment :: X11 Applications
|
||
Classifier: Framework :: IPython
|
||
Classifier: Framework :: Jupyter
|
||
Classifier: Intended Audience :: Developers
|
||
Classifier: Intended Audience :: Education
|
||
Classifier: Intended Audience :: End Users/Desktop
|
||
Classifier: Intended Audience :: Other Audience
|
||
Classifier: Intended Audience :: System Administrators
|
||
Classifier: License :: OSI Approved :: MIT License
|
||
Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
|
||
Classifier: Operating System :: MacOS
|
||
Classifier: Operating System :: MacOS :: MacOS X
|
||
Classifier: Operating System :: Microsoft
|
||
Classifier: Operating System :: Microsoft :: MS-DOS
|
||
Classifier: Operating System :: Microsoft :: Windows
|
||
Classifier: Operating System :: POSIX
|
||
Classifier: Operating System :: POSIX :: BSD
|
||
Classifier: Operating System :: POSIX :: BSD :: FreeBSD
|
||
Classifier: Operating System :: POSIX :: Linux
|
||
Classifier: Operating System :: POSIX :: SunOS/Solaris
|
||
Classifier: Operating System :: Unix
|
||
Classifier: Programming Language :: Python
|
||
Classifier: Programming Language :: Python :: 3
|
||
Classifier: Programming Language :: Python :: 3.7
|
||
Classifier: Programming Language :: Python :: 3.8
|
||
Classifier: Programming Language :: Python :: 3.9
|
||
Classifier: Programming Language :: Python :: 3.10
|
||
Classifier: Programming Language :: Python :: 3.11
|
||
Classifier: Programming Language :: Python :: 3 :: Only
|
||
Classifier: Programming Language :: Python :: Implementation
|
||
Classifier: Programming Language :: Python :: Implementation :: IronPython
|
||
Classifier: Programming Language :: Python :: Implementation :: PyPy
|
||
Classifier: Programming Language :: Unix Shell
|
||
Classifier: Topic :: Desktop Environment
|
||
Classifier: Topic :: Education :: Computer Aided Instruction (CAI)
|
||
Classifier: Topic :: Education :: Testing
|
||
Classifier: Topic :: Office/Business
|
||
Classifier: Topic :: Other/Nonlisted Topic
|
||
Classifier: Topic :: Software Development :: Build Tools
|
||
Classifier: Topic :: Software Development :: Libraries
|
||
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
||
Classifier: Topic :: Software Development :: Pre-processors
|
||
Classifier: Topic :: Software Development :: User Interfaces
|
||
Classifier: Topic :: System :: Installation/Setup
|
||
Classifier: Topic :: System :: Logging
|
||
Classifier: Topic :: System :: Monitoring
|
||
Classifier: Topic :: System :: Shells
|
||
Classifier: Topic :: Terminals
|
||
Classifier: Topic :: Utilities
|
||
Requires-Python: >=3.7
|
||
Description-Content-Type: text/x-rst
|
||
License-File: LICENCE
|
||
Requires-Dist: colorama ; platform_system == "Windows"
|
||
Provides-Extra: dev
|
||
Requires-Dist: pytest >=6 ; extra == 'dev'
|
||
Requires-Dist: pytest-cov ; extra == 'dev'
|
||
Requires-Dist: pytest-timeout ; extra == 'dev'
|
||
Requires-Dist: pytest-xdist ; extra == 'dev'
|
||
Provides-Extra: notebook
|
||
Requires-Dist: ipywidgets >=6 ; extra == 'notebook'
|
||
Provides-Extra: slack
|
||
Requires-Dist: slack-sdk ; extra == 'slack'
|
||
Provides-Extra: telegram
|
||
Requires-Dist: requests ; extra == 'telegram'
|
||
|
||
|Logo|
|
||
|
||
tqdm
|
||
====
|
||
|
||
|Py-Versions| |Versions| |Conda-Forge-Status| |Docker| |Snapcraft|
|
||
|
||
|Build-Status| |Coverage-Status| |Branch-Coverage-Status| |Codacy-Grade| |Libraries-Rank| |PyPI-Downloads|
|
||
|
||
|LICENCE| |OpenHub-Status| |binder-demo| |awesome-python|
|
||
|
||
``tqdm`` derives from the Arabic word *taqaddum* (تقدّم) which can mean "progress,"
|
||
and is an abbreviation for "I love you so much" in Spanish (*te quiero demasiado*).
|
||
|
||
Instantly make your loops show a smart progress meter - just wrap any
|
||
iterable with ``tqdm(iterable)``, and you're done!
|
||
|
||
.. code:: python
|
||
|
||
from tqdm import tqdm
|
||
for i in tqdm(range(10000)):
|
||
...
|
||
|
||
``76%|████████████████████████ | 7568/10000 [00:33<00:10, 229.00it/s]``
|
||
|
||
``trange(N)`` can be also used as a convenient shortcut for
|
||
``tqdm(range(N))``.
|
||
|
||
|Screenshot|
|
||
|Video| |Slides| |Merch|
|
||
|
||
It can also be executed as a module with pipes:
|
||
|
||
.. code:: sh
|
||
|
||
$ seq 9999999 | tqdm --bytes | wc -l
|
||
75.2MB [00:00, 217MB/s]
|
||
9999999
|
||
|
||
$ tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` \
|
||
> backup.tgz
|
||
32%|██████████▍ | 8.89G/27.9G [00:42<01:31, 223MB/s]
|
||
|
||
Overhead is low -- about 60ns per iteration (80ns with ``tqdm.gui``), and is
|
||
unit tested against performance regression.
|
||
By comparison, the well-established
|
||
`ProgressBar <https://github.com/niltonvolpato/python-progressbar>`__ has
|
||
an 800ns/iter overhead.
|
||
|
||
In addition to its low overhead, ``tqdm`` uses smart algorithms to predict
|
||
the remaining time and to skip unnecessary iteration displays, which allows
|
||
for a negligible overhead in most cases.
|
||
|
||
``tqdm`` works on any platform
|
||
(Linux, Windows, Mac, FreeBSD, NetBSD, Solaris/SunOS),
|
||
in any console or in a GUI, and is also friendly with IPython/Jupyter notebooks.
|
||
|
||
``tqdm`` does not require any dependencies (not even ``curses``!), just
|
||
Python and an environment supporting ``carriage return \r`` and
|
||
``line feed \n`` control characters.
|
||
|
||
------------------------------------------
|
||
|
||
.. contents:: Table of contents
|
||
:backlinks: top
|
||
:local:
|
||
|
||
|
||
Installation
|
||
------------
|
||
|
||
Latest PyPI stable release
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
|Versions| |PyPI-Downloads| |Libraries-Dependents|
|
||
|
||
.. code:: sh
|
||
|
||
pip install tqdm
|
||
|
||
Latest development release on GitHub
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
|GitHub-Status| |GitHub-Stars| |GitHub-Commits| |GitHub-Forks| |GitHub-Updated|
|
||
|
||
Pull and install pre-release ``devel`` branch:
|
||
|
||
.. code:: sh
|
||
|
||
pip install "git+https://github.com/tqdm/tqdm.git@devel#egg=tqdm"
|
||
|
||
Latest Conda release
|
||
~~~~~~~~~~~~~~~~~~~~
|
||
|
||
|Conda-Forge-Status|
|
||
|
||
.. code:: sh
|
||
|
||
conda install -c conda-forge tqdm
|
||
|
||
Latest Snapcraft release
|
||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
|Snapcraft|
|
||
|
||
There are 3 channels to choose from:
|
||
|
||
.. code:: sh
|
||
|
||
snap install tqdm # implies --stable, i.e. latest tagged release
|
||
snap install tqdm --candidate # master branch
|
||
snap install tqdm --edge # devel branch
|
||
|
||
Note that ``snap`` binaries are purely for CLI use (not ``import``-able), and
|
||
automatically set up ``bash`` tab-completion.
|
||
|
||
Latest Docker release
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
|Docker|
|
||
|
||
.. code:: sh
|
||
|
||
docker pull tqdm/tqdm
|
||
docker run -i --rm tqdm/tqdm --help
|
||
|
||
Other
|
||
~~~~~
|
||
|
||
There are other (unofficial) places where ``tqdm`` may be downloaded, particularly for CLI use:
|
||
|
||
|Repology|
|
||
|
||
.. |Repology| image:: https://repology.org/badge/tiny-repos/python:tqdm.svg
|
||
:target: https://repology.org/project/python:tqdm/versions
|
||
|
||
Changelog
|
||
---------
|
||
|
||
The list of all changes is available either on GitHub's Releases:
|
||
|GitHub-Status|, on the
|
||
`wiki <https://github.com/tqdm/tqdm/wiki/Releases>`__, or on the
|
||
`website <https://tqdm.github.io/releases>`__.
|
||
|
||
|
||
Usage
|
||
-----
|
||
|
||
``tqdm`` is very versatile and can be used in a number of ways.
|
||
The three main ones are given below.
|
||
|
||
Iterable-based
|
||
~~~~~~~~~~~~~~
|
||
|
||
Wrap ``tqdm()`` around any iterable:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm import tqdm
|
||
from time import sleep
|
||
|
||
text = ""
|
||
for char in tqdm(["a", "b", "c", "d"]):
|
||
sleep(0.25)
|
||
text = text + char
|
||
|
||
``trange(i)`` is a special optimised instance of ``tqdm(range(i))``:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm import trange
|
||
|
||
for i in trange(100):
|
||
sleep(0.01)
|
||
|
||
Instantiation outside of the loop allows for manual control over ``tqdm()``:
|
||
|
||
.. code:: python
|
||
|
||
pbar = tqdm(["a", "b", "c", "d"])
|
||
for char in pbar:
|
||
sleep(0.25)
|
||
pbar.set_description("Processing %s" % char)
|
||
|
||
Manual
|
||
~~~~~~
|
||
|
||
Manual control of ``tqdm()`` updates using a ``with`` statement:
|
||
|
||
.. code:: python
|
||
|
||
with tqdm(total=100) as pbar:
|
||
for i in range(10):
|
||
sleep(0.1)
|
||
pbar.update(10)
|
||
|
||
If the optional variable ``total`` (or an iterable with ``len()``) is
|
||
provided, predictive stats are displayed.
|
||
|
||
``with`` is also optional (you can just assign ``tqdm()`` to a variable,
|
||
but in this case don't forget to ``del`` or ``close()`` at the end:
|
||
|
||
.. code:: python
|
||
|
||
pbar = tqdm(total=100)
|
||
for i in range(10):
|
||
sleep(0.1)
|
||
pbar.update(10)
|
||
pbar.close()
|
||
|
||
Module
|
||
~~~~~~
|
||
|
||
Perhaps the most wonderful use of ``tqdm`` is in a script or on the command
|
||
line. Simply inserting ``tqdm`` (or ``python -m tqdm``) between pipes will pass
|
||
through all ``stdin`` to ``stdout`` while printing progress to ``stderr``.
|
||
|
||
The example below demonstrate counting the number of lines in all Python files
|
||
in the current directory, with timing information included.
|
||
|
||
.. code:: sh
|
||
|
||
$ time find . -name '*.py' -type f -exec cat \{} \; | wc -l
|
||
857365
|
||
|
||
real 0m3.458s
|
||
user 0m0.274s
|
||
sys 0m3.325s
|
||
|
||
$ time find . -name '*.py' -type f -exec cat \{} \; | tqdm | wc -l
|
||
857366it [00:03, 246471.31it/s]
|
||
857365
|
||
|
||
real 0m3.585s
|
||
user 0m0.862s
|
||
sys 0m3.358s
|
||
|
||
Note that the usual arguments for ``tqdm`` can also be specified.
|
||
|
||
.. code:: sh
|
||
|
||
$ find . -name '*.py' -type f -exec cat \{} \; |
|
||
tqdm --unit loc --unit_scale --total 857366 >> /dev/null
|
||
100%|█████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s]
|
||
|
||
Backing up a large directory?
|
||
|
||
.. code:: sh
|
||
|
||
$ tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` \
|
||
> backup.tgz
|
||
44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]
|
||
|
||
This can be beautified further:
|
||
|
||
.. code:: sh
|
||
|
||
$ BYTES=$(du -sb docs/ | cut -f1)
|
||
$ tar -cf - docs/ \
|
||
| tqdm --bytes --total "$BYTES" --desc Processing | gzip \
|
||
| tqdm --bytes --total "$BYTES" --desc Compressed --position 1 \
|
||
> ~/backup.tgz
|
||
Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]
|
||
Compressed: 42%|█████████▎ | 148M/352M [00:14<00:19, 10.9MB/s]
|
||
|
||
Or done on a file level using 7-zip:
|
||
|
||
.. code:: sh
|
||
|
||
$ 7z a -bd -r backup.7z docs/ | grep Compressing \
|
||
| tqdm --total $(find docs/ -type f | wc -l) --unit files \
|
||
| grep -v Compressing
|
||
100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]
|
||
|
||
Pre-existing CLI programs already outputting basic progress information will
|
||
benefit from ``tqdm``'s ``--update`` and ``--update_to`` flags:
|
||
|
||
.. code:: sh
|
||
|
||
$ seq 3 0.1 5 | tqdm --total 5 --update_to --null
|
||
100%|████████████████████████████████████| 5.0/5 [00:00<00:00, 9673.21it/s]
|
||
$ seq 10 | tqdm --update --null # 1 + 2 + ... + 10 = 55 iterations
|
||
55it [00:00, 90006.52it/s]
|
||
|
||
FAQ and Known Issues
|
||
--------------------
|
||
|
||
|GitHub-Issues|
|
||
|
||
The most common issues relate to excessive output on multiple lines, instead
|
||
of a neat one-line progress bar.
|
||
|
||
- Consoles in general: require support for carriage return (``CR``, ``\r``).
|
||
|
||
* Some cloud logging consoles which don't support ``\r`` properly
|
||
(`cloudwatch <https://github.com/tqdm/tqdm/issues/966>`__,
|
||
`K8s <https://github.com/tqdm/tqdm/issues/1319>`__) may benefit from
|
||
``export TQDM_POSITION=-1``.
|
||
|
||
- Nested progress bars:
|
||
|
||
* Consoles in general: require support for moving cursors up to the
|
||
previous line. For example,
|
||
`IDLE <https://github.com/tqdm/tqdm/issues/191#issuecomment-230168030>`__,
|
||
`ConEmu <https://github.com/tqdm/tqdm/issues/254>`__ and
|
||
`PyCharm <https://github.com/tqdm/tqdm/issues/203>`__ (also
|
||
`here <https://github.com/tqdm/tqdm/issues/208>`__,
|
||
`here <https://github.com/tqdm/tqdm/issues/307>`__, and
|
||
`here <https://github.com/tqdm/tqdm/issues/454#issuecomment-335416815>`__)
|
||
lack full support.
|
||
* Windows: additionally may require the Python module ``colorama``
|
||
to ensure nested bars stay within their respective lines.
|
||
|
||
- Unicode:
|
||
|
||
* Environments which report that they support unicode will have solid smooth
|
||
progressbars. The fallback is an ``ascii``-only bar.
|
||
* Windows consoles often only partially support unicode and thus
|
||
`often require explicit ascii=True <https://github.com/tqdm/tqdm/issues/454#issuecomment-335416815>`__
|
||
(also `here <https://github.com/tqdm/tqdm/issues/499>`__). This is due to
|
||
either normal-width unicode characters being incorrectly displayed as
|
||
"wide", or some unicode characters not rendering.
|
||
|
||
- Wrapping generators:
|
||
|
||
* Generator wrapper functions tend to hide the length of iterables.
|
||
``tqdm`` does not.
|
||
* Replace ``tqdm(enumerate(...))`` with ``enumerate(tqdm(...))`` or
|
||
``tqdm(enumerate(x), total=len(x), ...)``.
|
||
The same applies to ``numpy.ndenumerate``.
|
||
* Replace ``tqdm(zip(a, b))`` with ``zip(tqdm(a), b)`` or even
|
||
``zip(tqdm(a), tqdm(b))``.
|
||
* The same applies to ``itertools``.
|
||
* Some useful convenience functions can be found under ``tqdm.contrib``.
|
||
|
||
- `No intermediate output in docker-compose <https://github.com/tqdm/tqdm/issues/771>`__:
|
||
use ``docker-compose run`` instead of ``docker-compose up`` and ``tty: true``.
|
||
|
||
- Overriding defaults via environment variables:
|
||
e.g. in CI/cloud jobs, ``export TQDM_MININTERVAL=5`` to avoid log spam.
|
||
This override logic is handled by the ``tqdm.utils.envwrap`` decorator
|
||
(useful independent of ``tqdm``).
|
||
|
||
If you come across any other difficulties, browse and file |GitHub-Issues|.
|
||
|
||
Documentation
|
||
-------------
|
||
|
||
|Py-Versions| |README-Hits| (Since 19 May 2016)
|
||
|
||
.. code:: python
|
||
|
||
class tqdm():
|
||
"""
|
||
Decorate an iterable object, returning an iterator which acts exactly
|
||
like the original iterable, but prints a dynamically updating
|
||
progressbar every time a value is requested.
|
||
"""
|
||
|
||
@envwrap("TQDM_") # override defaults via env vars
|
||
def __init__(self, iterable=None, desc=None, total=None, leave=True,
|
||
file=None, ncols=None, mininterval=0.1,
|
||
maxinterval=10.0, miniters=None, ascii=None, disable=False,
|
||
unit='it', unit_scale=False, dynamic_ncols=False,
|
||
smoothing=0.3, bar_format=None, initial=0, position=None,
|
||
postfix=None, unit_divisor=1000, write_bytes=False,
|
||
lock_args=None, nrows=None, colour=None, delay=0):
|
||
|
||
Parameters
|
||
~~~~~~~~~~
|
||
|
||
* iterable : iterable, optional
|
||
Iterable to decorate with a progressbar.
|
||
Leave blank to manually manage the updates.
|
||
* desc : str, optional
|
||
Prefix for the progressbar.
|
||
* total : int or float, optional
|
||
The number of expected iterations. If unspecified,
|
||
len(iterable) is used if possible. If float("inf") or as a last
|
||
resort, only basic progress statistics are displayed
|
||
(no ETA, no progressbar).
|
||
If ``gui`` is True and this parameter needs subsequent updating,
|
||
specify an initial arbitrary large positive number,
|
||
e.g. 9e9.
|
||
* leave : bool, optional
|
||
If [default: True], keeps all traces of the progressbar
|
||
upon termination of iteration.
|
||
If ``None``, will leave only if ``position`` is ``0``.
|
||
* file : ``io.TextIOWrapper`` or ``io.StringIO``, optional
|
||
Specifies where to output the progress messages
|
||
(default: sys.stderr). Uses ``file.write(str)`` and ``file.flush()``
|
||
methods. For encoding, see ``write_bytes``.
|
||
* ncols : int, optional
|
||
The width of the entire output message. If specified,
|
||
dynamically resizes the progressbar to stay within this bound.
|
||
If unspecified, attempts to use environment width. The
|
||
fallback is a meter width of 10 and no limit for the counter and
|
||
statistics. If 0, will not print any meter (only stats).
|
||
* mininterval : float, optional
|
||
Minimum progress display update interval [default: 0.1] seconds.
|
||
* maxinterval : float, optional
|
||
Maximum progress display update interval [default: 10] seconds.
|
||
Automatically adjusts ``miniters`` to correspond to ``mininterval``
|
||
after long display update lag. Only works if ``dynamic_miniters``
|
||
or monitor thread is enabled.
|
||
* miniters : int or float, optional
|
||
Minimum progress display update interval, in iterations.
|
||
If 0 and ``dynamic_miniters``, will automatically adjust to equal
|
||
``mininterval`` (more CPU efficient, good for tight loops).
|
||
If > 0, will skip display of specified number of iterations.
|
||
Tweak this and ``mininterval`` to get very efficient loops.
|
||
If your progress is erratic with both fast and slow iterations
|
||
(network, skipping items, etc) you should set miniters=1.
|
||
* ascii : bool or str, optional
|
||
If unspecified or False, use unicode (smooth blocks) to fill
|
||
the meter. The fallback is to use ASCII characters " 123456789#".
|
||
* disable : bool, optional
|
||
Whether to disable the entire progressbar wrapper
|
||
[default: False]. If set to None, disable on non-TTY.
|
||
* unit : str, optional
|
||
String that will be used to define the unit of each iteration
|
||
[default: it].
|
||
* unit_scale : bool or int or float, optional
|
||
If 1 or True, the number of iterations will be reduced/scaled
|
||
automatically and a metric prefix following the
|
||
International System of Units standard will be added
|
||
(kilo, mega, etc.) [default: False]. If any other non-zero
|
||
number, will scale ``total`` and ``n``.
|
||
* dynamic_ncols : bool, optional
|
||
If set, constantly alters ``ncols`` and ``nrows`` to the
|
||
environment (allowing for window resizes) [default: False].
|
||
* smoothing : float, optional
|
||
Exponential moving average smoothing factor for speed estimates
|
||
(ignored in GUI mode). Ranges from 0 (average speed) to 1
|
||
(current/instantaneous speed) [default: 0.3].
|
||
* bar_format : str, optional
|
||
Specify a custom bar string formatting. May impact performance.
|
||
[default: '{l_bar}{bar}{r_bar}'], where
|
||
l_bar='{desc}: {percentage:3.0f}%|' and
|
||
r_bar='| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, '
|
||
'{rate_fmt}{postfix}]'
|
||
Possible vars: l_bar, bar, r_bar, n, n_fmt, total, total_fmt,
|
||
percentage, elapsed, elapsed_s, ncols, nrows, desc, unit,
|
||
rate, rate_fmt, rate_noinv, rate_noinv_fmt,
|
||
rate_inv, rate_inv_fmt, postfix, unit_divisor,
|
||
remaining, remaining_s, eta.
|
||
Note that a trailing ": " is automatically removed after {desc}
|
||
if the latter is empty.
|
||
* initial : int or float, optional
|
||
The initial counter value. Useful when restarting a progress
|
||
bar [default: 0]. If using float, consider specifying ``{n:.3f}``
|
||
or similar in ``bar_format``, or specifying ``unit_scale``.
|
||
* position : int, optional
|
||
Specify the line offset to print this bar (starting from 0)
|
||
Automatic if unspecified.
|
||
Useful to manage multiple bars at once (eg, from threads).
|
||
* postfix : dict or ``*``, optional
|
||
Specify additional stats to display at the end of the bar.
|
||
Calls ``set_postfix(**postfix)`` if possible (dict).
|
||
* unit_divisor : float, optional
|
||
[default: 1000], ignored unless ``unit_scale`` is True.
|
||
* write_bytes : bool, optional
|
||
Whether to write bytes. If (default: False) will write unicode.
|
||
* lock_args : tuple, optional
|
||
Passed to ``refresh`` for intermediate output
|
||
(initialisation, iterating, and updating).
|
||
* nrows : int, optional
|
||
The screen height. If specified, hides nested bars outside this
|
||
bound. If unspecified, attempts to use environment height.
|
||
The fallback is 20.
|
||
* colour : str, optional
|
||
Bar colour (e.g. 'green', '#00ff00').
|
||
* delay : float, optional
|
||
Don't display until [default: 0] seconds have elapsed.
|
||
|
||
Extra CLI Options
|
||
~~~~~~~~~~~~~~~~~
|
||
|
||
* delim : chr, optional
|
||
Delimiting character [default: '\n']. Use '\0' for null.
|
||
N.B.: on Windows systems, Python converts '\n' to '\r\n'.
|
||
* buf_size : int, optional
|
||
String buffer size in bytes [default: 256]
|
||
used when ``delim`` is specified.
|
||
* bytes : bool, optional
|
||
If true, will count bytes, ignore ``delim``, and default
|
||
``unit_scale`` to True, ``unit_divisor`` to 1024, and ``unit`` to 'B'.
|
||
* tee : bool, optional
|
||
If true, passes ``stdin`` to both ``stderr`` and ``stdout``.
|
||
* update : bool, optional
|
||
If true, will treat input as newly elapsed iterations,
|
||
i.e. numbers to pass to ``update()``. Note that this is slow
|
||
(~2e5 it/s) since every input must be decoded as a number.
|
||
* update_to : bool, optional
|
||
If true, will treat input as total elapsed iterations,
|
||
i.e. numbers to assign to ``self.n``. Note that this is slow
|
||
(~2e5 it/s) since every input must be decoded as a number.
|
||
* null : bool, optional
|
||
If true, will discard input (no stdout).
|
||
* manpath : str, optional
|
||
Directory in which to install tqdm man pages.
|
||
* comppath : str, optional
|
||
Directory in which to place tqdm completion.
|
||
* log : str, optional
|
||
CRITICAL|FATAL|ERROR|WARN(ING)|[default: 'INFO']|DEBUG|NOTSET.
|
||
|
||
Returns
|
||
~~~~~~~
|
||
|
||
* out : decorated iterator.
|
||
|
||
.. code:: python
|
||
|
||
class tqdm():
|
||
def update(self, n=1):
|
||
"""
|
||
Manually update the progress bar, useful for streams
|
||
such as reading files.
|
||
E.g.:
|
||
>>> t = tqdm(total=filesize) # Initialise
|
||
>>> for current_buffer in stream:
|
||
... ...
|
||
... t.update(len(current_buffer))
|
||
>>> t.close()
|
||
The last line is highly recommended, but possibly not necessary if
|
||
``t.update()`` will be called in such a way that ``filesize`` will be
|
||
exactly reached and printed.
|
||
|
||
Parameters
|
||
----------
|
||
n : int or float, optional
|
||
Increment to add to the internal counter of iterations
|
||
[default: 1]. If using float, consider specifying ``{n:.3f}``
|
||
or similar in ``bar_format``, or specifying ``unit_scale``.
|
||
|
||
Returns
|
||
-------
|
||
out : bool or None
|
||
True if a ``display()`` was triggered.
|
||
"""
|
||
|
||
def close(self):
|
||
"""Cleanup and (if leave=False) close the progressbar."""
|
||
|
||
def clear(self, nomove=False):
|
||
"""Clear current bar display."""
|
||
|
||
def refresh(self):
|
||
"""
|
||
Force refresh the display of this bar.
|
||
|
||
Parameters
|
||
----------
|
||
nolock : bool, optional
|
||
If ``True``, does not lock.
|
||
If [default: ``False``]: calls ``acquire()`` on internal lock.
|
||
lock_args : tuple, optional
|
||
Passed to internal lock's ``acquire()``.
|
||
If specified, will only ``display()`` if ``acquire()`` returns ``True``.
|
||
"""
|
||
|
||
def unpause(self):
|
||
"""Restart tqdm timer from last print time."""
|
||
|
||
def reset(self, total=None):
|
||
"""
|
||
Resets to 0 iterations for repeated use.
|
||
|
||
Consider combining with ``leave=True``.
|
||
|
||
Parameters
|
||
----------
|
||
total : int or float, optional. Total to use for the new bar.
|
||
"""
|
||
|
||
def set_description(self, desc=None, refresh=True):
|
||
"""
|
||
Set/modify description of the progress bar.
|
||
|
||
Parameters
|
||
----------
|
||
desc : str, optional
|
||
refresh : bool, optional
|
||
Forces refresh [default: True].
|
||
"""
|
||
|
||
def set_postfix(self, ordered_dict=None, refresh=True, **tqdm_kwargs):
|
||
"""
|
||
Set/modify postfix (additional stats)
|
||
with automatic formatting based on datatype.
|
||
|
||
Parameters
|
||
----------
|
||
ordered_dict : dict or OrderedDict, optional
|
||
refresh : bool, optional
|
||
Forces refresh [default: True].
|
||
kwargs : dict, optional
|
||
"""
|
||
|
||
@classmethod
|
||
def write(cls, s, file=sys.stdout, end="\n"):
|
||
"""Print a message via tqdm (without overlap with bars)."""
|
||
|
||
@property
|
||
def format_dict(self):
|
||
"""Public API for read-only member access."""
|
||
|
||
def display(self, msg=None, pos=None):
|
||
"""
|
||
Use ``self.sp`` to display ``msg`` in the specified ``pos``.
|
||
|
||
Consider overloading this function when inheriting to use e.g.:
|
||
``self.some_frontend(**self.format_dict)`` instead of ``self.sp``.
|
||
|
||
Parameters
|
||
----------
|
||
msg : str, optional. What to display (default: ``repr(self)``).
|
||
pos : int, optional. Position to ``moveto``
|
||
(default: ``abs(self.pos)``).
|
||
"""
|
||
|
||
@classmethod
|
||
@contextmanager
|
||
def wrapattr(cls, stream, method, total=None, bytes=True, **tqdm_kwargs):
|
||
"""
|
||
stream : file-like object.
|
||
method : str, "read" or "write". The result of ``read()`` and
|
||
the first argument of ``write()`` should have a ``len()``.
|
||
|
||
>>> with tqdm.wrapattr(file_obj, "read", total=file_obj.size) as fobj:
|
||
... while True:
|
||
... chunk = fobj.read(chunk_size)
|
||
... if not chunk:
|
||
... break
|
||
"""
|
||
|
||
@classmethod
|
||
def pandas(cls, *targs, **tqdm_kwargs):
|
||
"""Registers the current `tqdm` class with `pandas`."""
|
||
|
||
def trange(*args, **tqdm_kwargs):
|
||
"""Shortcut for `tqdm(range(*args), **tqdm_kwargs)`."""
|
||
|
||
Convenience Functions
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
.. code:: python
|
||
|
||
def tqdm.contrib.tenumerate(iterable, start=0, total=None,
|
||
tqdm_class=tqdm.auto.tqdm, **tqdm_kwargs):
|
||
"""Equivalent of `numpy.ndenumerate` or builtin `enumerate`."""
|
||
|
||
def tqdm.contrib.tzip(iter1, *iter2plus, **tqdm_kwargs):
|
||
"""Equivalent of builtin `zip`."""
|
||
|
||
def tqdm.contrib.tmap(function, *sequences, **tqdm_kwargs):
|
||
"""Equivalent of builtin `map`."""
|
||
|
||
Submodules
|
||
~~~~~~~~~~
|
||
|
||
.. code:: python
|
||
|
||
class tqdm.notebook.tqdm(tqdm.tqdm):
|
||
"""IPython/Jupyter Notebook widget."""
|
||
|
||
class tqdm.auto.tqdm(tqdm.tqdm):
|
||
"""Automatically chooses beween `tqdm.notebook` and `tqdm.tqdm`."""
|
||
|
||
class tqdm.asyncio.tqdm(tqdm.tqdm):
|
||
"""Asynchronous version."""
|
||
@classmethod
|
||
def as_completed(cls, fs, *, loop=None, timeout=None, total=None,
|
||
**tqdm_kwargs):
|
||
"""Wrapper for `asyncio.as_completed`."""
|
||
|
||
class tqdm.gui.tqdm(tqdm.tqdm):
|
||
"""Matplotlib GUI version."""
|
||
|
||
class tqdm.tk.tqdm(tqdm.tqdm):
|
||
"""Tkinter GUI version."""
|
||
|
||
class tqdm.rich.tqdm(tqdm.tqdm):
|
||
"""`rich.progress` version."""
|
||
|
||
class tqdm.keras.TqdmCallback(keras.callbacks.Callback):
|
||
"""Keras callback for epoch and batch progress."""
|
||
|
||
class tqdm.dask.TqdmCallback(dask.callbacks.Callback):
|
||
"""Dask callback for task progress."""
|
||
|
||
|
||
``contrib``
|
||
+++++++++++
|
||
|
||
The ``tqdm.contrib`` package also contains experimental modules:
|
||
|
||
- ``tqdm.contrib.itertools``: Thin wrappers around ``itertools``
|
||
- ``tqdm.contrib.concurrent``: Thin wrappers around ``concurrent.futures``
|
||
- ``tqdm.contrib.slack``: Posts to `Slack <https://slack.com>`__ bots
|
||
- ``tqdm.contrib.discord``: Posts to `Discord <https://discord.com>`__ bots
|
||
- ``tqdm.contrib.telegram``: Posts to `Telegram <https://telegram.org>`__ bots
|
||
- ``tqdm.contrib.bells``: Automagically enables all optional features
|
||
|
||
* ``auto``, ``pandas``, ``slack``, ``discord``, ``telegram``
|
||
|
||
Examples and Advanced Usage
|
||
---------------------------
|
||
|
||
- See the `examples <https://github.com/tqdm/tqdm/tree/master/examples>`__
|
||
folder;
|
||
- import the module and run ``help()``;
|
||
- consult the `wiki <https://github.com/tqdm/tqdm/wiki>`__;
|
||
|
||
* this has an
|
||
`excellent article <https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar>`__
|
||
on how to make a **great** progressbar;
|
||
|
||
- check out the `slides from PyData London <https://tqdm.github.io/PyData2019/slides.html>`__, or
|
||
- run the |binder-demo|.
|
||
|
||
Description and additional stats
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Custom information can be displayed and updated dynamically on ``tqdm`` bars
|
||
with the ``desc`` and ``postfix`` arguments:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm import tqdm, trange
|
||
from random import random, randint
|
||
from time import sleep
|
||
|
||
with trange(10) as t:
|
||
for i in t:
|
||
# Description will be displayed on the left
|
||
t.set_description('GEN %i' % i)
|
||
# Postfix will be displayed on the right,
|
||
# formatted automatically based on argument's datatype
|
||
t.set_postfix(loss=random(), gen=randint(1,999), str='h',
|
||
lst=[1, 2])
|
||
sleep(0.1)
|
||
|
||
with tqdm(total=10, bar_format="{postfix[0]} {postfix[1][value]:>8.2g}",
|
||
postfix=["Batch", {"value": 0}]) as t:
|
||
for i in range(10):
|
||
sleep(0.1)
|
||
t.postfix[1]["value"] = i / 2
|
||
t.update()
|
||
|
||
Points to remember when using ``{postfix[...]}`` in the ``bar_format`` string:
|
||
|
||
- ``postfix`` also needs to be passed as an initial argument in a compatible
|
||
format, and
|
||
- ``postfix`` will be auto-converted to a string if it is a ``dict``-like
|
||
object. To prevent this behaviour, insert an extra item into the dictionary
|
||
where the key is not a string.
|
||
|
||
Additional ``bar_format`` parameters may also be defined by overriding
|
||
``format_dict``, and the bar itself may be modified using ``ascii``:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm import tqdm
|
||
class TqdmExtraFormat(tqdm):
|
||
"""Provides a `total_time` format parameter"""
|
||
@property
|
||
def format_dict(self):
|
||
d = super(TqdmExtraFormat, self).format_dict
|
||
total_time = d["elapsed"] * (d["total"] or 0) / max(d["n"], 1)
|
||
d.update(total_time=self.format_interval(total_time) + " in total")
|
||
return d
|
||
|
||
for i in TqdmExtraFormat(
|
||
range(9), ascii=" .oO0",
|
||
bar_format="{total_time}: {percentage:.0f}%|{bar}{r_bar}"):
|
||
if i == 4:
|
||
break
|
||
|
||
.. code::
|
||
|
||
00:00 in total: 44%|0000. | 4/9 [00:00<00:00, 962.93it/s]
|
||
|
||
Note that ``{bar}`` also supports a format specifier ``[width][type]``.
|
||
|
||
- ``width``
|
||
|
||
* unspecified (default): automatic to fill ``ncols``
|
||
* ``int >= 0``: fixed width overriding ``ncols`` logic
|
||
* ``int < 0``: subtract from the automatic default
|
||
|
||
- ``type``
|
||
|
||
* ``a``: ascii (``ascii=True`` override)
|
||
* ``u``: unicode (``ascii=False`` override)
|
||
* ``b``: blank (``ascii=" "`` override)
|
||
|
||
This means a fixed bar with right-justified text may be created by using:
|
||
``bar_format="{l_bar}{bar:10}|{bar:-10b}right-justified"``
|
||
|
||
Nested progress bars
|
||
~~~~~~~~~~~~~~~~~~~~
|
||
|
||
``tqdm`` supports nested progress bars. Here's an example:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.auto import trange
|
||
from time import sleep
|
||
|
||
for i in trange(4, desc='1st loop'):
|
||
for j in trange(5, desc='2nd loop'):
|
||
for k in trange(50, desc='3rd loop', leave=False):
|
||
sleep(0.01)
|
||
|
||
For manual control over positioning (e.g. for multi-processing use),
|
||
you may specify ``position=n`` where ``n=0`` for the outermost bar,
|
||
``n=1`` for the next, and so on.
|
||
However, it's best to check if ``tqdm`` can work without manual ``position``
|
||
first.
|
||
|
||
.. code:: python
|
||
|
||
from time import sleep
|
||
from tqdm import trange, tqdm
|
||
from multiprocessing import Pool, RLock, freeze_support
|
||
|
||
L = list(range(9))
|
||
|
||
def progresser(n):
|
||
interval = 0.001 / (n + 2)
|
||
total = 5000
|
||
text = f"#{n}, est. {interval * total:<04.2}s"
|
||
for _ in trange(total, desc=text, position=n):
|
||
sleep(interval)
|
||
|
||
if __name__ == '__main__':
|
||
freeze_support() # for Windows support
|
||
tqdm.set_lock(RLock()) # for managing output contention
|
||
p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))
|
||
p.map(progresser, L)
|
||
|
||
Note that in Python 3, ``tqdm.write`` is thread-safe:
|
||
|
||
.. code:: python
|
||
|
||
from time import sleep
|
||
from tqdm import tqdm, trange
|
||
from concurrent.futures import ThreadPoolExecutor
|
||
|
||
L = list(range(9))
|
||
|
||
def progresser(n):
|
||
interval = 0.001 / (n + 2)
|
||
total = 5000
|
||
text = f"#{n}, est. {interval * total:<04.2}s"
|
||
for _ in trange(total, desc=text):
|
||
sleep(interval)
|
||
if n == 6:
|
||
tqdm.write("n == 6 completed.")
|
||
tqdm.write("`tqdm.write()` is thread-safe in py3!")
|
||
|
||
if __name__ == '__main__':
|
||
with ThreadPoolExecutor() as p:
|
||
p.map(progresser, L)
|
||
|
||
Hooks and callbacks
|
||
~~~~~~~~~~~~~~~~~~~
|
||
|
||
``tqdm`` can easily support callbacks/hooks and manual updates.
|
||
Here's an example with ``urllib``:
|
||
|
||
**``urllib.urlretrieve`` documentation**
|
||
|
||
| [...]
|
||
| If present, the hook function will be called once
|
||
| on establishment of the network connection and once after each block read
|
||
| thereafter. The hook will be passed three arguments; a count of blocks
|
||
| transferred so far, a block size in bytes, and the total size of the file.
|
||
| [...]
|
||
|
||
.. code:: python
|
||
|
||
import urllib, os
|
||
from tqdm import tqdm
|
||
urllib = getattr(urllib, 'request', urllib)
|
||
|
||
class TqdmUpTo(tqdm):
|
||
"""Provides `update_to(n)` which uses `tqdm.update(delta_n)`."""
|
||
def update_to(self, b=1, bsize=1, tsize=None):
|
||
"""
|
||
b : int, optional
|
||
Number of blocks transferred so far [default: 1].
|
||
bsize : int, optional
|
||
Size of each block (in tqdm units) [default: 1].
|
||
tsize : int, optional
|
||
Total size (in tqdm units). If [default: None] remains unchanged.
|
||
"""
|
||
if tsize is not None:
|
||
self.total = tsize
|
||
return self.update(b * bsize - self.n) # also sets self.n = b * bsize
|
||
|
||
eg_link = "https://caspersci.uk.to/matryoshka.zip"
|
||
with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,
|
||
desc=eg_link.split('/')[-1]) as t: # all optional kwargs
|
||
urllib.urlretrieve(eg_link, filename=os.devnull,
|
||
reporthook=t.update_to, data=None)
|
||
t.total = t.n
|
||
|
||
Inspired by `twine#242 <https://github.com/pypa/twine/pull/242>`__.
|
||
Functional alternative in
|
||
`examples/tqdm_wget.py <https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py>`__.
|
||
|
||
It is recommend to use ``miniters=1`` whenever there is potentially
|
||
large differences in iteration speed (e.g. downloading a file over
|
||
a patchy connection).
|
||
|
||
**Wrapping read/write methods**
|
||
|
||
To measure throughput through a file-like object's ``read`` or ``write``
|
||
methods, use ``CallbackIOWrapper``:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.auto import tqdm
|
||
from tqdm.utils import CallbackIOWrapper
|
||
|
||
with tqdm(total=file_obj.size,
|
||
unit='B', unit_scale=True, unit_divisor=1024) as t:
|
||
fobj = CallbackIOWrapper(t.update, file_obj, "read")
|
||
while True:
|
||
chunk = fobj.read(chunk_size)
|
||
if not chunk:
|
||
break
|
||
t.reset()
|
||
# ... continue to use `t` for something else
|
||
|
||
Alternatively, use the even simpler ``wrapattr`` convenience function,
|
||
which would condense both the ``urllib`` and ``CallbackIOWrapper`` examples
|
||
down to:
|
||
|
||
.. code:: python
|
||
|
||
import urllib, os
|
||
from tqdm import tqdm
|
||
|
||
eg_link = "https://caspersci.uk.to/matryoshka.zip"
|
||
response = getattr(urllib, 'request', urllib).urlopen(eg_link)
|
||
with tqdm.wrapattr(open(os.devnull, "wb"), "write",
|
||
miniters=1, desc=eg_link.split('/')[-1],
|
||
total=getattr(response, 'length', None)) as fout:
|
||
for chunk in response:
|
||
fout.write(chunk)
|
||
|
||
The ``requests`` equivalent is nearly identical:
|
||
|
||
.. code:: python
|
||
|
||
import requests, os
|
||
from tqdm import tqdm
|
||
|
||
eg_link = "https://caspersci.uk.to/matryoshka.zip"
|
||
response = requests.get(eg_link, stream=True)
|
||
with tqdm.wrapattr(open(os.devnull, "wb"), "write",
|
||
miniters=1, desc=eg_link.split('/')[-1],
|
||
total=int(response.headers.get('content-length', 0))) as fout:
|
||
for chunk in response.iter_content(chunk_size=4096):
|
||
fout.write(chunk)
|
||
|
||
**Custom callback**
|
||
|
||
``tqdm`` is known for intelligently skipping unnecessary displays. To make a
|
||
custom callback take advantage of this, simply use the return value of
|
||
``update()``. This is set to ``True`` if a ``display()`` was triggered.
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.auto import tqdm as std_tqdm
|
||
|
||
def external_callback(*args, **kwargs):
|
||
...
|
||
|
||
class TqdmExt(std_tqdm):
|
||
def update(self, n=1):
|
||
displayed = super(TqdmExt, self).update(n)
|
||
if displayed:
|
||
external_callback(**self.format_dict)
|
||
return displayed
|
||
|
||
``asyncio``
|
||
~~~~~~~~~~~
|
||
|
||
Note that ``break`` isn't currently caught by asynchronous iterators.
|
||
This means that ``tqdm`` cannot clean up after itself in this case:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.asyncio import tqdm
|
||
|
||
async for i in tqdm(range(9)):
|
||
if i == 2:
|
||
break
|
||
|
||
Instead, either call ``pbar.close()`` manually or use the context manager syntax:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.asyncio import tqdm
|
||
|
||
with tqdm(range(9)) as pbar:
|
||
async for i in pbar:
|
||
if i == 2:
|
||
break
|
||
|
||
Pandas Integration
|
||
~~~~~~~~~~~~~~~~~~
|
||
|
||
Due to popular demand we've added support for ``pandas`` -- here's an example
|
||
for ``DataFrame.progress_apply`` and ``DataFrameGroupBy.progress_apply``:
|
||
|
||
.. code:: python
|
||
|
||
import pandas as pd
|
||
import numpy as np
|
||
from tqdm import tqdm
|
||
|
||
df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))
|
||
|
||
# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`
|
||
# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)
|
||
tqdm.pandas(desc="my bar!")
|
||
|
||
# Now you can use `progress_apply` instead of `apply`
|
||
# and `progress_map` instead of `map`
|
||
df.progress_apply(lambda x: x**2)
|
||
# can also groupby:
|
||
# df.groupby(0).progress_apply(lambda x: x**2)
|
||
|
||
In case you're interested in how this works (and how to modify it for your
|
||
own callbacks), see the
|
||
`examples <https://github.com/tqdm/tqdm/tree/master/examples>`__
|
||
folder or import the module and run ``help()``.
|
||
|
||
Keras Integration
|
||
~~~~~~~~~~~~~~~~~
|
||
|
||
A ``keras`` callback is also available:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.keras import TqdmCallback
|
||
|
||
...
|
||
|
||
model.fit(..., verbose=0, callbacks=[TqdmCallback()])
|
||
|
||
Dask Integration
|
||
~~~~~~~~~~~~~~~~
|
||
|
||
A ``dask`` callback is also available:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.dask import TqdmCallback
|
||
|
||
with TqdmCallback(desc="compute"):
|
||
...
|
||
arr.compute()
|
||
|
||
# or use callback globally
|
||
cb = TqdmCallback(desc="global")
|
||
cb.register()
|
||
arr.compute()
|
||
|
||
IPython/Jupyter Integration
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
IPython/Jupyter is supported via the ``tqdm.notebook`` submodule:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.notebook import trange, tqdm
|
||
from time import sleep
|
||
|
||
for i in trange(3, desc='1st loop'):
|
||
for j in tqdm(range(100), desc='2nd loop'):
|
||
sleep(0.01)
|
||
|
||
In addition to ``tqdm`` features, the submodule provides a native Jupyter
|
||
widget (compatible with IPython v1-v4 and Jupyter), fully working nested bars
|
||
and colour hints (blue: normal, green: completed, red: error/interrupt,
|
||
light blue: no ETA); as demonstrated below.
|
||
|
||
|Screenshot-Jupyter1|
|
||
|Screenshot-Jupyter2|
|
||
|Screenshot-Jupyter3|
|
||
|
||
The ``notebook`` version supports percentage or pixels for overall width
|
||
(e.g.: ``ncols='100%'`` or ``ncols='480px'``).
|
||
|
||
It is also possible to let ``tqdm`` automatically choose between
|
||
console or notebook versions by using the ``autonotebook`` submodule:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.autonotebook import tqdm
|
||
tqdm.pandas()
|
||
|
||
Note that this will issue a ``TqdmExperimentalWarning`` if run in a notebook
|
||
since it is not meant to be possible to distinguish between ``jupyter notebook``
|
||
and ``jupyter console``. Use ``auto`` instead of ``autonotebook`` to suppress
|
||
this warning.
|
||
|
||
Note that notebooks will display the bar in the cell where it was created.
|
||
This may be a different cell from the one where it is used.
|
||
If this is not desired, either
|
||
|
||
- delay the creation of the bar to the cell where it must be displayed, or
|
||
- create the bar with ``display=False``, and in a later cell call
|
||
``display(bar.container)``:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.notebook import tqdm
|
||
pbar = tqdm(..., display=False)
|
||
|
||
.. code:: python
|
||
|
||
# different cell
|
||
display(pbar.container)
|
||
|
||
The ``keras`` callback has a ``display()`` method which can be used likewise:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.keras import TqdmCallback
|
||
cbk = TqdmCallback(display=False)
|
||
|
||
.. code:: python
|
||
|
||
# different cell
|
||
cbk.display()
|
||
model.fit(..., verbose=0, callbacks=[cbk])
|
||
|
||
Another possibility is to have a single bar (near the top of the notebook)
|
||
which is constantly re-used (using ``reset()`` rather than ``close()``).
|
||
For this reason, the notebook version (unlike the CLI version) does not
|
||
automatically call ``close()`` upon ``Exception``.
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.notebook import tqdm
|
||
pbar = tqdm()
|
||
|
||
.. code:: python
|
||
|
||
# different cell
|
||
iterable = range(100)
|
||
pbar.reset(total=len(iterable)) # initialise with new `total`
|
||
for i in iterable:
|
||
pbar.update()
|
||
pbar.refresh() # force print final status but don't `close()`
|
||
|
||
Custom Integration
|
||
~~~~~~~~~~~~~~~~~~
|
||
|
||
To change the default arguments (such as making ``dynamic_ncols=True``),
|
||
simply use built-in Python magic:
|
||
|
||
.. code:: python
|
||
|
||
from functools import partial
|
||
from tqdm import tqdm as std_tqdm
|
||
tqdm = partial(std_tqdm, dynamic_ncols=True)
|
||
|
||
For further customisation,
|
||
``tqdm`` may be inherited from to create custom callbacks (as with the
|
||
``TqdmUpTo`` example `above <#hooks-and-callbacks>`__) or for custom frontends
|
||
(e.g. GUIs such as notebook or plotting packages). In the latter case:
|
||
|
||
1. ``def __init__()`` to call ``super().__init__(..., gui=True)`` to disable
|
||
terminal ``status_printer`` creation.
|
||
2. Redefine: ``close()``, ``clear()``, ``display()``.
|
||
|
||
Consider overloading ``display()`` to use e.g.
|
||
``self.frontend(**self.format_dict)`` instead of ``self.sp(repr(self))``.
|
||
|
||
Some submodule examples of inheritance:
|
||
|
||
- `tqdm/notebook.py <https://github.com/tqdm/tqdm/blob/master/tqdm/notebook.py>`__
|
||
- `tqdm/gui.py <https://github.com/tqdm/tqdm/blob/master/tqdm/gui.py>`__
|
||
- `tqdm/tk.py <https://github.com/tqdm/tqdm/blob/master/tqdm/tk.py>`__
|
||
- `tqdm/contrib/slack.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/slack.py>`__
|
||
- `tqdm/contrib/discord.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/discord.py>`__
|
||
- `tqdm/contrib/telegram.py <https://github.com/tqdm/tqdm/blob/master/tqdm/contrib/telegram.py>`__
|
||
|
||
Dynamic Monitor/Meter
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
You can use a ``tqdm`` as a meter which is not monotonically increasing.
|
||
This could be because ``n`` decreases (e.g. a CPU usage monitor) or ``total``
|
||
changes.
|
||
|
||
One example would be recursively searching for files. The ``total`` is the
|
||
number of objects found so far, while ``n`` is the number of those objects which
|
||
are files (rather than folders):
|
||
|
||
.. code:: python
|
||
|
||
from tqdm import tqdm
|
||
import os.path
|
||
|
||
def find_files_recursively(path, show_progress=True):
|
||
files = []
|
||
# total=1 assumes `path` is a file
|
||
t = tqdm(total=1, unit="file", disable=not show_progress)
|
||
if not os.path.exists(path):
|
||
raise IOError("Cannot find:" + path)
|
||
|
||
def append_found_file(f):
|
||
files.append(f)
|
||
t.update()
|
||
|
||
def list_found_dir(path):
|
||
"""returns os.listdir(path) assuming os.path.isdir(path)"""
|
||
listing = os.listdir(path)
|
||
# subtract 1 since a "file" we found was actually this directory
|
||
t.total += len(listing) - 1
|
||
# fancy way to give info without forcing a refresh
|
||
t.set_postfix(dir=path[-10:], refresh=False)
|
||
t.update(0) # may trigger a refresh
|
||
return listing
|
||
|
||
def recursively_search(path):
|
||
if os.path.isdir(path):
|
||
for f in list_found_dir(path):
|
||
recursively_search(os.path.join(path, f))
|
||
else:
|
||
append_found_file(path)
|
||
|
||
recursively_search(path)
|
||
t.set_postfix(dir=path)
|
||
t.close()
|
||
return files
|
||
|
||
Using ``update(0)`` is a handy way to let ``tqdm`` decide when to trigger a
|
||
display refresh to avoid console spamming.
|
||
|
||
Writing messages
|
||
~~~~~~~~~~~~~~~~
|
||
|
||
This is a work in progress (see
|
||
`#737 <https://github.com/tqdm/tqdm/issues/737>`__).
|
||
|
||
Since ``tqdm`` uses a simple printing mechanism to display progress bars,
|
||
you should not write any message in the terminal using ``print()`` while
|
||
a progressbar is open.
|
||
|
||
To write messages in the terminal without any collision with ``tqdm`` bar
|
||
display, a ``.write()`` method is provided:
|
||
|
||
.. code:: python
|
||
|
||
from tqdm.auto import tqdm, trange
|
||
from time import sleep
|
||
|
||
bar = trange(10)
|
||
for i in bar:
|
||
# Print using tqdm class method .write()
|
||
sleep(0.1)
|
||
if not (i % 3):
|
||
tqdm.write("Done task %i" % i)
|
||
# Can also use bar.write()
|
||
|
||
By default, this will print to standard output ``sys.stdout``. but you can
|
||
specify any file-like object using the ``file`` argument. For example, this
|
||
can be used to redirect the messages writing to a log file or class.
|
||
|
||
Redirecting writing
|
||
~~~~~~~~~~~~~~~~~~~
|
||
|
||
If using a library that can print messages to the console, editing the library
|
||
by replacing ``print()`` with ``tqdm.write()`` may not be desirable.
|
||
In that case, redirecting ``sys.stdout`` to ``tqdm.write()`` is an option.
|
||
|
||
To redirect ``sys.stdout``, create a file-like class that will write
|
||
any input string to ``tqdm.write()``, and supply the arguments
|
||
``file=sys.stdout, dynamic_ncols=True``.
|
||
|
||
A reusable canonical example is given below:
|
||
|
||
.. code:: python
|
||
|
||
from time import sleep
|
||
import contextlib
|
||
import sys
|
||
from tqdm import tqdm
|
||
from tqdm.contrib import DummyTqdmFile
|
||
|
||
|
||
@contextlib.contextmanager
|
||
def std_out_err_redirect_tqdm():
|
||
orig_out_err = sys.stdout, sys.stderr
|
||
try:
|
||
sys.stdout, sys.stderr = map(DummyTqdmFile, orig_out_err)
|
||
yield orig_out_err[0]
|
||
# Relay exceptions
|
||
except Exception as exc:
|
||
raise exc
|
||
# Always restore sys.stdout/err if necessary
|
||
finally:
|
||
sys.stdout, sys.stderr = orig_out_err
|
||
|
||
def some_fun(i):
|
||
print("Fee, fi, fo,".split()[i])
|
||
|
||
# Redirect stdout to tqdm.write() (don't forget the `as save_stdout`)
|
||
with std_out_err_redirect_tqdm() as orig_stdout:
|
||
# tqdm needs the original stdout
|
||
# and dynamic_ncols=True to autodetect console width
|
||
for i in tqdm(range(3), file=orig_stdout, dynamic_ncols=True):
|
||
sleep(.5)
|
||
some_fun(i)
|
||
|
||
# After the `with`, printing is restored
|
||
print("Done!")
|
||
|
||
Redirecting ``logging``
|
||
~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Similar to ``sys.stdout``/``sys.stderr`` as detailed above, console ``logging``
|
||
may also be redirected to ``tqdm.write()``.
|
||
|
||
Warning: if also redirecting ``sys.stdout``/``sys.stderr``, make sure to
|
||
redirect ``logging`` first if needed.
|
||
|
||
Helper methods are available in ``tqdm.contrib.logging``. For example:
|
||
|
||
.. code:: python
|
||
|
||
import logging
|
||
from tqdm import trange
|
||
from tqdm.contrib.logging import logging_redirect_tqdm
|
||
|
||
LOG = logging.getLogger(__name__)
|
||
|
||
if __name__ == '__main__':
|
||
logging.basicConfig(level=logging.INFO)
|
||
with logging_redirect_tqdm():
|
||
for i in trange(9):
|
||
if i == 4:
|
||
LOG.info("console logging redirected to `tqdm.write()`")
|
||
# logging restored
|
||
|
||
Monitoring thread, intervals and miniters
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
``tqdm`` implements a few tricks to increase efficiency and reduce overhead.
|
||
|
||
- Avoid unnecessary frequent bar refreshing: ``mininterval`` defines how long
|
||
to wait between each refresh. ``tqdm`` always gets updated in the background,
|
||
but it will display only every ``mininterval``.
|
||
- Reduce number of calls to check system clock/time.
|
||
- ``mininterval`` is more intuitive to configure than ``miniters``.
|
||
A clever adjustment system ``dynamic_miniters`` will automatically adjust
|
||
``miniters`` to the amount of iterations that fit into time ``mininterval``.
|
||
Essentially, ``tqdm`` will check if it's time to print without actually
|
||
checking time. This behaviour can be still be bypassed by manually setting
|
||
``miniters``.
|
||
|
||
However, consider a case with a combination of fast and slow iterations.
|
||
After a few fast iterations, ``dynamic_miniters`` will set ``miniters`` to a
|
||
large number. When iteration rate subsequently slows, ``miniters`` will
|
||
remain large and thus reduce display update frequency. To address this:
|
||
|
||
- ``maxinterval`` defines the maximum time between display refreshes.
|
||
A concurrent monitoring thread checks for overdue updates and forces one
|
||
where necessary.
|
||
|
||
The monitoring thread should not have a noticeable overhead, and guarantees
|
||
updates at least every 10 seconds by default.
|
||
This value can be directly changed by setting the ``monitor_interval`` of
|
||
any ``tqdm`` instance (i.e. ``t = tqdm.tqdm(...); t.monitor_interval = 2``).
|
||
The monitor thread may be disabled application-wide by setting
|
||
``tqdm.tqdm.monitor_interval = 0`` before instantiation of any ``tqdm`` bar.
|
||
|
||
|
||
Merch
|
||
-----
|
||
|
||
You can buy `tqdm branded merch <https://tqdm.github.io/merch>`__ now!
|
||
|
||
Contributions
|
||
-------------
|
||
|
||
|GitHub-Commits| |GitHub-Issues| |GitHub-PRs| |OpenHub-Status| |GitHub-Contributions| |CII Best Practices|
|
||
|
||
All source code is hosted on `GitHub <https://github.com/tqdm/tqdm>`__.
|
||
Contributions are welcome.
|
||
|
||
See the
|
||
`CONTRIBUTING <https://github.com/tqdm/tqdm/blob/master/CONTRIBUTING.md>`__
|
||
file for more information.
|
||
|
||
Developers who have made significant contributions, ranked by *SLoC*
|
||
(surviving lines of code,
|
||
`git fame <https://github.com/casperdcl/git-fame>`__ ``-wMC --excl '\.(png|gif|jpg)$'``),
|
||
are:
|
||
|
||
==================== ======================================================== ==== ================================
|
||
Name ID SLoC Notes
|
||
==================== ======================================================== ==== ================================
|
||
Casper da Costa-Luis `casperdcl <https://github.com/casperdcl>`__ ~80% primary maintainer |Gift-Casper|
|
||
Stephen Larroque `lrq3000 <https://github.com/lrq3000>`__ ~9% team member
|
||
Martin Zugnoni `martinzugnoni <https://github.com/martinzugnoni>`__ ~3%
|
||
Daniel Ecer `de-code <https://github.com/de-code>`__ ~2%
|
||
Richard Sheridan `richardsheridan <https://github.com/richardsheridan>`__ ~1%
|
||
Guangshuo Chen `chengs <https://github.com/chengs>`__ ~1%
|
||
Helio Machado `0x2b3bfa0 <https://github.com/0x2b3bfa0>`__ ~1%
|
||
Kyle Altendorf `altendky <https://github.com/altendky>`__ <1%
|
||
Noam Yorav-Raphael `noamraph <https://github.com/noamraph>`__ <1% original author
|
||
Matthew Stevens `mjstevens777 <https://github.com/mjstevens777>`__ <1%
|
||
Hadrien Mary `hadim <https://github.com/hadim>`__ <1% team member
|
||
Mikhail Korobov `kmike <https://github.com/kmike>`__ <1% team member
|
||
==================== ======================================================== ==== ================================
|
||
|
||
Ports to Other Languages
|
||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
A list is available on
|
||
`this wiki page <https://github.com/tqdm/tqdm/wiki/tqdm-ports>`__.
|
||
|
||
|
||
LICENCE
|
||
-------
|
||
|
||
Open Source (OSI approved): |LICENCE|
|
||
|
||
Citation information: |DOI|
|
||
|
||
|README-Hits| (Since 19 May 2016)
|
||
|
||
.. |Logo| image:: https://tqdm.github.io/img/logo.gif
|
||
.. |Screenshot| image:: https://tqdm.github.io/img/tqdm.gif
|
||
.. |Video| image:: https://tqdm.github.io/img/video.jpg
|
||
:target: https://tqdm.github.io/video
|
||
.. |Slides| image:: https://tqdm.github.io/img/slides.jpg
|
||
:target: https://tqdm.github.io/PyData2019/slides.html
|
||
.. |Merch| image:: https://tqdm.github.io/img/merch.jpg
|
||
:target: https://tqdm.github.io/merch
|
||
.. |Build-Status| image:: https://img.shields.io/github/actions/workflow/status/tqdm/tqdm/test.yml?branch=master&label=tqdm&logo=GitHub
|
||
:target: https://github.com/tqdm/tqdm/actions/workflows/test.yml
|
||
.. |Coverage-Status| image:: https://img.shields.io/coveralls/github/tqdm/tqdm/master?logo=coveralls
|
||
:target: https://coveralls.io/github/tqdm/tqdm
|
||
.. |Branch-Coverage-Status| image:: https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg
|
||
:target: https://codecov.io/gh/tqdm/tqdm
|
||
.. |Codacy-Grade| image:: https://app.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177
|
||
:target: https://www.codacy.com/gh/tqdm/tqdm/dashboard
|
||
.. |CII Best Practices| image:: https://bestpractices.coreinfrastructure.org/projects/3264/badge
|
||
:target: https://bestpractices.coreinfrastructure.org/projects/3264
|
||
.. |GitHub-Status| image:: https://img.shields.io/github/tag/tqdm/tqdm.svg?maxAge=86400&logo=github&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/releases
|
||
.. |GitHub-Forks| image:: https://img.shields.io/github/forks/tqdm/tqdm.svg?logo=github&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/network
|
||
.. |GitHub-Stars| image:: https://img.shields.io/github/stars/tqdm/tqdm.svg?logo=github&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/stargazers
|
||
.. |GitHub-Commits| image:: https://img.shields.io/github/commit-activity/y/tqdm/tqdm.svg?logo=git&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/graphs/commit-activity
|
||
.. |GitHub-Issues| image:: https://img.shields.io/github/issues-closed/tqdm/tqdm.svg?logo=github&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/issues?q=
|
||
.. |GitHub-PRs| image:: https://img.shields.io/github/issues-pr-closed/tqdm/tqdm.svg?logo=github&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/pulls
|
||
.. |GitHub-Contributions| image:: https://img.shields.io/github/contributors/tqdm/tqdm.svg?logo=github&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/graphs/contributors
|
||
.. |GitHub-Updated| image:: https://img.shields.io/github/last-commit/tqdm/tqdm/master.svg?logo=github&logoColor=white&label=pushed
|
||
:target: https://github.com/tqdm/tqdm/pulse
|
||
.. |Gift-Casper| image:: https://img.shields.io/badge/dynamic/json.svg?color=ff69b4&label=gifts%20received&prefix=%C2%A3&query=%24..sum&url=https%3A%2F%2Fcaspersci.uk.to%2Fgifts.json
|
||
:target: https://cdcl.ml/sponsor
|
||
.. |Versions| image:: https://img.shields.io/pypi/v/tqdm.svg
|
||
:target: https://tqdm.github.io/releases
|
||
.. |PyPI-Downloads| image:: https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white
|
||
:target: https://pepy.tech/project/tqdm
|
||
.. |Py-Versions| image:: https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white
|
||
:target: https://pypi.org/project/tqdm
|
||
.. |Conda-Forge-Status| image:: https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge
|
||
:target: https://anaconda.org/conda-forge/tqdm
|
||
.. |Snapcraft| image:: https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft
|
||
:target: https://snapcraft.io/tqdm
|
||
.. |Docker| image:: https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white
|
||
:target: https://hub.docker.com/r/tqdm/tqdm
|
||
.. |Libraries-Rank| image:: https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white
|
||
:target: https://libraries.io/pypi/tqdm
|
||
.. |Libraries-Dependents| image:: https://img.shields.io/librariesio/dependent-repos/pypi/tqdm.svg?logo=koding&logoColor=white
|
||
:target: https://github.com/tqdm/tqdm/network/dependents
|
||
.. |OpenHub-Status| image:: https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif
|
||
:target: https://www.openhub.net/p/tqdm?ref=Thin+badge
|
||
.. |awesome-python| image:: https://awesome.re/mentioned-badge.svg
|
||
:target: https://github.com/vinta/awesome-python
|
||
.. |LICENCE| image:: https://img.shields.io/pypi/l/tqdm.svg
|
||
:target: https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE
|
||
.. |DOI| image:: https://img.shields.io/badge/DOI-10.5281/zenodo.595120-blue.svg
|
||
:target: https://doi.org/10.5281/zenodo.595120
|
||
.. |binder-demo| image:: https://mybinder.org/badge_logo.svg
|
||
:target: https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb
|
||
.. |Screenshot-Jupyter1| image:: https://tqdm.github.io/img/jupyter-1.gif
|
||
.. |Screenshot-Jupyter2| image:: https://tqdm.github.io/img/jupyter-2.gif
|
||
.. |Screenshot-Jupyter3| image:: https://tqdm.github.io/img/jupyter-3.gif
|
||
.. |README-Hits| image:: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif
|
||
:target: https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://tqdm.github.io/img/favicon.png&f=https://tqdm.github.io/img/logo.gif&style=social
|