123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432 |
- Metadata-Version: 2.1
- Name: ruamel.ordereddict
- Version: 0.4.15
- Summary: a version of dict that keeps keys in insertion resp. sorted order
- Home-page: https://bitbucket.org/ruamel/ordereddict
- Author: Anthon van der Neut
- Author-email: a.van.der.neut@ruamel.eu
- License: MIT license
- Platform: UNKNOWN
- Classifier: Development Status :: 5 - Production/Stable
- Classifier: Intended Audience :: System Administrators
- Classifier: License :: OSI Approved :: MIT License
- Classifier: Operating System :: POSIX :: Linux
- Classifier: Programming Language :: Python
- The ordereddict module in short
- -------------------------------
- This is an implementation of an ordered dictionary with Key Insertion
- Order (KIO: updates of values do not affect the position of the key),
- Key Value Insertion Order (KVIO, an existing key's position is removed
- and put at the back). The standard library module OrderedDict, implemented
- later, implements a subset of ``ordereddict`` functionality.
- Sorted dictionaries are also provided. Currently only with Key Sorted
- Order (KSO, no sorting function can be specified, but you can specify a
- transform to apply on the key before comparison (e.g. string.lower)).
- This package is hosted on BitBucket and installable from PyPI::
- pip install ruamel.ordereddict
- For Windows there are 32 and 64 bit installable wheels available.
- Usage::
- from ruamel.ordereddict import ordereddict
- kio = ordereddict()
- kvio = ordereddict(kvio=True)
- # without relax unordered initalisation is not allowed
- d = ordereddict({'a':1, 'b': 2}, relax=True)
- sd = sorteddict({'a':1, 'b': 2}) # sorteddict is always relaxed
- **please note that starting with 0.4.6 you should not import _ordereddict
- directly**
- This module has been tested under:
- ============= ========================= ==========
- OS compiler Python
- Linux Mint 17 gcc 4.8.4 2.7.13
- Windows Visual Studio 2010 2.7.13-32
- Windows Visual Studio 2010 2.7.13-64
- ============= ========================= ==========
- Older versions of this module has been tested under
- and I expect those to still work:
- ============= ======================== =========
- OS compiler Python
- Windows XP-64 Visual Studio 2010 2.7.10-32
- Windows XP-64 Visual Studio 2010 2.7.10-64
- Windows XP-64 Visual Studio 2008 2.6.9-32
- Windows XP-64 Visual Studio 2008 2.6.9-64
- Linux Mint 17 gcc 4.8.2 2.6.9
- Ubuntu 12.04 gcc 4.7.2 2.7.6
- Ubuntu 12.04 gcc 4.7.2 2.6.8
- Ubuntu 8.04 gcc 4.2.4 2.7.6
- Ubuntu 8.04 gcc 4.2.4 2.5.2
- Windows XP Visual C++ 2008 Express 2.7.6
- Windows 7 64 Windows SDK for Win7 SP1 2.7.6
- Ubuntu 12.04 gcc 4.6.3 2.7.3
- Ubuntu 8.04 gcc 4.2.4 2.6.4
- Ubuntu 8.04 gcc 4.2.4 2.5.2
- Ubuntu 8.10 gcc 4.3.2 2.5.4
- Ubuntu 8.10 gcc 4.3.2 2.4.6
- Ubuntu 7.04 gcc 4.1.2 2.5.1
- Ubuntu 7.04 gcc 4.1.2 2.4.4
- Ubuntu 6.06 gcc 2.5.1
- Windows XP Visual Studio 2003 2.5.1
- Windows XP Visual C++ 2008 Express 2.6.5
- Windows MingGW 4.7.0 2.7.3
- Solaris 10 GCC 4.4.x 2.7.3
- ============= ======================== =========
- Version 0.4.1 was tested and found working on SuSE Linux Enterprise Server
- (GCC 4.1.0 and Intel C/C++ 10.1) by Stuart Stock.
- MingGW and Solaris were tested and reported to work by Wladimir with version
- 0.4.5
- Home
- ----------------------------
- https://bitbucket.org/ruamel/ordereddict is ordereddict's home on the web.
- Clone the repository there if you want to work from the source.
- http://www.xs4all.nl/~anthon/Python/ordereddict used to be
- ordereddict's home on the web.
- There you can still find the links for downloading the older version (0.4.5).
- Installation
- ------------
- .. comment: To install the package you can use::
- pip install ruamel.ordereddict
- You can clone and checkout the sources, and then run::
- python setup.py install
- Bugreporting
- ------------
- If you find any problems, please let me know, but also realise that I
- have a spamfilter that catches over 100 emails a day and yours might
- get in there unnoticed. So if there is no response within a few days
- please try again.
- Functionality
- -------------
- ordereddict has all of the functionality of dict() except that there
- is no keyword based initialisation and that you cannot pass a normal
- dict to the initialisation of the basic ordereddict (however see the
- relax-ed keyword below). sorteddict cannot be initialised from keywords
- either, but can be initialised from normal dict (ie. they are always
- relaxed).
- As you probably would expect .keys(), .values(), .items(),
- .iterkeys(), itervalues(), iteritems() and "for i in some_ordereddict"
- have elements ordered based on the key insertion order (or key value
- insertion order if kvio is specified, or sort order for sorteddict).
- ordered/sorteddicts can be pickled.
- Some methods have been slightly changed:
- - initialisation of ordereddict takes keywords:
- - kvio: if set to True, then move an existing key on update
- - relax: if set to True, the ordereddict is relaxed for its life regarding
- initialisation and/or update from unordered data (read a normal dict).
- - initialisation of sorteddict takes keyword:
- - key: specifies a function to apply on key (e.g. string.lower)
- - .popitem() takes an optional argument (defaulting to -1) indicating which
- key/value pair to return (by default the last one available)
- - .dict()/.values()/.items()/.iterdict()/.itervalues()/.iteritems()
- all take an optional reverse (default False) parameter that gives
- the list reversed order resp. iterates in reverse
- (the non-iterator can also be done relatively efficient with e.g.
- od.dict().reverse() )
- - .update(): takes an optional relax=True which allows one time
- ordereddict update from normal dictionaries regardless of
- initialisation time relax setting.
- In addition to that ordereddict and sorteddict have some extra methods:
- - .index(key) - gives an integer value that is the index of the key
- - .setkeys()/.setvalues()/.setitems(), work like those in the Larosa/Foord
- implementation, although they might throw different exceptions:
- - setvalues' argument must be an itereable that returns the same number of
- items as the length of the ordereddict
- - setitems' argument is free in length, it performs a clear and adds
- the items in order.
- - slice retrieval for all
- and ordereddict only also has:
- - .setkeys(), works like the one in the Larosa/Foord
- implementation. Argument must be an itereable returning a permutation of the
- existing keys ( that implies having the same length as the ordereddict)
- - .reverse() - reverses the keys in place
- - .insert(position, key, value) - this will put a key at a particular position
- so that afterwards .index(key) == position, if the key was already there
- the original position (and value) is lost to the new position. This often
- means moving keys to new positions!
- - slice deletion/assignment:
- - stepped deletion could be optimized a bit (individual items are deleted
- which can require memmoving multiple items)
- - assignment only from OrderedDict (with the same length as the slice). This
- could also be optimised as I first delete, then insert individual items.
- If the assigned items contain keys that are still there after the deletion
- 'phase' then retrieving that slice does not always give the original
- assigned ordereddict (depending on the position of the items
- with those keys in either ordereddict)
- - .rename(oldkey, newkey) renames a key, but keeps the items position and value
- The new OrderedDict in the standard collections module
- ------------------------------------------------------
- With Python 3.1 and backported to 2.7 there is an OrderedDict class
- available in the collections modules. Raymond Hettinger indicated in
- 2009 at EuroPython that he preferred to start from a minimal
- OrderedDict instead of using the Larosa/Foord
- implementation. Unfortunately the available tests (for the
- functionality that the simple collections.OrderedDict supports) were
- not used either resulting in preventable bugs like repr initially not
- working on recursive OrderedDicts.
- ordereddict (and the Larosa/Foord implementation) is essentially
- a superset of collections.OrderedDict, but there are a few
- differences:
- - OrderedDict is by default relax-ed.
- - repr of recursive OrderedDict does not give any indication of the
- value of the recursive key, as it only displays `...`. ordereddict
- displays `ordereddict([...])` as value. Just using the dots like
- OrderedDict does is going to be ambiguous as soon as you have two different
- types A and B and nest A in B in A or B in B in A.
- - some newer build-in functions available in OrderedDict are not
- available in ordereddict ( __reversed__, viewkeys, viewvalues, viewitems).
- All of the differences can be straightened out in small (70 lines of
- Python) OrderedDict wrapper around ordereddict. With this wrapper the
- OrderedDict tests in the standard test_collections.py all pass.
- Testing
- -------
- testordereddict.py in the test subdirectory has been used to test the module.
- You can use::
- python testordereddict
- to run the tests (py.test support has been dropped as newer versions
- of py.test were not compatible).
- There is a somewhat patched copy of the python lib/Test dictionary testing
- routines included as well, it fails on the _update test however
- because the default is not to use a relaxed ordereddict.
- You can run it with::
- cd test/unit
- python test_dict.py
- To Do
- -----
- - implement Value Sorted Order (VSO: specify value=True for normal
- value comparison), or a value rewrite function for VSO ( e.g.
- value=string.lower )
- - implement Item Sorted Order (ISO): compare value then key ( the other way
- around would not make sense with unique keys, but we might have
- non-unique values).
- - implement slice deletion for sorteddict
- - more testing of sorteddict functionality
- - speedtest slices
- - speedtest sorteddict
- - check on the test_update unittest in test_dict.py
- To Consider
- -----------
- - comparing ordereddicts (as per Larosa/Foord)
- - implement the whole (optionally) using pointers in the DictObject Items
- (Faster on insertion/deletion, slower on accessing slices, makes
- implementing algorithms somewhat more difficult), would have to seperate
- code for sorteddict as key position determination would be much slower.
- - supply a pure Python implementation of exactly the functionality in
- ordereddict
- - test on older versions (< 2.4) of Python and make portable (if this can
- be done without too much clutter) or port.
- - test on the Mac
- - optimise searching for an item pointer for sorteddict with binary search
- (for deletion)
- Background information
- ----------------------
- ordereddict is directly derived from Python's own dictobject.c file.
- The extensions and the representation of ordereddicts() are based
- on Larosa/Foord's excellent pure Python OrderedDict() module
- (http://www.voidspace.org.uk/python/odict.html).
- The implemenation adds a vector of pointers to elements to the basic
- dictionary structure and keeps this vector compact (and in order) so
- indexing is fast. The elements do not know about their position (so
- nothing needs to be updated there if that position changes, but then
- finding an item's index is expensive. Insertion/deletion is also relatively
- expensive in that on average half of the vector of pointers needs to
- be memmove-d one position.
- There is also a long value for bit info like kvio, relaxed.
- The sorteddict structure has an additional 3 pointers of which only
- one (sd_key) is currently used (the others are sd_cmp and sd_value).
- Speed
- -----
- Based on some tests with best of 10 iterations of 10000 iterations of various
- functions under Ubuntu 7.10 (see test/timeordereddict.py and test/ta.py)::
- Results in seconds:
- ------------------------------- dict ordereddict Larosa/Ford collections
- OrderedDict OrderedDict
- empty 0.023 0.025 0.023 0.024
- create_empty 0.028 0.031 0.147 0.329
- create_five_entry 0.037 0.042 0.384 0.558
- create_26_entry 0.187 0.203 1.494 1.602
- create_676_entry 5.330 5.574 36.797 34.810
- get_keys_from_26_entry 0.209 0.231 1.501 1.762
- pop_5_items_26_entry 0.219 0.247 1.952 1.864
- pop_26_items_676_entry 7.550 8.127 46.578 41.851
- popitem_last_26_entry 0.203 0.225 1.624 1.734
- popitem_last_676_entry 5.285 5.534 36.912 34.799
- popitem_100_676_entry -------- 5.552 36.577 --------
- walk_26_iteritems -------- 0.494 2.792 2.238
- ------------------------------- dict ordereddict Larosa/Ford collections
- OrderedDict OrderedDict
- empty 0.930 1.000 0.950 0.966
- create_empty 0.909 1.000 4.728 10.594
- create_five_entry 0.892 1.000 9.201 13.374
- create_26_entry 0.923 1.000 7.368 7.901
- create_676_entry 0.956 1.000 6.601 6.245
- get_keys_from_26_entry 0.908 1.000 6.508 7.641
- pop_5_items_26_entry 0.888 1.000 7.916 7.559
- pop_26_items_676_entry 0.929 1.000 5.732 5.150
- popitem_last_26_entry 0.901 1.000 7.222 7.712
- popitem_last_676_entry 0.955 1.000 6.670 6.288
- popitem_100_676_entry -------- 1.000 6.588 --------
- walk_26_iteritems -------- 1.000 5.653 4.532
- Why
- ---
- Because I am orderly ;-O, and because I use dictionaries to
- store key/value information read from some text file quite often.
- Unfortunately comparing those files with diff when written from
- normal dictionaries often obfucates changes because of the reordering
- of lines when key/value pairs are added and then written.
- I have special routine for YAML files that takes lines like::
- - key1: val1
- - key2: val3
- - key3:
- - val3a
- - val3b
- (i.e. a list of key-value pairs) directly to a single ordered dictionary
- and back. (I find it kind of strange to finally have a structured,
- human readeable, format that does not try to preserve the
- order of key-value pairs so that comparing files is difficult with
- 'standard' text tools).
- Older versions
- --------------
- http://www.xs4all.nl/~anthon/Python/ordereddict used to be
- ordereddict's home on the web.
- There you can still find the links for downloading the older version (0.4.5).
- History
- -------
- ``0.4.13``: 2017-0723
- -
- | ``0.4.9 2015-08-10``
- | typos fixed by Gianfranco Costamagna
- |
- | ``0.4.8 2015-05-31``
- | dependent on ruamel.base
- | version number in a single place
- | using py.test under tox
- | generate wheel for 32/64bit py26/py27 on windows
- |
- | ``0.4.6 2014-01-18``
- | Move to ruamel namespace, hosted on bitbucket, MIT License
- | Testing with tox
- |
- | ``0.4.5 2012-06-17``
- | Fix for a bug while inserting last item again beyond last position (reported
- | by Volkan Çetin / volki tolki ( cetinv at gmail.com )
- | Fix for repeated deletion and insertion fail. Found by and solution provided
- | by Darren Dowker (including tests). Also found by Fabio Zadronzy (including
- | a less elegant fix).
- | applied reindent to .py and astyle to .c files
- |
- | ``0.4.3 2009-05-11``
- | Fix for a bug in slicing SortedDicts.
- | Found by, and fix provided by, Migel Anguel (linos.es)
- |
- | ``0.4.2 2009-03-27``
- | Bug found and by Alexandre Andrade and Fabio Zadrozny in
- | doing deepcopy
- |
- | ``0.4.1 2007-11-06``
- | Bug found and fixed by Fabio Zadrozny on resizing dictionaries
- |
- | ``0.4 2007-10-30``
- | added pickling, added relaxed initialisation/update (from unordered dicts)
- | added KVIO (Key Value Insertion Order ie. key moves to back on update)
- | implemented sorteddict, with KSO, Key Sorted Order. You can specify
- | a function for key transformation before comparison (such as string.lower)
- | sorteddict does not have all of the ordereddict methods as not all make
- | sense (eg. slice assignment, rename, setkeys)
- |
- | ``0.3 2007-10-24``
- | added setkeys/setvalues/setitems; slice retrieval, deletion, assignment
- | .rename(oldkey, newkey) rename a key keeping same value and position
- | .index() of non-existing key now returns ValueError instead of SystemError
- | Changed the module name to _ordereddict (from ordereddict), as Jason
- | Kirstland probably rightfully suggested that any private implementation
- | likely has the (file)name ordereddict.py. A modulename with leading
- | underscore seams more common for extension modules anyway.
- |
- | ``0.2a 2007-10-16``
- | Solved the potential GC problem on Windows
- |
- | ``0.2 2007-10-16``
- | First release, with some tests, and possible still a GC problem
- | with Windows.
- |
- | ``0.1 2007-10-..``
- | This version was never released. While testing it I was far in writing
- | an email to comp.lang.python about why timing with timeit did seem to
- | be memory hungry ....
- | and then I realised ordereddict had a memory leak %-)
|