Commit Graph

207 Commits (master)

Author SHA1 Message Date
Stephen Finucane c3bda9eeb1 Switch to*
The abstract base classes previously defined in 'collections' were moved
to '' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:

  $ ag -l 'collections.($TYPES)' | \
      xargs sed -i 's/\(collections\)\.\($TYPES\)/\\2/g'

Where $TYPES is the list of moved ABCs from [1].


Signed-off-by: Stephen Finucane <>
Change-Id: Ib07d778a01275d7c985e059156e95abc112e81c8
2021-02-01 11:11:27 +00:00
Stephen Finucane 4670f0949c Remove six
Signed-off-by: Stephen Finucane <>
Change-Id: I2f37e055838ea50627562d3585d6951f8d8d46aa
2021-02-01 11:11:17 +00:00
Dirk Mueller 0b006f0722 Replace assertItemsEqual with assertCountEqual
assertItemsEqual was removed from Python's unittest.TestCase in
Python 3.3 [1][2]. We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.

[1] -
[2] -
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277

Change-Id: I1591e8603262d377d8e0801af152928787374e79
2020-06-11 11:55:50 +02:00
Hervé Beraud 356c9651b0 Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]


Change-Id: I042e2fd85b2255cd3be509504b8f992ad02783fa
2020-06-02 21:07:55 +02:00
Andreas Jaeger 1f35cce0e7 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Update local hacking checks for new flake8.

Change-Id: Idffc923ac339978714001be22e3c9ef285e907f5
2020-03-30 12:00:00 +02:00
Zuul 5a727ea9de Merge "Use __slots__ for more yaqltypes" 2018-12-25 08:41:24 +00:00
Zane Bitter 74cb81b280 Make changes to groupBy() backward-compatible
The behaviour of the groupBy() function was fixed in 1.1.2 by
3fb9178401 to pass only the list of values
to be aggregated to the aggregator function, instead of passing both the
key and the list of values (and expecting both the key and the
aggregated value to be returned). This fix was incompatible with
existing expressions that used the aggregator argument to groupBy().

In the event of an error, fall back trying the previous syntax and see
if something plausible gets returned. If not, re-raise the original error.
This should mean that pre-existing expressions will continue to work,
while most outright bogus expressions should fail in a manner consistent
with the correct definition of the function.

Change-Id: Ic6c54be4ed99003fe56cf1a5329f3f1d84fd43c8
Closes-Bug: #1750032
2018-03-05 17:16:40 -05:00
Zane Bitter a3a83aba0d Use __slots__ for more yaqltypes
The change e693e6ecef added slots to the
concrete classes in yaqltypes, however the abstract parent classes
HiddenParameterType and LazyParameterType were omitted, with the result
that their derived types would still contain an __dict__ attribute.

Change-Id: I8e7335ced58b06cfd0de81ceb721dc2f8396f85f
2018-02-19 17:40:13 -05:00
Xavier Hardy fafc9a17be Make the CLI YAQL engine entirely configurable
Change-Id: I982ec52f8f8046d1fb4e1a430a02b9e3631d3a8b
2017-06-27 08:47:24 +02:00
Jenkins 2d08340e69 Merge "Added engine options to disable input/output data conversion" 2017-02-22 17:58:19 +00:00
Stan Lagun 38aa2c946d Improvements to yaql CLI
* now it is possible to pass one or more YAQL expressions in command line.
  This also disables interactive mode
* input file name can be specified as "-" to read from stdin
* added option to treat input as a string rather than JSON
* added option to treat input as an array (strings or JSONs)
* added option to output result in Python format rather than JSON
* removed redundant code that left from yaql v0.2
* print error messages to stderr rather than stdout

Change-Id: Ieb1037bc2ba59c2d2360edc4ac9b414e62a0cc79
2017-02-20 21:11:15 -05:00
Stan Lagun 67d58bc17f Added engine options to disable input/output data conversion
'yaql.convertInputData' and 'yaql.convertOutputData' engine
options were added.  By setting them to false one can suppress
input or output data conversion. For the input data this will prevent
yaql from converting mutable data structures (lists, dicts, sets) to
their immutable versions, which will break some of the constructs that
require hashable structures (for example set of lists, or list as a
dictionary key), for the output it will not expand produced iterators
and not convert tuples to lists. However this can greatly improve
performance in some cases

Change-Id: I240ce6646fe7dbc9522624739600b6c364bb9618
2017-02-20 19:13:32 -05:00
Georgy Kibardin e7a7f60631 Pass re.VERBOSE explicitly
ply doesn't add re.VERBOSE flag by default anymore, so we need to pass it

Change-Id: I42b0830b7f30806ddbe42f57f54cc050a262e2e2
2017-02-03 08:19:16 +03:00
Dougal Matthews 59aa918b49 Use six.moves.input for Python 3 compatability
Currently, without this, the YAQL repl will error under Python 3.

Change-Id: I5c53f83a0cb069127e33ab629255b0178f272e62
2016-12-07 11:32:57 +00:00
Jenkins c9206c9c54 Merge "Use __slots__ for yaqltypes" 2016-12-01 01:22:19 +00:00
Jenkins b4a4ffe814 Merge "Use __slots__ for Parameter/Function definition" 2016-12-01 01:21:25 +00:00
Stan Lagun 81ad11b60e Merge of documenter and sphinx extension
- documenter script functionality was merged
   into sphinx extension
- heavy refactoring of the documenter code
- grouping of overload methods in documentation
- several minor fixes in doc strings

Change-Id: I9bccd6b1ff1750d966d8c39558d204fcaa4ad185
2016-11-10 00:35:49 -08:00
Stan Lagun fa798e1e15 Auto building of std library docs
- Sphinx extension to generate YAQL autodoc was written
- Added option not to generate package header for the
   YAQL doc-strings -> RST generator

Change-Id: I63020ea1fc2cb10f18d7d6bbc6ad33a62b846f6c
2016-11-07 21:42:16 -08:00
Zane Bitter e693e6ecef Use __slots__ for yaqltypes
Collectively, these types are fairly common in the context (about 400 in
the default context). Defining __slots__ for them so that they don't
need a __dict__ object to store attributes results in a fairly
inconsequential saving of memory with the default context, but likely
could result in significant savings depending on the amount of data yaql
is operating on.

Change-Id: Iaae6b48672a3293f3920125347835a100883ea76
2016-11-03 16:19:40 -04:00
Zane Bitter 42fed1a86d Use __slots__ for Parameter/Function definition
These are the two most common types of object in a yaql context (about
800 exist in the default context).  Defining __slots__ for them so that
they don't need a __dict__ object to store attributes results in a
modest saving of memory (around 10% of the default context).

Change-Id: I5c7027bcd48a1f2282a369c1469107d2ab9c5083
2016-11-03 16:18:56 -04:00
Jenkins c9d1f7bf75 Merge "[docs] Docstrings for queries YAQL functions" 2016-10-13 14:29:41 +00:00
Jenkins cfa457797b Merge "[docs] Docstrings for legacy YAQL functions" 2016-10-13 14:20:46 +00:00
Jenkins 5072b14c14 Merge "[docs] Docstrings for collections YAQL functions" 2016-10-13 14:19:03 +00:00
Omar Shykhkerimov 24b8e2bdb8 [docs] Docstrings for queries YAQL functions
Documented functions:
* join
* repeat
* cycle
* takeWhile
* skipWhile
* indexOf
* lastIndexOf
* indexWhere
* lastIndexWhere
* slice
* splitWhere
* sliceWhere
* splitAt
* aggregate
* reverse
* mergeWith
* isIterable
* accumulate
* generate
* generateMany
* defaultIfEmpty

Change-Id: Id236fa9e9e86992a42aba65b2f8e123d207e3592
2016-10-11 14:54:49 +03:00
Omar Shykhkerimov b2cd595d2f Create child context in regex/search function
Change-Id: I1197e7a345109c5fb29e1bfc48df8f051b48bac9
Closes-bug: #1630682
2016-10-09 22:17:24 +03:00
Jenkins 43f28180e7 Merge "Add help info to CLI options" 2016-10-07 14:15:06 +00:00
Jenkins cc25089114 Merge "[docs] Docstrings for yaqlized YAQL functions" 2016-10-07 14:14:53 +00:00
Jenkins bd4de1dce6 Merge "[docs] Docstrings for system YAQL functions" 2016-10-07 14:14:48 +00:00
Jenkins 0bf109af3e Merge "[docs] Docstrings for regex YAQL functions" 2016-10-07 14:14:41 +00:00
Omar Shykhkerimov 6b129fdf33 [docs] Docstrings for collections YAQL functions
Change-Id: I33202ddc502544fbbc54851201d29b00e14ade2e
2016-10-06 20:00:40 +03:00
Omar Shykhkerimov 245102b015 [docs] Docstrings for legacy YAQL functions
Change-Id: I7ca1905852f9084c19bdebc25f271a12815db6a9
2016-10-06 16:58:53 +00:00
Jenkins 3c16c1870e Merge "Add a test proving escaped back references work" 2016-10-05 21:02:58 +00:00
Omar Shykhkerimov fad65b35aa [docs] Docstrings for regex YAQL functions
Change-Id: I9ee189f0c53108e30d0d47f7bfc2aa1350527709
2016-10-05 20:20:09 +03:00
Omar Shykhkerimov 855b6839db [docs] Docstrings for system YAQL functions
Change-Id: Idb9036f2a5eed1a9c4960e84bfc92e4cb0573097
2016-10-05 13:45:04 +00:00
Omar Shykhkerimov 9a344902df [docs] Docstrings for yaqlized YAQL functions
Change-Id: I40857f5f4548866eb4c94cf056f7605e6f508e67
2016-10-05 16:34:59 +03:00
Jenkins a721de9b4a Merge "[docs] Docstrings for common YAQL functions" 2016-10-05 12:13:16 +00:00
Omar Shykhkerimov de3472da30 Add help info to CLI options
Change-Id: I268246f12040ca09047b9776dd4233c907743cb0
2016-10-05 14:42:59 +03:00
Omar Shykhkerimov 29b7589b94 [docs] Docstrings for common YAQL functions
Change-Id: I82747775ac194d0b56c7faf3abef46c3d0efe4d9
2016-10-04 01:04:54 +03:00
Omar Shykhkerimov b2ffd4845d [docs] Docstrings for queries YAQL functions
Documented functions:
* where
* select
* operator . (collection_attribution)
* skip
* limit
* append
* distinct
* enumerate
* any
* all
* concat
* len
* count
* memorize
* sum
* max
* min
* first
* single
* last
* selectMany
* range
* sequence
* orderBy
* orderByDescending
* thenBy
* thenByDescending
* groupBy
* zip
* zipLongest

Change-Id: Id8290e90033483c123cd3b3ee38e8bb8db6bc002
2016-10-04 00:02:33 +03:00
Jenkins a7393f446a Merge "groupBy() aggregator was fixed" 2016-09-26 13:08:03 +00:00
Jenkins e7938e4189 Merge "[docs] Docstrings for boolean YAQL functions" 2016-09-22 10:19:50 +00:00
Stan Lagun 3fb9178401 groupBy() aggregator was fixed
groupBy method accepts `aggregator`
parameter which is supposed to be
a function to aggregate value withing each group
but instead it was applied to the whole group list

Change-Id: Ic38bbe7bed7c624aa3e17f5f4ed4708fc216c278
Closes-Bug: #1626234
2016-09-21 19:16:19 +00:00
Steven Hardy e208d21ee3 Add a test proving escaped back references work
I'd like to use the regex replace interface to do a pattern
substitution, and couldn't find any docs or tests illustrating
how to do it when using backreferences, so adding this test
which proves it works and provides an example for other folks
trying to do this.

Change-Id: Ibbda3cac83955ae6ed6a32a2d51e0fb511220d7a
2016-09-03 13:18:51 +03:00
Jenkins f71a030508 Merge "Replace ValueError with StopIteration in single() method" 2016-08-29 19:40:36 +00:00
Omar Shykhkerimov a9a71408c7 [docs] Docstrings for boolean YAQL functions
Change-Id: Ie4ff39c2a29e57934ec85d1974e2ec8d53b95d08
2016-08-29 20:01:28 +03:00
Jenkins b377dce0f2 Merge "[docs] Docstrings for branching YAQL functions" 2016-08-29 12:04:29 +00:00
Jenkins a8493b143b Merge "[docs] Docstrings for strings YAQL functions" 2016-08-29 12:02:29 +00:00
Jenkins 45090927a5 Merge "[docs] Docstrings for math YAQL functions" 2016-08-29 12:02:17 +00:00
Omar Shykhkerimov 4e822b0aaa Replace ValueError with StopIteration in single() method
The thrown error for single() method in case of collection to
have more than one element should be StopIteration unlike ValueError,
for consistency with other code.

Change-Id: Ib55b04c7c0a5b2afcd7db118e0291c9aa4e46858
2016-08-29 11:00:48 +03:00
Omar Shykhkerimov 529c231794 [docs] Docstrings for strings YAQL functions
Change-Id: I99b947cf86a8131b205351c8f60d1437b6907559
2016-08-12 16:05:49 +03:00