Updated README

README.md

@@ -1,4 +1,101 @@
-yaql
+YAQL - Yet Another Query Language
 ====
 
-yet another query language
+At the beginning of millennium the growing trend towards data formats standardization and application integrability made
+XML extremely popular. XML became lingua franca of the data. Applications tended to process lots of XML files ranging
+from small config files to very large datasets. As these data often had a complex structure with many levels of
+nestedness it is quickly became obvious that there is a need for specially crafted domain specific languages to query
+these data sets. This  is how XPath and later XQL were born.
+
+With later popularization of REST services and Web 2.0 JSON started to take XML’s place. JSON’s main advantage (besides
+being simpler than XML) is that is closely reassembles data structures found in most programming languages (arrays,
+dictionaries, scalars) making it very convenient for data serialization. As JSON lacked all the brilliant XML-related
+technologies like XSLT, XML Schema, XPath etc. various attempts to develop similar languages for JSON were made. One of
+those efforts was JSONPath library developed in 2007 by Stefan Gössner. Initial implementation was for PHP and
+JavaScript languages, but later on ports to other languages including Python were written.
+
+JSONPath allows navigation and querying, well, JSONs.
+Suppose we have JSON as in following:
+
+    {
+      "customers": [
+      {
+        "customer_id": 1,
+        "name": "John",
+        "orders": [{
+          "order_id": 1,
+          "item": "Guitar",
+         "quantity": 1
+        }]
+      },{
+       "customer_id": 2,
+       "name": "Paul",
+       "orders": [ {
+          "order_id": 2,
+          "item": "Banjo",
+          "quantity": 2
+         },{
+         "order_id": 3,
+         "item": "Piano",
+         "quantity": 1
+        }]
+      }
+     ]
+    }
+
+
+then
+
+`jsonpath(data, "$.customers[0].name") -> [‘John’]`
+`jsonpath(data, "$.customers[*].orders[*].order_id") -> [1, 2, 3]`
+
+But what if we need, for example to find order having ID = 2? Here is how it done in JSONPath:
+
+`jsonpath(data, "$.customers[*].orders[?(@.order_id == 2)") ->  [{'order_id': 2, 'item': 'Banjo', 'quantity': 2}]`
+
+The construct `[?(expression)]` allows to filter items using any Python expression in our case. `@` character is
+replaced with current value and then the whole expression is evaluated. Evaluation of arbitrary Python expression
+requires using `eval()` function unless one wants to develop his own complete parser and interpreter of Python
+programming language. Needless to say that `eval()` is a great security breach. If JSONPath expressions are used to
+simplify program logic it would not be a big deal, but what if JSONPath is written by program users?
+
+JSONPath expression is just a plain string. There is no such concept as parameter. That is if one want to find order
+having ID = some variable value he has to dynamically construct expression string using string formatting or
+concatenation. And again that is might be okay for internal usage but would became difficult for external usage and also
+open the doors for injection attacks (remember SQL injection?)
+
+Another limitation of JSONPath is JSON itself. Technically speaking JSONPath operates not on the JSON itself (i.e. text
+representation) but on a JSON-like object model that is mixture of arrays, dictionaries and scalar values. But what is
+one want to query object model consisting of custom objects? What if some parts of this model are dynamically computed?
+Or the model is a graph rather than a tree?
+
+It seems like JSONPath is good enough to use in Python code when you can `eval()` things and have many helper function
+to work with data besides JSONPath capabilities but is not enough for external use when you need to have sufficient
+power to query model without manual coding and have it still secure.
+This is why we designed YAQL. YAQL follows the JSONPath ideas and has very similar syntax but offers much more for data
+querying.
+
+Expressions are quite similar to JSONPath. Here is how examples above can be translated to YAQL:
+
+`$.customers[0].name -> $.customers[0].name (no change)`
+`$.customers[*].orders[*].order_id -> $.customers.orders.order_id`
+
+the main addition to JSONPath is functions and operators. Consider the following YAQL expressions:
+
+`$.customers.orders[$.quantity > 0].quantity.sum() -> 4`
+`$.customers.orders.select($.quantity * $.quantity).sum() -> 6`
+`$.customers.orders.order_id.orderDesc($) -> [3, 2, 1]`
+`$.customers.orders.order_id.orderDesc($).take(2) -> [3, 2]`
+`$.customers.orders.order_id.orderDesc($).first() -> 3`
+
+Does it mean that YAQL has large built-in function and operator library?. Yes, YAQL library has a out of the box large
+set of commonly used functions. But they are not built-in. All the functions and operators (which are also function:
+`a + b = operator_+(a, b)` etc) are user-supplied. User is free to add other functions that could be used in expressions
+and to remove standard ones.
+
+JSONPath library needs 2 arguments - input JSON data and an a expression. YAQL library requires third
+parameter - context.
+
+Context is a repository of functions and variables that can be used in expressions. So all the functions above are just
+ordinary Python functions that are registered in Context object. But because they all need to be registered in Context
+user can always customize them, add his own model-specific ones and have full control over the expression evaluation.