RINQ – Concept of a Ruby Integrated Query Language

Posted: March 12th, 2008 | Author: Sten Friedrich | Filed under: AOQL, Application Development, C#, Java, Programming Languages, Query Languages, Ruby, SBQL, SQL | Tags: .NET, AOQL, C#, Database, db4o, Domain Specific Language, Hibernate, HQL, Java, LINQ, Native Queries, Object Databases, Query Expressions, Query Language, RINQ, Ruby, SBQL, SODA, SQL | Comments Off

After a lot of research into query languages and persistence of objects I finished my bachelor’s thesis about creating a query language for object-databases in Ruby in January 2008. This post is a short summary of the results. I also wrote a short paper about it. You can find it here. It introduces a Ruby Integrated Query Language for heterogeneous data sources. The concept is based on a research into different query languages regarding general and specific criteria for query language development and the possibilities of dynamic programming languages like Ruby. Native integration in the programming language and the capability to use it with heterogeneous data sources are the main goals. With this concept the impedance mismatch on the application layer can be resolved.

Current situation in application development

The introduction of the object-oriented programming paradigm at the application layer created an impedance mismatch between programming and query languages and an impedance mismatch between the application and persistence layer. The following table gives an overview of the current situation in application development.

Table 1: Current situation in application development

How to create a query language?

Query languages have to meet a lot of requirements to be a complete query language. There are general criteria and criteria for easy learning and acceptance.

General criteria

The following table is based on the criteria list of Saake, Schmitt and Türker. The list is expanded by a few points to meet the latest developments needs in the field of query languages.

Table 2: General Criteria

Specific criteria for easy learning and acceptance

SQL is an approved language for queries. To follow the principles of SQL will guarantee the acceptance of the new language.

Role of the programming language

Table 3: Specific criteria for easy learning and acceptance

The relation to the programming language is an important aspect in creating a query language. It can be natively integrated in the programming language or combined with the data store. Native integration delivers type safety, refactoring and early error detection to the programmer. The combination with the data store delivers data independence of the application layer. For integration the programming language has to achieve some conditions in order to implement e.g. projection and deferred execution. Ruby meets this demands because of its dynamic, flexibility, dynamic typing and so on.

Conclusion

The important impedance mismatch for the application development is the mismatch at the application layer. Integrating the query language natively into the programming language is the main goal to resolve this impedance mismatch. The mismatch to the persistence layer can be resolved with automatic mapping or native data storage in object databases. For this reasons, the LINQ concept will be the inspiration for the Ruby Integrated Query language.

RINQ – Ruby Integrated Query Language

Conception

The new query language in Ruby is called RINQ, like Ruby Integrated Query and is natively integrated in the programming language, has sql-like query operators and supports heterogeneous data sources. RINQ is available for types which include the module Enumerable. Furthermore it provides two interfaces RExpressionTree and ROODB for data source specific implementation of the query operators.

Table 4: Concept of RINQ

RINQ can appear in two kinds, as query operators (methods) or as query expressions (DSL-like). Ruby already contains keywords such as select, which are implemented for enumerable data types. That’s why it is necessary to extend the new RINQ query operator keywords with a „q“, because they should be valid for different types.

Query operators (query methods) – language definition

Example

customernames = customers.
              qwhere {|c| c.address.city == "London"}.
              qselect  {|c| {:firstname => c.firstname,
              :lastname  => c.lastname}}.
              qorderby {|c| c.lastname}

Syntax

RINQ: RESTRICTION Operator

[sourceElem] = source.qwhere{|sourceElem| predicate}

RINQ: PROJECTION Operators

[resultElem] = source.qselect{|sourceElem| selector}
[innerElem]  = source.qselectmany{|sourceElem| selector}
[resultElem] = source.qselectmany{|sourceElem|
                                 [selector,resultSelector]}

RINQ: JOIN Operators

# qgroupjoin analog, equalKeyComparer optional
[resultElem] = source.qjoin(inner,equalKeyComparer)
                      {|sourceElem,innerElem| [keySelector,
                      innerKeySelector,resultSelector]}

RINQ: ORDERING Operators

# qorderbydesc analog, KeyComparer optional
[orderdSourceElem] = source.qorderby(KeyComparer)
                            {|sourceElem| keySelector}
# qthenbydesc analog, KeyComparer optional
[orderdSourceElem] = orderdSource.qthenby(KeyComparer)
                            {|orderdSourceElem| keySelector}
[reverseSourceElem] = source.qreverse

RINQ: GROUPING Operators

# selector & equalKeyComparer optional
{key => [resultElem]} = source.qgroupby(equalKeyComparer)
                        {|sourceElem| [keySelector,selector]}

RINQ: QUANTIFIERS

# equalSourceComparer optional
bool = source.qcontains(other, equalSourceComparer)
bool = source.qany{|sourceElem| predicate}
bool = source.qall{|sourceElem| predicate}

RINQ: AGGREGATE Operators

int = source.qcount
int = source.qcount{|sourceElem| predicate}
# qavg analog
numeric = numericSource.qsum
numeric = source.qsum{|sourceElem| numericSelector}
# qmax analog, sourceComparer & resultComparer optional
sourceElem = source.qmin(sourceComparer)
resultElem = source.qmin(resultComparer)
                        {|sourceElem| Selector}

RINQ: SET Operators

[sourceElem] = source.qdistinct(equalSourceComparer)
# qintersect & qexcept analog, equalSourceComparer optional
[sourceElem] = source.qunion(second,equalSourceComparer)

ELEMENT, PARTITIONING, CONCATENATION, EQUALITY Operators also exist.

Query expressions (DSL) – language definition

To increase the abstraction level of the new query language you can create and implement it as a DSL (Domain specific language). This can be realised in Ruby with Metaprogramming. Rubys open syntax and dynamic concept offer good premises. Peter Vanbroekhoven gives a tutorial about that in his presentation. With Metaprogramming it would also be possible to get rid of the „q“ in the query operator method names. Query expressions have been made as abstract keywords which call the query methods of RINQ.

Example

customerfirstnames = query do
    qfrom :c => customers
    qwhere c.lastname == "Mitchel"
    qselect c.firstname
end

Syntax

query-expression   ::= qfrom-clause query-body
query-body         ::= query-body-clause* final-query-clause
query-body-clause  ::= (qfrom-clause | qwhere-clause)
qfrom-clause       ::= qfrom name => source
qwhere-clause      ::= qwhere predicate
qselect-clause     ::= qselect selector
final-query-clause ::= qselect-clause

Can abstract and language integrated queries be combined?

RINQ represents the trend to integrate the query language into the programming language, but the impedance mismatch between the application and persistence layer still exists. With RINQ you can’t create application independent queries. This is only possible with a data storage integrated query language (maybe with imperative elements like PL/SQL). To realise this approach the Object Database Technology Working Group (ODBTWG) released a White Paper about an Abstract Object Query Language based on a Stack-Based Query Language(SBQL) with a Stack-Based Data Storage Architecture (SBA), which was created by Prof. Kazimierz Subieta, Warsaw.

AOQL – Abstract Object Query Language

AOQL is an abstract query language that is combined with an abstract data store. To connect it with the programming language is an unsolved problem.

Table 5: AOQL – Abstract Object Query Language

AOQL and RINQ together – a vision

The combination of RINQ and AOQL could be a possible solution to resolve the impedance mismatch at the application layer as well as the impedance mismatch between the application and the persistence layer.

Table 6: AOQL and RINQ together – a vision

The query operators of RINQ could be used to create AOQL queries which realise the access to the data store. So AOQL is always the interface to the data store. That means you have two options, queries with RINQ methods from the application or queries with AOQL directly. So application development could be free of impedance mismatches.

References

Don Box, Anders Hejlsberg: .NET Language-Integrated Query. 2007.
http://msdn2.microsoft.com/en-us/library/bb308959.aspx

Michael Card: Next-Generation Object Database Standardization. 2007.
http://odbms.org/experts.html#article15

Gunter Saake, Ingo Schmitt, Can Türker: Objektdatenbanken- Konzepte, Sprachen, Architekturen. International Thomson Publishing. 2003.

Stack-Based Architecture (SBA) and Stack-Based Query Language (SBQL).
http://www.sbql.pl/

Peter Vanbroekhoven: How to create a Domain Specific Language? – Metaprogramming in Ruby. 2007.
http://www.xaop.com/blog/2007/10/07/video-how-to-create-adomain-specific-language-/

---