DeforaNetworks/corteza
3
0
Fork 0
Code Activity
3,942 Commits 2 Branches 228 Tags
ef72ffa3739bcd7693c64d8652e453bb765d73c6
Commit Graph

91 Commits

Author SHA1 Message Date
Vivek Patel
ef72ffa373 Fix total and page navigation for records 
Introduce helper func(IteratorPaging) to count total and create page navigation for records and also fixes timezone and milliseconds fir dateTime field value for query.
 2022-09-30 18:29:38 +05:30
Tomaž Jerman
7eaab982b7 Fix lapsus with join row merge mapping 2022-09-30 13:41:07 +02:00
Tomaž Jerman
d18c8bd80d Quick fix for invalid aggregate attribute type determination 
The logic should change along with how pipeline description
is done and how the pipeline represents attributes.
 2022-09-29 22:21:46 +02:00
Tomaž Jerman
8d480e67c0 Allow rdbms model ops to use already parsed filter expressions 2022-09-29 16:34:00 +02:00
Tomaž Jerman
75a3394d15 Make DAL QL functions consistent and introduce DAY 
* date outputs the date without the time part
* day outputs the day of the month
 2022-09-29 15:40:36 +02:00
Tomaž Jerman
c587db760e Prefix DAL pipeline join attributes with source identifier 
This helps avoid name collisions in case the identifiers weren't
provided manually.
 2022-09-29 13:43:54 +02:00
Tomaž Jerman
5fa45e0cf7 Make ql parser init more flexible 2022-09-29 13:43:54 +02:00
Tomaž Jerman
92c2a0d12e Add system field translations for report data frames 2022-09-28 11:05:51 +02:00
Tomaž Jerman
64ad7e0535 Reword link predicate check error messages 2022-09-26 14:24:49 +02:00
Tomaž Jerman
53f3297579 Add tests for multi-value fields for DAL join step 2022-09-23 16:13:14 +02:00
Tomaž Jerman
7e2fec6da1 Improve aggregate's multi-value support to match the legacy version 2022-09-23 13:58:44 +02:00
Tomaž Jerman
95d4eba4eb Fix limit condition lapsus in DAL pipeline join 2022-09-22 12:32:12 +02:00
Tomaž Jerman
956547a893 Revert "Skip broken tests for dev.3 release" 
This reverts commit 340873c82e.
 2022-09-22 12:28:17 +02:00
Tomaž Jerman
d7ecbafa66 Fixed improper limit check for aggregate and join 2022-09-22 12:25:46 +02:00
Vivek Patel
340873c82e Skip broken tests for dev.3 release 2022-09-21 19:34:05 +05:30
Tomaž Jerman
b3cdbd060f Add paging tests for pipeline aggregate and join 2022-09-21 13:24:10 +02:00
Tomaž Jerman
408bfbcc7d Fix aggregator incorrectly determining if we're copying values 
In case the expression uses simple attr. reference, there is no
need to initialize expr. runners.
The old logic didn't do this correctly.
 2022-09-20 15:45:01 +02:00
Tomaž Jerman
c5c7ca011b Use module field label for DAL pipeline outputs 2022-09-20 13:11:24 +02:00
Tomaž Jerman
40bce0b2c6 Fix remaining tests 2022-09-20 12:10:37 +02:00
Tomaž Jerman
1d7f82172e Refactor execution steps to offload more state preparation 
Extract most validation/initialization/preparation logic from
pipeline execution steps into definition steps.
 2022-09-19 16:22:37 +02:00
Denis Arh
3050cda023 Add RDBMS DAL model validation 
This will prevent misconfigured models with atributes that read from the
same columns. This constraint could/should be removed in the
future.
 2022-09-16 12:57:42 +02:00
Tomaž Jerman
077788d9b8 Remove operations from models and connections 
For now, operations will be defined on a DB driver.
 2022-09-15 13:55:21 +02:00
Tomaž Jerman
da24de8b09 Add attribute ident normalization to support legacy names 
Older versions allowed more variations for the same system ident.
New one does not; normalization is a temporary solution for it.
 2022-09-15 12:06:49 +02:00
Tomaž Jerman
9fdb86c4cf Add date_format to supported QL functions 2022-09-15 12:06:49 +02:00
Denis Arh
d1dd4d1a0c Support SQLite table & index creation 2022-09-14 12:18:43 +02:00
Denis Arh
44ba447678 Add support for meta data on attribute types (rdbms:type=integer) 2022-09-14 12:18:43 +02:00
Denis Arh
6939d9e226 Model definition fixes (dal, index) 2022-09-14 12:18:43 +02:00
Denis Arh
4e96804d46 Add support for RDBMS index creation 2022-09-14 12:18:43 +02:00
Denis Arh
5992471337 Add support for default values for RDBMS columns 2022-09-14 12:18:43 +02:00
Denis Arh
a6c0d8ecee Base support for creating RDBMS tables from models 2022-09-14 12:18:43 +02:00
Tomaž Jerman
7d18058f58 Rework the cmpose report endpoint to the DAL pipeline 2022-09-14 10:17:22 +02:00
Tomaž Jerman
ccc1064fac Expand test suite, cleanup/refactor based on results 2022-09-01 16:55:20 +02:00
Tomaž Jerman
7e0d55dfa1 Move step definition validators to the end 
This allows the init to correct invalid/missing bits.
 2022-09-01 16:55:20 +02:00
Tomaž Jerman
3f15fa2a82 Fix pkg/dal tests 2022-09-01 16:55:20 +02:00
Tomaž Jerman
625d4c2e0f Update dal internal filter to conform to new interface 2022-09-01 16:55:20 +02:00
Tomaž Jerman
eb57c03e6c Port the old system reporter to the DAL pipeline 2022-09-01 16:55:20 +02:00
Tomaž Jerman
b378b87313 Make pkg/dal.Row more robust 2022-09-01 16:55:20 +02:00
Tomaž Jerman
5fe71bf3b1 Tweak DAL pipeline preparation & exec flow 
* Add a dryrun option; split init into init and exec
* move the iterator method from pipeline to service to allow
  more context when preprocessing/initializing the pipeline
* add an extra value to link's right rows to simplify identification
 2022-09-01 16:55:20 +02:00
Tomaž Jerman
dc70e2aeb8 Export dal.Row for reusability 2022-09-01 16:55:20 +02:00
Tomaž Jerman
377e07cbd9 Move dal attribute types to constants for reusability 2022-09-01 16:55:20 +02:00
Tomaž Jerman
b6cb376d83 Introduce base model constraints and rework record filter constraints 2022-09-01 16:55:20 +02:00
Tomaž Jerman
1ad7323ce7 Define base dal pipeline implementation 2022-09-01 16:55:20 +02:00
Tomaž Jerman
f8595ca827 Implement pipeline link (left) execution step 2022-09-01 16:55:20 +02:00
Tomaž Jerman
bad80e5d81 Implement pipeline join (left) execution step 2022-09-01 16:55:20 +02:00
Tomaž Jerman
5b89af4e01 Implement pipeline aggregate execution step 2022-09-01 16:55:20 +02:00
Tomaž Jerman
f80a0b8107 Add an inmemory buffer for easier testing 2022-09-01 16:55:20 +02:00
Tomaž Jerman
a4c61d63ad Add base testing boilerplate and generic utilities 2022-09-01 16:55:20 +02:00
Tomaž Jerman
a6abc01927 Outline base pipeline step definition and costing model 
Pipeline step definitions are used to construct underlaying
execution nodes operate over datasources.

The costing model is the initial approach to provide optimizers
more context on what they should do.
 2022-09-01 16:55:20 +02:00
Tomaž Jerman
40dbbae730 Add an internalFilter wrapping struct for filter.Filter 
The struct simplifies filter manipulation when performing
analysis and optimization.
 2022-09-01 16:55:20 +02:00
Tomaž Jerman
61051e998e Define tester & evaluator interfaces for asserting/computing attrs 2022-09-01 16:55:20 +02:00