Tomaž Jerman
c5c7ca011b
Use module field label for DAL pipeline outputs
2022-09-20 13:11:24 +02:00
Tomaž Jerman
40bce0b2c6
Fix remaining tests
2022-09-20 12:10:37 +02:00
Tomaž Jerman
1d7f82172e
Refactor execution steps to offload more state preparation
...
Extract most validation/initialization/preparation logic from
pipeline execution steps into definition steps.
2022-09-19 16:22:37 +02:00
Denis Arh
3050cda023
Add RDBMS DAL model validation
...
This will prevent misconfigured models with atributes that read from the
same columns. This constraint could/should be removed in the
future.
2022-09-16 12:57:42 +02:00
Tomaž Jerman
077788d9b8
Remove operations from models and connections
...
For now, operations will be defined on a DB driver.
2022-09-15 13:55:21 +02:00
Tomaž Jerman
da24de8b09
Add attribute ident normalization to support legacy names
...
Older versions allowed more variations for the same system ident.
New one does not; normalization is a temporary solution for it.
2022-09-15 12:06:49 +02:00
Tomaž Jerman
9fdb86c4cf
Add date_format to supported QL functions
2022-09-15 12:06:49 +02:00
Denis Arh
d1dd4d1a0c
Support SQLite table & index creation
2022-09-14 12:18:43 +02:00
Denis Arh
44ba447678
Add support for meta data on attribute types (rdbms:type=integer)
2022-09-14 12:18:43 +02:00
Denis Arh
6939d9e226
Model definition fixes (dal, index)
2022-09-14 12:18:43 +02:00
Denis Arh
4e96804d46
Add support for RDBMS index creation
2022-09-14 12:18:43 +02:00
Denis Arh
5992471337
Add support for default values for RDBMS columns
2022-09-14 12:18:43 +02:00
Denis Arh
a6c0d8ecee
Base support for creating RDBMS tables from models
2022-09-14 12:18:43 +02:00
Tomaž Jerman
7d18058f58
Rework the cmpose report endpoint to the DAL pipeline
2022-09-14 10:17:22 +02:00
Tomaž Jerman
ccc1064fac
Expand test suite, cleanup/refactor based on results
2022-09-01 16:55:20 +02:00
Tomaž Jerman
7e0d55dfa1
Move step definition validators to the end
...
This allows the init to correct invalid/missing bits.
2022-09-01 16:55:20 +02:00
Tomaž Jerman
3f15fa2a82
Fix pkg/dal tests
2022-09-01 16:55:20 +02:00
Tomaž Jerman
625d4c2e0f
Update dal internal filter to conform to new interface
2022-09-01 16:55:20 +02:00
Tomaž Jerman
eb57c03e6c
Port the old system reporter to the DAL pipeline
2022-09-01 16:55:20 +02:00
Tomaž Jerman
b378b87313
Make pkg/dal.Row more robust
2022-09-01 16:55:20 +02:00
Tomaž Jerman
5fe71bf3b1
Tweak DAL pipeline preparation & exec flow
...
* Add a dryrun option; split init into init and exec
* move the iterator method from pipeline to service to allow
more context when preprocessing/initializing the pipeline
* add an extra value to link's right rows to simplify identification
2022-09-01 16:55:20 +02:00
Tomaž Jerman
dc70e2aeb8
Export dal.Row for reusability
2022-09-01 16:55:20 +02:00
Tomaž Jerman
377e07cbd9
Move dal attribute types to constants for reusability
2022-09-01 16:55:20 +02:00
Tomaž Jerman
b6cb376d83
Introduce base model constraints and rework record filter constraints
2022-09-01 16:55:20 +02:00
Tomaž Jerman
1ad7323ce7
Define base dal pipeline implementation
2022-09-01 16:55:20 +02:00
Tomaž Jerman
f8595ca827
Implement pipeline link (left) execution step
2022-09-01 16:55:20 +02:00
Tomaž Jerman
bad80e5d81
Implement pipeline join (left) execution step
2022-09-01 16:55:20 +02:00
Tomaž Jerman
5b89af4e01
Implement pipeline aggregate execution step
2022-09-01 16:55:20 +02:00
Tomaž Jerman
f80a0b8107
Add an inmemory buffer for easier testing
2022-09-01 16:55:20 +02:00
Tomaž Jerman
a4c61d63ad
Add base testing boilerplate and generic utilities
2022-09-01 16:55:20 +02:00
Tomaž Jerman
a6abc01927
Outline base pipeline step definition and costing model
...
Pipeline step definitions are used to construct underlaying
execution nodes operate over datasources.
The costing model is the initial approach to provide optimizers
more context on what they should do.
2022-09-01 16:55:20 +02:00
Tomaž Jerman
40dbbae730
Add an internalFilter wrapping struct for filter.Filter
...
The struct simplifies filter manipulation when performing
analysis and optimization.
2022-09-01 16:55:20 +02:00
Tomaž Jerman
61051e998e
Define tester & evaluator interfaces for asserting/computing attrs
2022-09-01 16:55:20 +02:00
Denis Arh
258068bdfb
Improve model-by-ref handling
2022-08-28 12:41:06 +02:00
Denis Arh
fb8765ff39
Improved default-connection handlong on module reload
2022-08-21 16:41:38 +02:00
Denis Arh
818f0ccb9b
Generate models for all resources
...
Some refactoring of CUE files
2022-08-21 15:54:03 +02:00
Denis Arh
554ae45ddf
Various DAL connection improvements & config reorg
2022-08-15 16:57:42 +02:00
Denis Arh
ff69698027
Properly implement model cache management on DAL RDMBS
2022-08-12 16:50:12 +02:00
Denis Arh
345eee72cb
Various DAL codebase improvements and cleanups
2022-08-10 21:38:04 +02:00
Denis Arh
df97fb676b
Improve handling modules with issues
...
Allow adding incomplete (no or invalid connection) to DAL service
to avoid crashes.
2022-08-09 20:29:50 +02:00
Denis Arh
1ac06a374b
Ignore modules with invalid connection ID
2022-08-08 13:54:44 +02:00
Denis Arh
cbfb1d49bd
Refactor model ident validation
2022-08-08 13:25:32 +02:00
Denis Arh
015d8ada6e
Refactor module-model conversion
...
Remove redundant parts of config (eg partititoned flag),
refactored and cleand-up field-to-attribute conversion
2022-08-08 13:19:01 +02:00
Tomaž Jerman
9e1375e99a
Refactor DAL capabilities into operations and properties
2022-08-02 12:57:22 +02:00
Denis Arh
7d91b4eaaf
Improve error message when database is misconfigured
2022-07-28 19:57:28 +02:00
Denis Arh
34c767127f
Fix DAL terminology (remove "record")
2022-07-28 14:16:01 +02:00
Denis Arh
55b4f4ece7
Remove wrapError utility func
2022-07-28 13:02:15 +02:00
Denis Arh
98722f67eb
Rename ModelFilter to ModelRef
2022-07-28 12:58:40 +02:00
Denis Arh
f725624697
Move DAL iterator interface and add IteratorEncodeJSON
2022-07-28 11:04:56 +02:00
Tomaž Jerman
978c538748
Make DAL Purge method not remove the primary store since its invalid
2022-07-27 16:53:04 +02:00
