3
0
Commit Graph

1091 Commits

Author SHA1 Message Date
Tomaž Jerman
8ac3b50077 WIP/POC pkg for common gval handlers 2022-09-01 16:55:21 +02:00
Tomaž Jerman
319a29fdc6 Cleanup system/reporter - DAL/pipeline migration 2022-09-01 16:55:21 +02:00
Tomaž Jerman
4b65286df1 Add refs to DAL models constructed from compose modules 2022-09-01 16:55:20 +02:00
Tomaž Jerman
ccc1064fac Expand test suite, cleanup/refactor based on results 2022-09-01 16:55:20 +02:00
Tomaž Jerman
7e0d55dfa1 Move step definition validators to the end
This allows the init to correct invalid/missing bits.
2022-09-01 16:55:20 +02:00
Tomaž Jerman
3f15fa2a82 Fix pkg/dal tests 2022-09-01 16:55:20 +02:00
Tomaž Jerman
625d4c2e0f Update dal internal filter to conform to new interface 2022-09-01 16:55:20 +02:00
Tomaž Jerman
eb57c03e6c Port the old system reporter to the DAL pipeline 2022-09-01 16:55:20 +02:00
Tomaž Jerman
b378b87313 Make pkg/dal.Row more robust 2022-09-01 16:55:20 +02:00
Tomaž Jerman
5fe71bf3b1 Tweak DAL pipeline preparation & exec flow
* Add a dryrun option; split init into init and exec
* move the iterator method from pipeline to service to allow
  more context when preprocessing/initializing the pipeline
* add an extra value to link's right rows to simplify identification
2022-09-01 16:55:20 +02:00
Tomaž Jerman
dc70e2aeb8 Export dal.Row for reusability 2022-09-01 16:55:20 +02:00
Tomaž Jerman
377e07cbd9 Move dal attribute types to constants for reusability 2022-09-01 16:55:20 +02:00
Tomaž Jerman
9aba555179 Move pkg/report types to sys/types; drop pkg/report package 2022-09-01 16:55:20 +02:00
Tomaž Jerman
787d51e136 Move ASTNode merging methods to QL package 2022-09-01 16:55:20 +02:00
Tomaž Jerman
b6cb376d83 Introduce base model constraints and rework record filter constraints 2022-09-01 16:55:20 +02:00
Tomaž Jerman
1ad7323ce7 Define base dal pipeline implementation 2022-09-01 16:55:20 +02:00
Tomaž Jerman
f8595ca827 Implement pipeline link (left) execution step 2022-09-01 16:55:20 +02:00
Tomaž Jerman
bad80e5d81 Implement pipeline join (left) execution step 2022-09-01 16:55:20 +02:00
Tomaž Jerman
5b89af4e01 Implement pipeline aggregate execution step 2022-09-01 16:55:20 +02:00
Tomaž Jerman
f80a0b8107 Add an inmemory buffer for easier testing 2022-09-01 16:55:20 +02:00
Tomaž Jerman
a4c61d63ad Add base testing boilerplate and generic utilities 2022-09-01 16:55:20 +02:00
Tomaž Jerman
a6abc01927 Outline base pipeline step definition and costing model
Pipeline step definitions are used to construct underlaying
execution nodes operate over datasources.

The costing model is the initial approach to provide optimizers
more context on what they should do.
2022-09-01 16:55:20 +02:00
Tomaž Jerman
40dbbae730 Add an internalFilter wrapping struct for filter.Filter
The struct simplifies filter manipulation when performing
analysis and optimization.
2022-09-01 16:55:20 +02:00
Tomaž Jerman
61051e998e Define tester & evaluator interfaces for asserting/computing attrs 2022-09-01 16:55:20 +02:00
Tomaž Jerman
985c063d89 Expand pkg/filter for easier paging cursor construction/use
* Add a cursor method to return an ql.ASTNode representation.
* Add a package function to construct a cursor from an arbitrary
  value input.
2022-09-01 16:55:20 +02:00
Tomaž Jerman
689be25a89 Cover ASTNode.String() nil ptr edgecase 2022-09-01 16:55:20 +02:00
Denis Arh
5d45846fdd Prevent panic in locale/LocalizedList fn 2022-08-29 07:39:25 +02:00
Denis Arh
258068bdfb Improve model-by-ref handling 2022-08-28 12:41:06 +02:00
Denis Arh
4bb8270189 Fix DAL implementation with Envoy 2022-08-23 08:12:38 +02:00
Tomaž Jerman
6799d3f8de Fix global value store URL construction with custom paths 2022-08-22 15:59:49 +02:00
Peter Grlica
91252ac93d Fixed apigw payload processer invalid function registration 2022-08-22 14:41:59 +02:00
Denis Arh
fb8765ff39 Improved default-connection handlong on module reload 2022-08-21 16:41:38 +02:00
Denis Arh
818f0ccb9b Generate models for all resources
Some refactoring of CUE files
2022-08-21 15:54:03 +02:00
Denis Arh
554ae45ddf Various DAL connection improvements & config reorg 2022-08-15 16:57:42 +02:00
Denis Arh
ff69698027 Properly implement model cache management on DAL RDMBS 2022-08-12 16:50:12 +02:00
Denis Arh
4f0d0ad6c1 Change compose record labels to meta 2022-08-11 19:33:07 +02:00
Denis Arh
345eee72cb Various DAL codebase improvements and cleanups 2022-08-10 21:38:04 +02:00
Denis Arh
df97fb676b Improve handling modules with issues
Allow adding incomplete (no or invalid connection) to DAL service
to avoid crashes.
2022-08-09 20:29:50 +02:00
Denis Arh
1ac06a374b Ignore modules with invalid connection ID 2022-08-08 13:54:44 +02:00
Denis Arh
cbfb1d49bd Refactor model ident validation 2022-08-08 13:25:32 +02:00
Denis Arh
015d8ada6e Refactor module-model conversion
Remove redundant parts of config (eg partititoned flag),
refactored and cleand-up field-to-attribute conversion
2022-08-08 13:19:01 +02:00
Denis Arh
8ec05a032c Optimize hostname detection
When corteza is running inside docker container
HOSTNAME env var is set to container ID and this effectively
misconfigures URLs and domain names.

HOSTNAME and HOST are now removed and default hostname is now
'localhost' (no longer 'local.cortezaproject.org')
2022-08-04 15:41:35 +02:00
Vivek Patel
75b7d78eac Fix SMTP values in settings on server restart
It addresses issue with SMTP_* setting value, which were overwritten by values from env on server restart and also, removed default value of SMTP_HOST and SMTP_PORT, which were also causing the same issue.
2022-08-04 18:06:29 +05:30
Denis Arh
609bb5fe00 Rename json tag for operation property 2022-08-03 18:00:46 +02:00
Denis Arh
4faed45a98 Fix Join-Gateway step scope corruption 2022-08-03 12:34:15 +02:00
Denis Arh
14026ec47e Workflow sessions can be canceled via API 2022-08-03 12:33:14 +02:00
Denis Arh
8e42876010 Return workflow session ID when executing wf
It also fixes missimplemented async exec.
2022-08-03 12:33:14 +02:00
Tomaž Jerman
bba02ec9df Implement base valuestore for env variables
* Base facility to store generic values,
* integrate with templates,
* integrate with workflows.
2022-08-02 13:13:28 +02:00
Tomaž Jerman
9e1375e99a Refactor DAL capabilities into operations and properties 2022-08-02 12:57:22 +02:00
Denis Arh
c9442260c5 Add support for revisions on compose records 2022-08-01 08:55:46 +02:00