3
0

176 Commits

Author SHA1 Message Date
Tomaž Jerman
b2abfbf613 Improve node splicing
Reuse existing spliced nodes to reduce the node count.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
7e80cac4e9 Improve sys fields; fix deleted record detection 2020-03-22 12:08:20 +01:00
Tomaž Jerman
e2781bf901 Cleanup 2020-03-22 12:08:20 +01:00
Tomaž Jerman
d487ef3abf Fix user migration node buffers 2020-03-22 12:08:20 +01:00
Tomaž Jerman
45b11a6eb9 Add a nice progress bar 2020-03-22 12:08:20 +01:00
Tomaž Jerman
0f22006643 Don't die if User not provided 2020-03-22 12:08:20 +01:00
Tomaž Jerman
8f41cc6323 Improve README.adoc 2020-03-22 12:08:20 +01:00
Tomaž Jerman
eddc114be3 Add some benchmarking 2020-03-22 12:08:20 +01:00
Tomaž Jerman
ac1fd84413 Add support for migration mapping
This adds support for splitting a single source into
multiple modules.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
52e4e6c176 Run migration in a transaction
Big performance gains, since indexes are defered to
the end.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
e30edaae4f Don't insert empty record values
Big performance gains on sparse data sets.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
5a10619242 Assure node parent order on migration
Link splicing caused order to change; broke internal
loops.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
440cf71798 Simplify migration source requirements
Remove the need of stream seeking.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
8b82eaee1d Improve thread syncing 2020-03-22 12:08:20 +01:00
Tomaž Jerman
37a669125e Make migrateable source an io.ReadSeeker 2020-03-22 12:08:20 +01:00
Tomaž Jerman
b781151aa9 Paralelize the migration system 2020-03-22 12:08:20 +01:00
Tomaž Jerman
18ad7dea98 Remove debug spew 2020-03-22 12:08:20 +01:00
Tomaž Jerman
f7fcccf0a1 Allow only satisfied dependencies to appear as a leaf node 2020-03-22 12:08:20 +01:00
Tomaž Jerman
1d8182e365 Add a bit extra documentation regarding the algorithm 2020-03-22 12:08:20 +01:00
Tomaž Jerman
c5cb20eb9f POC graph based data migration system
Implement a POC data migration system.
See README for more details.
2020-03-22 12:08:20 +01:00
Denis Arh
47db9e7ab9 Add version cli command 2020-03-16 08:54:30 +01:00
Denis Arh
700fae5e00 Call coredor (via gRPC) with independent context & timeout
Reusing context caused random case of context cancelation
on server side and "illegal buffer" error on corredor
2020-03-12 20:17:28 +01:00
Denis Arh
2aee9f80b9 Add capability to set CORREDOR_MAX_RECEIVE_MESSAGE_SIZE, set default to 16MB 2020-03-04 14:19:41 +01:00
Denis Arh
7f0fcc72b9 Add @todo for IN operator support in pkg/ql 2020-03-04 12:28:58 +01:00
Denis Arh
9405fb84c3 Improve handle checking & generation 2020-02-28 16:59:32 +01:00
Denis Arh
efb82f721d Remove obsolete automation files & references to it 2020-02-26 08:32:17 +01:00
Denis Arh
a78235252f Use clean eventbus for integration tests 2020-02-25 08:12:40 +01:00
Denis Arh
19e50302df Comply with corredor settings (enabled/disabled) 2020-02-07 13:00:31 +01:00
Denis Arh
4e1a201327 Allow indirect (no current user) script execution 2020-01-31 00:29:15 +01:00
Denis Arh
37079d77e1 Implement if-modified-since logic for client/server script loading 2020-01-30 17:17:42 +01:00
Denis Arh
5a16452f51 Fix PluckString test 2020-01-30 17:03:45 +01:00
Denis Arh
666145a837 Protobuf codegen, add uiProps 2020-01-29 20:25:49 +01:00
Denis Arh
eeb1a9a831 Refactord manual event type filtering logic & strategy 2020-01-29 08:16:40 +01:00
Denis Arh
4cf645d11a Add HasString and PluckString to slice pkg 2020-01-29 08:15:17 +01:00
Denis Arh
2706d9f019 More robust grpc connection testing 2020-01-28 18:51:47 +01:00
Denis Arh
3548742b85 Flip checking logic for url signature check 2020-01-28 10:13:17 +01:00
Denis Arh
b89b1d1056 Bundle code serving 2020-01-22 18:07:49 +01:00
Denis Arh
b9cd860bf2 Improve script filtering 2020-01-22 17:39:14 +01:00
Denis Arh
0cdc03d60a Min typo fix, import reordering 2020-01-22 08:14:33 +01:00
Denis Arh
9fd7adcaa2 Implements script security, add on-manual sec. check 2020-01-22 07:46:49 +01:00
Denis Arh
c26704b4cb Fix broken tests, typos 2020-01-21 10:52:56 +01:00
Denis Arh
db07e26fc8 Update protobuf, rename event&resource props on triggers 2020-01-20 21:21:25 +01:00
Denis Arh
19f007f886 Tweaked config default values for corredor 2020-01-19 11:05:03 +01:00
Denis Arh
50fdd5c4c3 Renamed triggers to handlers 2020-01-18 07:05:35 +01:00
Denis Arh
78525b48ea Mark pkg/automation as deprecated 2020-01-18 07:05:35 +01:00
Denis Arh
c86f83b8ed Port legacy mail matching to event(bus) 2020-01-18 07:05:35 +01:00
Denis Arh
abf4d106bb Improve eventbus & constraints handling
Constraints are now preparsed into active structs
that can be used for matching
2020-01-18 07:05:35 +01:00
Denis Arh
1e096d1abc Unify trigger endpoint names 2020-01-18 07:05:35 +01:00
Denis Arh
1195534dcd Automation REST handler & filter cleanup 2020-01-18 07:05:35 +01:00
Denis Arh
4d6cb13f70 Add basic automation script exporting 2020-01-18 07:05:34 +01:00