3
0

1775 Commits

Author SHA1 Message Date
Denis Arh
cb9682e404 Merge branch 'feature-corredor-iterators' into develop 2020-03-26 19:07:46 +01:00
Denis Arh
2fac997b61 Fix script exec permission testing
Explicity triggered scripts are no longer the only source of
permissions - iterators use the same mechanisms
2020-03-26 19:07:14 +01:00
Denis Arh
6e5047bd22 Add ability to manually trigger record iterator via API 2020-03-26 19:07:14 +01:00
Denis Arh
a3b1450e16 Add support for iterators on record service 2020-03-26 19:07:14 +01:00
Denis Arh
33721bf870 Refactor corredor service, add supprot for iterators 2020-03-26 19:07:14 +01:00
Denis Arh
2767bbb38a Add onIteration compose record event 2020-03-26 19:07:13 +01:00
Denis Arh
58e9447aec Add support for find-by-any on namespace 2020-03-26 19:07:13 +01:00
Denis Arh
3622fa2b61 Add support for find-by-any on module 2020-03-26 19:07:13 +01:00
Denis Arh
a7e0a2306d Add pagination parsing to rh package 2020-03-26 19:07:13 +01:00
Denis Arh
db789e10fd Add debug routes for eventbus & corredor service 2020-03-26 19:07:13 +01:00
Denis Arh
ce10e46e71 Refactor corredor service and make it less events-oriented 2020-03-26 19:07:13 +01:00
Denis Arh
44f818e180 Refactor handler ops maker functions 2020-03-26 19:07:13 +01:00
Denis Arh
5cba50e3b2 Update registerTriggers fn() comment 2020-03-26 19:07:13 +01:00
Denis Arh
027fb6af1b Minor var. name fixes 2020-03-26 19:07:13 +01:00
Denis Arh
f7191963dc Improve handling of complex constraint checks 2020-03-26 19:07:13 +01:00
Tomaž Jerman
314cf76f10 Merge branch 'feature-ng-importer' into develop 2020-03-25 15:01:02 +01:00
Tomaž Jerman
e6ab67d2fb Post feedback tweaks 2020-03-25 14:54:48 +01:00
Tomaž Jerman
ecf85b340d Refactor system's naming convention 2020-03-25 13:37:43 +01:00
Tomaž Jerman
9fada4be65 Add some extra error logs 2020-03-25 10:06:18 +01:00
Tomaž Jerman
9b1fb21872 Allow complex expressions for field mapping condition 2020-03-24 22:43:25 +01:00
Tomaž Jerman
a43811150a Include current row in value mapping context 2020-03-24 22:42:29 +01:00
Tomaž Jerman
62a8ebe965 Map alias to handle on user migration 2020-03-24 13:31:35 +01:00
Tomaž Jerman
03f6024c49 Assure date time format 2020-03-24 13:31:02 +01:00
Tomaž Jerman
d034197092 Add support for expression evaluation 2020-03-24 12:32:06 +01:00
Tomaž Jerman
8e31d43c29 Add support for composite keys with join opp. 2020-03-24 11:00:46 +01:00
Tomaž Jerman
b17b31319d Allow composition for migration maps
This will reduce their size.
2020-03-23 22:11:04 +01:00
Tomaž Jerman
e153da5dab Allow different path for migration meta files 2020-03-23 22:00:11 +01:00
Tomaž Jerman
6ccc4c7294 Add support for per field value mapping 2020-03-23 21:39:10 +01:00
Tomaž Jerman
457d5a1561 Improve migration source joining system 2020-03-23 18:11:54 +01:00
Tomaž Jerman
db90a03c01 More descriptive stream splitting errors 2020-03-23 17:56:41 +01:00
Tomaž Jerman
0a7a3bf6b5 Rework splitted stream header determination 2020-03-23 17:56:11 +01:00
Tomaž Jerman
09aa097791 Add missing deps 2020-03-22 12:08:20 +01:00
Tomaž Jerman
d4fdc7f63b Don't include the original migration node for strem splitting 2020-03-22 12:08:20 +01:00
Tomaž Jerman
4b995099d2 Add support for migration source joins 2020-03-22 12:08:20 +01:00
Tomaž Jerman
53288a00cd Update dep references 2020-03-22 12:08:20 +01:00
Tomaž Jerman
eeae3dac24 Small data handling tweak 2020-03-22 12:08:20 +01:00
Tomaž Jerman
7a8c16a090 Fix SF's white space encoding
They use some special character, not supported by our
character encoding.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
c49124bf31 Improve reference fallbacks
If reference can't be established; remove the value so it
doesn't break record resolution.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
39cb2fa434 Cleanup field validation 2020-03-22 12:08:20 +01:00
Tomaž Jerman
b2abfbf613 Improve node splicing
Reuse existing spliced nodes to reduce the node count.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
7e80cac4e9 Improve sys fields; fix deleted record detection 2020-03-22 12:08:20 +01:00
Tomaž Jerman
e2781bf901 Cleanup 2020-03-22 12:08:20 +01:00
Tomaž Jerman
d487ef3abf Fix user migration node buffers 2020-03-22 12:08:20 +01:00
Tomaž Jerman
45b11a6eb9 Add a nice progress bar 2020-03-22 12:08:20 +01:00
Tomaž Jerman
0f22006643 Don't die if User not provided 2020-03-22 12:08:20 +01:00
Tomaž Jerman
8f41cc6323 Improve README.adoc 2020-03-22 12:08:20 +01:00
Tomaž Jerman
eddc114be3 Add some benchmarking 2020-03-22 12:08:20 +01:00
Tomaž Jerman
ac1fd84413 Add support for migration mapping
This adds support for splitting a single source into
multiple modules.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
52e4e6c176 Run migration in a transaction
Big performance gains, since indexes are defered to
the end.
2020-03-22 12:08:20 +01:00
Tomaž Jerman
e30edaae4f Don't insert empty record values
Big performance gains on sparse data sets.
2020-03-22 12:08:20 +01:00