DeforaNetworks/corteza
3
0
Fork 0
Code Activity

Refresh testing documentation

Browse Source
This commit is contained in:
Denis Arh
2021-12-20 07:52:48 +01:00
parent b03f265a90
commit ff7f642681
4 changed files with 73 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
TESTING.md Normal file
View File

@@ -0,0 +1,16 @@
# Corteza Server automated testing
Corteza server has three types of automated tests: unit, store and integration tests.
## Unit tests
Unit tests follow common Golang testing patterns with `_test.go` files inside packages.
Test files use the same package name as the non-test files to allow testing of unexported functions
## Store tests
[Store tests](store/tests) are intended to verify implementation of the persistence layer across multiple store backends.
## Integration tests
[integration tests](tests/README.md) are intended to verify correct exectution of API request to storage layer and back.

54
store/README.adoc
View File

@@ -1,54 +0,0 @@
= Corteza storage layer
Provides unified storage for Corteza with composable and interchangable backends.
Backends can be any RDBMS, Key-Value, NoSQL, FS, Memory.
== Development
====
@todo definition (YAML) structure
@todo how to add definitions
@todo how to add stores
====
== FAQ:
Why are we changing create/update function signature (input struct is no longer returned)?::
Because store functions are no longer manipulating the input.
Why naming inconsistency between search/lookup and create/update/...?::
To ensure function names sound more natural
Why changing find prefix to search/lookup?::
To be consistent with actions
Why do we use custom mapping (and not db:... tag on struct)?::
Separation of concerns
+ consistency with store backends that do not support db tags
+ de-cluttering types* namespace
== Testing
Running store tests:
====
make test.store
====
Running store tests with code coverage:
====
make test.store
====
See Makefile for details
== Known issues
SQLite, transactions & locking::
Transactions in SQLite are explicitly disabled due to issues
with locking (see `sqlite/sqlite.go`, line with `cfg.TxDisabled = true`)
Until this is resolved, SQLite storage backend should not be used
in production environment.
We're still keeping and maintaining it to provide the simplest and most performant backend for (service) unit testing to avoid overly complex mocking of storage.
This will likely change when we have support for built-in in-memory storage (MEMORY_DSN)

41
store/README.md Normal file
View File

@@ -0,0 +1,41 @@
# Corteza storage layer
Provides unified storage for Corteza with composable and interchangeable backends.
Backends can be any RDBMS, Key-Value, NoSQL, FS, Memory.
## FAQ
* Why are we changing create/update function signature (input struct is no longer returned)?
** Because store functions are no longer manipulating the input.
* Why naming inconsistency between search/lookup and create/update/...?
** To ensure function names sound more natural
* Why changing find prefix to search/lookup?
** To be consistent with actions
* Why do we use custom mapping (and not db:... tag on struct)?
Separation of concerns
** consistency with store backends that do not support db tags
** de-cluttering types* namespace
## Testing
Running store tests:
```shell script
make test.store
```
See [Makefile](Makefile) for details
## Known issues
SQLite, transactions & locking::
Transactions in SQLite are explicitly disabled due to issues
with locking (see `sqlite/sqlite.go`, line with `cfg.TxDisabled # true`)
Until this is resolved, SQLite storage backend should not be used
in production environment.
We're still keeping and maintaining it to provide the simplest and most performant backend for (service) unit testing to avoid overly complex mocking of storage.
This will likely change when we have support for built-in in-memory storage (MEMORY_DSN)

30
tests/README.md
View File

@@ -1,27 +1,27 @@
# Corteza Server integration and e2e tests
# Corteza Server integration tests
Main goal of these integration tests is to test entire system with the database
and without any other external services.
and without any external services.
What, how and why we test:
- Handling input:
- Handling input
How HTTP traffic is handled (standard HTTP requests, websocket communication),
to ensure proper response on valid, invalid or harmful input.
- Configuration:
- Configuration
How system behaves under different settings to cover as much different
configuration scenarios as possible.
- Security:
- Security
Are we handling access control and log events (audit log) properly?
All services and data should be protected to prevent unwanted access
and modifications.
- Scenarios:
- Scenarios
Are complex scenarios executed as designed (e.g. is password recovery email
sent and can link from the email be used)
@@ -33,12 +33,10 @@ What, how and why we test:
- Database schema & data migrations
Database migration is executed as one of the first steps in automation tests.
This can be turned off to enable running tests on live/testing database.
Tests can be executed on SQLite in-memory database for performance and isolation
purpuses
Entire test suite is ran inside transaction so we can rollback all changes
that occurred while testing
# Running tests
@@ -84,6 +82,10 @@ GOTEST=$GOPATH/bin/gotest make watch.test.integration
```
### Fine-tune test execution with `TEST_FLAGS`
Examples:
- `TEST_FLAGS="-v" make ....`
- `TEST_FLAGS="-v -run testName" make ....`
```shell script
make watch.test.integration.workflow TEST_FLAGS="-v -run Test0011_termination"
```
This will watch for file changes and run workflow tests in verbose mode.
Only tests starting with Test0011_termination will be executed.