3.4.4. Use

Hub

You can manage the hub as a whole using the v6 hub command, and manage individual components using their respective commands, i.e. v6 hq, v6 algorithm-store, v6 auth.

To quickly set up a local environment, see the Quickstart section.

Production - Quick start

To create a new vantage6 hub, run the command below. A questionnaire will be started that allows you to set up configuration files for the hub’s components, i.e. for HQ, algorithm store, and authentication service.

v6 hub new

For more details, check out the hub-configure section.

To run a vantage6 hub, execute the command below.

v6 hub start --name <your_hub>

To show the current logs of the hub components, you can use the following commands:

v6 hq attach
v6 algorithm-store attach
v6 auth attach

Finally, a vantage6 hub can be stopped again with:

v6 hub stop --name <your_hub>

Available commands

The following commands are available for the hub as a whole. To see all the options that are available per command use the --help flag, e.g. v6 hub start --help.

Table 3.1 Available commands

Command	Description
v6 hub new	Create configuration files for all hub’s components
v6 hub start	Start the hub
v6 hub stop	Stop the hub

Managing individual components

The individual components of the hub can be managed separately. The separate commands offer more options, since some actions cannot be performed on the hub as a whole. For example, the attach command that shows the logs would be confusing if shown for all components simultaneously.

Available commands

The commands v6 hq, v6 algorithm-store and v6 auth contain roughly the same set of subcommands. Below, the commands they have in common are listed, as example for HQ.

Table 3.2 Commands available for all hub components

Command	Description
v6 hq start	Start HQ
v6 hq stop	Stop HQ
v6 hq files	List the files that HQ is using
v6 hq attach	Show HQ’s logs in the current terminal
v6 hq remove	Remove the configuration files and data directories associated with HQ
v6 hq list	List the available HQ instances

There are also a few commands that are specific to a certain hub component. These are listed below.

Table 3.3 Commands available only for specific hub components

Command	Description
v6 hq import	Import HQ entities such as organizations, users and collaborations
v6 hq version	Shows the HQ version.
v6 algorithm-store version	Shows the algorithm store version.
v6 auth install-keycloak	Installs custom Kubernetes resources required to run the authentication service.

To see all the options that are available per command use the --help flag, e.g. v6 hq start --help.

Testing

You can test the infrastructure via the v6 sandbox and v6 test commands. The purpose of v6 sandbox is to easily setup and run a test hub accompanied by N nodes locally (default is 3 nodes). For example, if you have N = 10 datasets to test a particular algorithm on, then you can spawn a hub and 10 nodes with a single command. By default, the nodes are given access to a test dataset of olympic medal winners, containing data on e.g. their age, height, length, weight, medal type and sport.

The v6 test command is used to run the v6-diagnostics algorithm. You can run it on an existing network or create a v6 sandbox network and run the test on that in one go.

You can view all available commands in the table below, or alternatively, use v6 sandbox --help. By using --help with the individual commands (e.g. v6 sandbox start --help), you can view more details on how to execute them.

Command	Description
v6 sandbox new	Create a new network, and start it
v6 sandbox start	Start the network
v6 sandbox stop	Stop the network
v6 sandbox remove	Remove the network completely
v6 test feature-test	Run the feature-tester algorithm on an existing network
v6 test integration-test	Create a dev network and run feature-tester on that network

An overview of the tests that the v6-diagnostics algorithm runs is given below.

• Environment variables: Reports the environment variables that are set in the algorithm container by the node instance. For example the location of the input, token and output files.

• Input file: Reports the contents of the input file, usually the algorithm method and its arguments. You can verify that the input set by the client is actually received by the algorithm.

• Output file: Writes ‘test’ to the output file and reads it back.

• Token: Check that central compute tasks receive a token through the environment variables.

• Local proxy: Sends a request to the local proxy. The local proxy is used to reach HQ from the algorithm container. This is needed as parent containers need to be able to create child containers (=subtasks). The local proxy also handles encryption/decryption of the algorithm arguments and results as the algorithm container is not allowed to know the private key.

• Subtask creation: Creates a subtask (using the local proxy) and waits for the result.

• Isolation test: Checks if the algorithm container is isolated such that it can not reach the internet. It tests this by trying to reach google.nl, so make sure this is not a whitelisted domain when testing.

• Database readable: Check if the file-based database is readable.

Import resources into HQ

Note

If you are using a sandbox or development environment, a test set of users, organizations and collaborations is created for you automatically.

You can easily create a set of test users, organizations and collaborations by using a batch import. To do this, use the v6 hq import /path/to/file.yaml command. An example yaml file is provided below.

You can download this file here.

Example batch import

organizations:

  - name:       IKNL
    domain:     iknl.nl
    address1:   Godebaldkwartier 419
    address2:   ""
    zipcode:    3511DT
    country:    Netherlands
    users:
      - username: admin
        password: Password123!
      - username: frank
        password: Password123!
      - username: bart
        password: Password123!
    public_key: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUF2eU4wWVZhWWVZcHVWRVlpaDJjeQphTjdxQndCUnB5bVVibnRQNmw2Vk9OOGE1eGwxMmJPTlQyQ1hwSEVGUFhZQTFFZThQRFZwYnNQcVVKbUlseWpRCkgyN0NhZTlIL2lJbUNVNnViUXlnTzFsbG1KRTJQWDlTNXVxendVV3BXMmRxRGZFSHJLZTErUUlDRGtGSldmSEIKWkJkczRXMTBsMWlxK252dkZ4OWY3dk8xRWlLcVcvTGhQUS83Mm52YlZLMG9nRFNaUy9Jc1NnUlk5ZnJVU1FZUApFbGVZWUgwYmI5VUdlNUlYSHRMQjBkdVBjZUV4dXkzRFF5bXh2WTg3bTlkelJsN1NqaFBqWEszdUplSDAwSndjCk80TzJ0WDVod0lLL1hEQ3h4eCt4b3cxSDdqUWdXQ0FybHpodmdzUkdYUC9wQzEvL1hXaVZSbTJWZ3ZqaXNNaisKS2VTNWNaWWpkUkMvWkRNRW1QU29rS2Y4UnBZUk1lZk0xMWtETTVmaWZIQTlPcmY2UXEyTS9SMy90Mk92VDRlRgorUzVJeTd1QWk1N0ROUkFhejVWRHNZbFFxTU5QcUpKYlRtcGlYRWFpUHVLQitZVEdDSC90TXlrRG1JK1dpejNRCjh6SVo1bk1IUnhySFNqSWdWSFdwYnZlTnVaL1Q1aE95aE1uZHU0c3NpRkJyUXN5ZGc1RlVxR3lkdE1JMFJEVHcKSDVBc1ovaFlLeHdiUm1xTXhNcjFMaDFBaDB5SUlsZDZKREY5MkF1UlNTeDl0djNaVWRndEp5VVlYN29VZS9GKwpoUHVwVU4rdWVTUndGQjBiVTYwRXZQWTdVU2RIR1diVVIrRDRzTVQ4Wjk0UVl2S2ZCanU3ZXVKWSs0Mmd2Wm9jCitEWU9ZS05qNXFER2V5azErOE9aTXZNQ0F3RUFBUT09Ci0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLQo=

  - name:       Small Organization
    domain:     small-organization.example
    address1:   Big Ambitions Drive 4
    address2:   ""
    zipcode:    1234AB
    country:    Nowhereland
    users:
      - username: admin-small-organization
        password: Password123!
      - username: stan
        password: Password123!
    public_key: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUF2eU4wWVZhWWVZcHVWRVlpaDJjeQphTjdxQndCUnB5bVVibnRQNmw2Vk9OOGE1eGwxMmJPTlQyQ1hwSEVGUFhZQTFFZThQRFZwYnNQcVVKbUlseWpRCkgyN0NhZTlIL2lJbUNVNnViUXlnTzFsbG1KRTJQWDlTNXVxendVV3BXMmRxRGZFSHJLZTErUUlDRGtGSldmSEIKWkJkczRXMTBsMWlxK252dkZ4OWY3dk8xRWlLcVcvTGhQUS83Mm52YlZLMG9nRFNaUy9Jc1NnUlk5ZnJVU1FZUApFbGVZWUgwYmI5VUdlNUlYSHRMQjBkdVBjZUV4dXkzRFF5bXh2WTg3bTlkelJsN1NqaFBqWEszdUplSDAwSndjCk80TzJ0WDVod0lLL1hEQ3h4eCt4b3cxSDdqUWdXQ0FybHpodmdzUkdYUC9wQzEvL1hXaVZSbTJWZ3ZqaXNNaisKS2VTNWNaWWpkUkMvWkRNRW1QU29rS2Y4UnBZUk1lZk0xMWtETTVmaWZIQTlPcmY2UXEyTS9SMy90Mk92VDRlRgorUzVJeTd1QWk1N0ROUkFhejVWRHNZbFFxTU5QcUpKYlRtcGlYRWFpUHVLQitZVEdDSC90TXlrRG1JK1dpejNRCjh6SVo1bk1IUnhySFNqSWdWSFdwYnZlTnVaL1Q1aE95aE1uZHU0c3NpRkJyUXN5ZGc1RlVxR3lkdE1JMFJEVHcKSDVBc1ovaFlLeHdiUm1xTXhNcjFMaDFBaDB5SUlsZDZKREY5MkF1UlNTeDl0djNaVWRndEp5VVlYN29VZS9GKwpoUHVwVU4rdWVTUndGQjBiVTYwRXZQWTdVU2RIR1diVVIrRDRzTVQ4Wjk0UVl2S2ZCanU3ZXVKWSs0Mmd2Wm9jCitEWU9ZS05qNXFER2V5azErOE9aTXZNQ0F3RUFBUT09Ci0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLQo=

  - name:       Big Organization
    domain:     big-organization.example
    address1:   Offshore Accounting Drive 19
    address2:   ""
    zipcode:    54331
    country:    Nowhereland
    users:
      - username: admin-big-organization
        password: Password123!
    public_key: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUF2eU4wWVZhWWVZcHVWRVlpaDJjeQphTjdxQndCUnB5bVVibnRQNmw2Vk9OOGE1eGwxMmJPTlQyQ1hwSEVGUFhZQTFFZThQRFZwYnNQcVVKbUlseWpRCkgyN0NhZTlIL2lJbUNVNnViUXlnTzFsbG1KRTJQWDlTNXVxendVV3BXMmRxRGZFSHJLZTErUUlDRGtGSldmSEIKWkJkczRXMTBsMWlxK252dkZ4OWY3dk8xRWlLcVcvTGhQUS83Mm52YlZLMG9nRFNaUy9Jc1NnUlk5ZnJVU1FZUApFbGVZWUgwYmI5VUdlNUlYSHRMQjBkdVBjZUV4dXkzRFF5bXh2WTg3bTlkelJsN1NqaFBqWEszdUplSDAwSndjCk80TzJ0WDVod0lLL1hEQ3h4eCt4b3cxSDdqUWdXQ0FybHpodmdzUkdYUC9wQzEvL1hXaVZSbTJWZ3ZqaXNNaisKS2VTNWNaWWpkUkMvWkRNRW1QU29rS2Y4UnBZUk1lZk0xMWtETTVmaWZIQTlPcmY2UXEyTS9SMy90Mk92VDRlRgorUzVJeTd1QWk1N0ROUkFhejVWRHNZbFFxTU5QcUpKYlRtcGlYRWFpUHVLQitZVEdDSC90TXlrRG1JK1dpejNRCjh6SVo1bk1IUnhySFNqSWdWSFdwYnZlTnVaL1Q1aE95aE1uZHU0c3NpRkJyUXN5ZGc1RlVxR3lkdE1JMFJEVHcKSDVBc1ovaFlLeHdiUm1xTXhNcjFMaDFBaDB5SUlsZDZKREY5MkF1UlNTeDl0djNaVWRndEp5VVlYN29VZS9GKwpoUHVwVU4rdWVTUndGQjBiVTYwRXZQWTdVU2RIR1diVVIrRDRzTVQ4Wjk0UVl2S2ZCanU3ZXVKWSs0Mmd2Wm9jCitEWU9ZS05qNXFER2V5azErOE9aTXZNQ0F3RUFBUT09Ci0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLQo=

collaborations:

  - name: ZEPPELIN
    participants:
      - name: IKNL
      - name: Small Organization
      - name: Big Organization
    encrypted: false

  - name: PIPELINE
    participants:
      - name: IKNL
      - name: Big Organization
    encrypted: false

  - name: SLIPPERS
    participants:
      - name: Small Organization
      - name: Big Organization
    encrypted: false

Warning

All users that are imported using v6 hq import receive all permissions. Therefore, this should only be used for testing purposes.