3 | Petronas
Shell
Managing the database of the vantage6 server can be done by an administrator through the shell.
The preferred method of managing entities is using the API, instead of the Shell interface. This because the API will perform validation of the input, whereas in the shell all inputs are accepted.
Through the shell it is possible to manage all server entities. To start the shell, use
vserver shell [options]. 
The initialization of the shell
In the next sections the different database models that are available are explained. You can retrieve any record and edit any property of it. Don't forget to call
.save() once you are done editing.Organizations
Organizations have a public key that is used for end-to-end encryption. This key is automatically created and/or uploaded by the Node the first time it's run.
To store an organization you can use the
db.Organization model:1
# create new organiztion
2
organization = db.Organization(
3
name="IKNL",
4
domain="iknl.nl",
5
address1="Zernikestraat 29",
6
address2="Eindhoven",
7
zipcode="5612HZ",
8
country="Netherlands"
9
)
10
​
11
# store organization in the database
12
organization.save()
Copied!
Retrieving organizations from the database:
1
# get all organizations in the database
2
organizations = db.Organization.get()
3
​
4
# get organization by its unique id
5
organization = db.Organization.get(1)
6
​
7
# get organization by its name
8
organization = db.Organization.get_by_name("IKNL")
Copied!
A lot of entities (e.g. users) at the server are connected to an organization. E.g. you can see which (computation) tasks are issued by the organization or see which collaborations it is participating in.
1
# retrieve organization from which we want to know more
2
organization = db.Organization.get_by_name("IKNL")
3
​
4
# get all collaborations in which the organization participates
5
collaborations = organization.collaborations
6
​
7
# get all users from the organization
8
users = organization.users
9
​
10
# get all created tasks (from all users)
11
tasks = organization.created_tasks
12
​
13
# get the results of all these tasks
14
results = organization.results
15
​
16
# get all nodes of this organization (for each collaboration
17
# an organization participates in, it needs a node)
18
nodes = organization.nodes
Copied!
Roles and Rules
A user can have multiple roles and rules assigned to them. These are used to determine if the user has permission to view, edit, create or delete certain resources using the API. A role is a collection of rules.
1
# display all available rules
2
db.Rule.get()
3
​
4
# display rule 1
5
db.Rule.get(1)
6
​
7
# display all available roles
8
db.Role.get()
9
​
10
# display role 3
11
db.Role.get(3)
12
​
13
# show all rules that belong to role 3
14
db.Role.get(3).rules
15
​
16
# retrieve a certain rule from the DB
17
rule = db.Rule.get_by_("node", Scope, Operation)
18
​
19
# create a new role
20
role = db.Role(name="role-name", rules=[rule])
21
role.save()
22
​
23
# or assign the rule directly to the user
24
user = db.User.get_by_username("some-existing-username")
25
user.rules.append(rule)
26
user.save()
Copied!
Users
Users belong to an organization. So if you have not created an organization as of yet, then you should do this first. To create a user you can use the
db.User model:1
# first obtain the organization to which the new user belongs
2
org = db.Organization.get_by_name("IKNL")
3
​
4
# obtain role 3 to assign to the new user
5
role_3 = db.Role.get(3)
6
​
7
# create the new users, see section Roles and Rules on how to
8
# deal with permissions
9
new_user = db.User(
10
username="root",
11
password="super-secret",
12
firstname="John",
13
lastname="Doe",
14
roles=[role_3],
15
rules=[],
16
organization=org
17
)
18
​
19
# store the user in the database
20
new_user.save()
Copied!
You can retrieve users in the following ways:
1
# get all users
2
db.User.get()
3
​
4
# get user 1
5
db.User.get(1)
6
​
7
# get user by username
8
db.User.getByUsername("root")
9
​
10
# get all users from the organization IKNL
11
db.Organization.get_by_name("IKNL").users
Copied!
To modify a user, simply adjust the properties and save the object.
1
user = db.User.getByUsername("some-existing-username")
2
​
3
# update the firstname
4
user.firstname = "Brandnew"
5
​
6
# update the password; it is automatically hashed.
7
user.password = "something-new"
8
​
9
# store the updated user in the database
10
user.save()
Copied!
Collaborations
A collaboration consists of one or more organizations. To create a collaboration you need at least one organization in your database. To create a collaboration you can use the
db.Collaboration model:1
# create a second organization to collaborate with
2
other_organization = db.Organization(
3
name="IKNL",
4
domain="iknl.nl",
5
address1="Zernikestraat 29",
6
address2="Eindhoven",
7
zipcode="5612HZ",
8
country="Netherlands"
9
)
10
other_organization.save()
11
​
12
# get organization we have created earlier
13
iknl = db.Organization.get_by_name("IKNL")
14
​
15
# create the collaboration
16
collaboration = db.Collaboration(
17
name="collaboration-name",
18
encrypted=False,
19
organizations=[iknl, other_organization]
20
)
21
​
22
# store the collaboration in the database
23
collaboration.save()
Copied!
Tasks, nodes and organizations are directly related to collaborations. We can obtain these by:
1
# obtain a collaboration which we like to inspect
2
collaboration = db.Collaboration.get(1)
3
​
4
# get all nodes
5
collaboration.nodes
6
​
7
# get all tasks issued for this collaboration
8
collaboration.tasks
9
​
10
# get all organizations
11
collaboration.organizations
Copied!
Setting the encryption to false at the server does not mean that the nodes will send encrypted results. This is only the case if the nodes also agree on this setting.
Nodes
Before nodes can login, they need to exist in the server's database. A new node can be created as follows:
1
# we'll use a uuid as the API-key, but you can use anything as
2
# API key
3
from uuid import uuid1
4
​
5
# nodes always belong to an organization *and* a collaboration,
6
# this combination needs to be unique!
7
iknl = db.Organization.get_by_name("IKNL")
8
collab = iknl.collaborations[0]
9
​
10
# generate and save
11
api_key = str(uuid1())
12
print(api_key)
13
​
14
node = db.Node(
15
name = f"IKNL Node - Collaboration {collab.name}",
16
organization = iknl,
17
collaboration = collab,
18
api_key = api_key
19
)
20
​
21
# save the new node to the database
22
node.save()
Copied!
API keys are hashed before stored in the database. Therefore you need to save the API key immediately. If you lose it, you can reset the API key later via the shell or via the API.
Tasks and Results
Tasks(/results) created from the shell are not picked up by nodes that are already running. The signal to notify them of a new task cannot be emitted this way.
A task is intended for one or more organizations. For each organization the task is intended for, a corresponding (initially empty) result should be created. Each task can have multiple results, for example a result from each organization.
1
# obtain organization from which this task is posted
2
iknl = db.Organization.get_by_name("IKNL")
3
​
4
# obtain collaboration for which we want to create a task
5
collaboration = db.Collaboration.get(1)
6
​
7
# obtain the next run_id. Tasks sharing the same run_id
8
# can share the temporary volumes at the nodes. Usually this
9
# run_id is assigned through the API (as the user is not allowed
10
# to do so). All tasks from a master-container share the
11
# same run_id
12
run_id = db.Task.next_run_id()
13
​
14
task = db.Task(
15
name="some-name",
16
description="some human readable description",
17
image="docker-registry.org/image-name",
18
collaboration=collaboration,
19
run_id=run_id,
20
database="default",
21
initiator=iknl,
22
)
23
task.save()
24
​
25
# input the algorithm container (docker-registry.org/image-name)
26
# expects
27
input_ = {
28
}
29
​
30
import datetime
31
​
32
# now create a result model for each organization within the
33
# collaboration. This could also be a subset
34
for org in collaboration.organizations:
35
res = db.Result(
36
input=input_,
37
organization=org,
38
task=task,
39
assigned_at=datetime.datetime.now()
40
)
41
res.save()
Copied!
Tasks can have a child/parent relationship. Note that the
run_id is for parent and child tasks the same.1
# get a task to which we want to create some
2
# child tasks
3
parent_task = db.Task.get(1)
4
​
5
child_task = db.Task(
6
name="some-name",
7
description="some human readable description",
8
image="docker-registry.org/image-name",
9
collaboration=collaboration,
10
run_id=parent_task.run_id,
11
database="default",
12
initiator=iknl,
13
parent=parent_task
14
)
15
child_task.save()
Copied!
Tasks that share a
run_id have access to the same temporary folder at the node. This allows for multi-stage algorithms.Obtaining results:
1
# obtain all Results
2
db.Result.get()
3
​
4
# obtain only completed results
5
[result for result in db.Result.get() if result.complete]
6
​
7
# obtain result by its unique id
8
db.Result.get(1)
Copied!
Last modified 1mo ago
Export as PDF
Copy link