Created: 2023-08-11 | Updated: 2024-09-21

Load resources from YAML file

New (v0.3.8)

Table of contents

  1. Option 1: All resources are from YAML
  2. Option 2: resources are in the code (Diagrams as a Code), additionally augmented from YAML
  3. Option 3: Existing diagram has all resources, additionally augmented from YAML

Ingest extra resources from yaml configuration into DRAWIO diagram. Allows to add vertices and edges to empty diagram or to existing one. To cover the case if there are relationships that you can not programmatically query but want to visualtize.

YAML-format supports declaration using 3 forms dialects for vertices and edges and also any mix of them:

  • src/dst are linked to yaml vertices by name
  • src/dst are linked by ARN (ARN can be present in same yaml, or loaded programmatically)
  • mixed of 1st and 2nd

Option 1: All resources are from YAML

All of these dialects are present in the first yaml snippet:

yaml file with declaration of resource:

vertices:
  - name: prod-lambda-name
    type: lambda_function
    id: arn:aws:lambda:eu-west-1:123456789012:function:prod-lambda-name
  - name: role-lambda-name
    type: iam_role
    id: arn:aws:iam::123456789012:role/prod-lambda-name
  - name: prod-cloudwatch-policy
    type: iam_policy
    id: arn:aws:iam::123456789012:policy/prod-cloudwatch-policy
  - name: prod-s3-policy
    type: iam_policy
    id: arn:aws:iam::123456789012:policy/prod-s3-policy
  - name: prod-dynamodb-policy
    type: iam_policy
    id: arn:aws:iam::123456789012:policy/prod-dynamo-policy
edges:
#  1st approach src/dst are linked to yaml vertices by name
  - { src: prod-lambda-name, dst: role-lambda-name, label: HasRole, link_type: bi }
  - { src: role-lambda-name, dst: prod-cloudwatch-policy, label: Allow CloudWatch logs, link_type: none }
#  2nd approach src/dst are linked by ARN (ARN can be present in same yaml, or loaded programmatically)
  - {
    src_id: arn:aws:iam::123456789012:role/prod-lambda-name,
    src_type: iam_role,
    dst_id: arn:aws:iam::123456789012:policy/prod-s3-policy,
    dst_type: iam_policy,
    label: Allow S3 access,
    link_type: none }
# 3rd approach is mixed of 1st and 2nd
  - {
    src_id: arn:aws:iam::123456789012:role/prod-lambda-name,
    src_type: iam_role,
    dst: prod-dynamodb-policy,
    label: Allow DynamoDB read access,
    link_type: none }

Code Snippet:

read_coords_from_file is used for positioning Vertices - coordinates from previous version drawio file will be reused for Vertices that have same ID. This function does not reload Vertices from previous version, it only operates with coordinates history.

# given
mcd = MultiCloudDiagrams()
output_file = 'docs/docs/core-components/output/drawio/yaml.drawio'
mcd.read_coords_from_file(output_file)

# when
# Read cloud resources declared in yaml format
mcd.augment_from_yaml('samples/samples/augmented_resources.yaml')

Rendering:

lambda

drawio file:

Download generated yaml.drawio:

Download

Option 2: resources are in the code (Diagrams as a Code), additionally augmented from YAML

Other option is to combine Diagrams as Code (DaC) by declaring all resources in code and also ingest additional vertices from YAML file. This is very useful when you have custom resources that are not supported by libraries to query (not available in boto3 or on-prem resources).

yaml file with resources declaration:

vertices:
  - name: event-broker
    type: mq_broker
    id: host.broker.com
  - name: on-prem-server
    type: http
    id: domain.com
edges:
  - {
    src_id: arn:aws:lambda:eu-west-1:123456789012:function:prod-lambda-name,
    src_type: lambda_function,
    dst: event-broker,
    label: Publish user activity log,
    link_type: none }
  - {
    src_id: arn:aws:lambda:eu-west-1:123456789012:function:prod-lambda-name,
    src_type: lambda_function,
    dst: on-prem-server,
    label: HTTP GET user actions,
    link_type: none }

Code Snippet:

# given
mcd = MultiCloudDiagrams()
output_file = 'docs/docs/core-components/output/drawio/yaml2.drawio'
mcd.read_coords_from_file(output_file)

# Diagrams as a Code: add Lambda and DynamoDB
mcd.add_vertex(node_id="arn:aws:lambda:eu-west-1:123456789012:function:prod-lambda-name",
               node_name='prod-lambda-name',
               node_type='lambda_function')
mcd.add_vertex(node_id='arn:aws:dynamodb:eu-west-1:123456789012:table/prod-dynamo-table',
               node_name='prod-dynamo-table',
               node_type='dynamo')
mcd.add_link('lambda_function:arn:aws:lambda:eu-west-1:123456789012:function:prod-lambda-name',
             'dynamo:arn:aws:dynamodb:eu-west-1:123456789012:table/prod-dynamo-table',
             action=['GET permissions'])

# when
# Add resources from YAML-file
mcd.augment_from_yaml('samples/samples/augmented_resources2.yaml')

Rendering:

lambda

drawio file:

Download generated yaml2.drawio:

Download

Option 3: Existing diagram has all resources, additionally augmented from YAML

We already have drawio file with rendered resources (from previous example). Now we want to instrument multicloud-diagrams that there are additional resources that are declared in yaml that should be added. Previous resources will stay in drawio diagram, yaml-based resources will be added to existing one.

yaml file with resources declaration:

vertices:
  - name: oauth-server
    type: http
    id: domain2.com
edges:
  - {
    src_id: arn:aws:lambda:eu-west-1:123456789012:function:prod-lambda-name,
    src_type: lambda_function,
    dst: oauth-server,
    label: check user authority,
    link_type: none }

Code Snippet:

# given
mcd = MultiCloudDiagrams()
input_file_previous_version = 'docs/docs/core-components/output/drawio/yaml2.drawio'
# all nodes are loaded and operable, now we can build augmented relations from external sources
mcd.read_nodes_from_file(input_file_previous_version)

# when
# Add additional resources from YAML-file
mcd.augment_from_yaml('samples/samples/augmented_resources3.yaml')

Rendering:

lambda

drawio file:

Download generated yaml3.drawio:

Download