On start up K8s service account token (SAT) is used to authenticate with
Vault.
Vault verifies SAT with K8s API and issues certificate and private key.
Broker connects to NATS using mTLS with provided certificate and key.
Start gRPC server used by components using certificate.
Starts HTTP/HTTPS servers adapters.
Component Connects
Component opens connections to broker's gRPC server using root CA to verify
certificate.
Component calls subscribe service (2-way streaming) and sends a registration
event, which contains component name, commit, id, routes, and K8s SAT.
Broker authorizes by verifying SAT with K8s API and ensures claims match
attributes provided by component.
The broker starts a subscription for the component that is used to pass
events.
Broker puts component's registration into NATS k/v store using the component
name + commit as key (not id).
Broker creates NATS consumer which listens for new messages on the component
group's subject (evt.js.{compName}.{compCommit}).
Process Genesis Event
HTTP Server
Currently only HTTP server receives genesis events.
HTTP request comes in to http server and is converted to event.
If request contains deployment and environment headers/query params they are
set on the event.
The default TTL, which can be set as a flag on broker, is set on event.
http server records request event id in map.
sends event to broker for routing
http server then blocks and waits either for ttl to expire or response to
come from broker
if ttl expires 503 is returned to client
if response comes back it is converted to http response and sent back to
client.
Broker
Broker receives http request event from http server.
Event will either have no context which indicates it should match a release
or have the deployment and environment set.
If environment or deployment is set without the other the request is
rejected.
Based on the context a matcher for the specified deployment or for all
released components is used. These are built by inspecting the deployed and
released components and looking up their routes in the registration stored in
the NATS.
If request does not match any routes a not found error is returned.
If matched the event target is updated with the matched component's name,
commit, and matched route id.
broker checks if a matching component is subscribed locally via grpc and if
so sends the event to that component.
otherwise the event is publish onto the component group's NATS subject so
that another broker with a subscription for that component can process it.
ttl is updated before sending
Process Event from Component
gRPC Server
Event is received by grpc server for an existing subscription.
grpc server verifies the event source matches the subscribed component, if it
does not the broker drops the event and closes the subscription.
The grpc server passes the event to the broker for routing.
Broker
The event should already have context set as it is not a genesis event, if it
doesn't it is rejected and dropped.
The event source should be complete with component name, commit, and id
The event target should either contain only the component's name or be
complete.
Either is not true event is dropped
If only target name is set then event is matched based on context and target
commit and route id are added
If event is not matched it is dropped (currently there is no response sent to
component)
the deployment or release are then pulled and broker verifies both the source
and target components are part of the deployment spec.
if they are not event is dropped.
broker checks if a matching component is subscribed locally via grpc and if
so sends the event to that component.
otherwise the event is publish onto the component's NATS subject so that
another broker with a subscription for that component can process it.
ttl is updated before sending
Process Event from NATS
NATS Client
Receives and unmarshals event, ttl is updated based on when msg was created.
Sends event to broker for routing.
Broker
Workflow is same as for grpc but target should have both name and commit set.
If there is matching component it means that the component got unsubscribed
while process the event, otherwise the broker would not have been listening
on the subject. Is this case the event is republished onto the same subject.