Stateful Orchestation

The SysDesign Meetup episode is out.

Resuming after a long pause. The topic is Stateful Orchestration, although in reality I just talk through a lot of modern-day SysDesign concepts to set the table for the episodes to come!

Errata sheet: Temporal itself is open source, released under the MIT license. So, if you're an enthusiast, feel free to play with it. The enterprise solution is Temporal Cloud, http://temporal.io/cloud.

Timekeys:

0:00 Intro
2:00 Plot
3:04 Problem statement for stateful orchestration
6:09 Meta-functional requirements for software engineering industry
9:31 Generic solutions and the metric to optimize for
11:57 Standard SysDesign boxes
12:15 Rant on message buses vs. brokers
14:07 Standard SysDesign problems
18:24 There is such thing as "too much consistency"
22:12 Well-established terminology
26:06 Two mantras to unstuck oneself
30:35 The third mind trick to get unstuck
33:09 Intro to part two
34:06 Strong vs. eventual consistency
38:52 Idempotency
40:35 Total vs. partial order
46:25 Transaction isolation levels
49:02 Read your own writes
50:43 Deterministic code
57:47 CRDTs
1:02:02 At least / at most / exactly once
1:07:50 Source-of-truth
1:12:50 Two-phase commits
1:16:00 A detour on optimistic vs. pessimistic approaches
1:18:48 Decoupling producers from consumers
1:30:13 Topics / partitions, consumer groups, and offsets
1:39:01 Orchestration vs. choreography
1:41:25 Disintegrated storage
1:44:03 Workflow orchestration
1:52:13 A real-life problem
1:56:22 Solution
1:59:13 Done!
1:59:46 Q: The future of workflows vs. activities in Temporal terminology, and are we getting to exactly-once?

Video: https://tinyurl.com/sdm-orchestration
Slides: https://tinyurl.com/sdm-orchestration-slides