You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Toolshed is a collection of individial HTTP APIs that each perform tasks in specific domains and areas of interest. The spirit of Toolshed is to make space for us to quickly design, build, test, and iterate on discrete functionality within our system.
3
+
The Toolshed is a collection of individial HTTP APIs that each perform tasks in
4
+
specific domains and areas of interest. The spirit of Toolshed is to make space
5
+
for us to quickly design, build, test, and iterate on discrete functionality
6
+
within our system.
4
7
5
-
Practically speaking, Toolshed is a monolith Deno2 Hono HTTP API scaffolding where we can quickly prototype new backend HTTP services that power and support user-facing workflows. Additionally we will utilize Deno KV and Deno Queue to build a task queue for running background tasks.
8
+
Practically speaking, Toolshed is a monolith Deno 2 Hono HTTP API scaffolding
9
+
where we can quickly prototype new backend HTTP services that power and support
10
+
user-facing workflows. Additionally we will utilize Deno KV and Deno Queue to
11
+
build a task queue for running background tasks.
6
12
7
13
The reason we use a monolith is to centralize around a single CI flow, which produces a single binary artifact, which we can sign and distribute for running in several places. For example, this would be able to run locally, in the cloud, in private cloud enclave, or a raspberry pi.
8
14
9
15
This provides an easy path for building a feature complete API that's easy to distribute, sign, and run in our confidential compute environments.
10
16
11
-
# Philosophy
17
+
##Philosophy
12
18
13
19
We are a tiny crew. We don't have the luxury of time and resources, we need to quickly stub out new functionality in the service of creating a product that people love.
14
20
15
21
Due to our constraints of needing to run inside of a secure private cloud enclave, and our need for remote attestation, we should lean-in to a few clarifying principals.
16
22
17
-
### Personal computing, not webscale computing.
23
+
### Personal computing, not webscale computing
18
24
19
25
Each user will have their own instance of Toolshed.
20
26
21
27
That means we don't need to worry about web-scale tradeoffs. (ie scaling a task queue to >100k messages per second)
22
28
23
29
Instead, we can optimize for individual-user-scale. (ie scaling a task queue to < 100 messages per second)
24
30
25
-
### Minimize complexity.
31
+
### Minimize complexity
26
32
27
33
**Essential complexity** is inevitable, it is the essence of the problem you're trying to solve. **Accidental complexity** is what creeps in from all of the tech debt and decisions we make.
28
34
@@ -34,46 +40,58 @@ Don't be afraid to duplicate code, especially if it frees you from tracking yet-
34
40
35
41
Practically speaking, I think this mostly means that shared code should be general purpose utility code (reading cookies, dealing with auth, accessing a data store, etc); but avoid importing code from other endpoints/services. Instead, just use HTTP to use the other services.
36
42
37
-
### Product before protocol.
43
+
### Product before protocol
38
44
39
-
At our stage, the most important thing is that we build a product that people love.
45
+
At our stage, the most important thing is that we build a product that people
46
+
love.
40
47
41
-
Stay flexible with how things interact and talk to eachother, but don't make rigid permanent decisions that are difficult to change. We aren't ready to commit to set-in-stone protocols while we're still exploring the product-fit.
48
+
Stay flexible with how things interact and talk to eachother, but don't make
49
+
rigid permanent decisions that are difficult to change. We aren't ready to
50
+
commit to set-in-stone protocols while we're still exploring the product-fit.
42
51
43
-
Focus on delivering discrete functionality that that helps us unsderstand or support a user-facing usecase.
52
+
Focus on delivering discrete functionality that that helps us unsderstand or
53
+
support a user-facing usecase.
44
54
45
-
### Ship first, optimize later.
55
+
### Ship first, optimize later
46
56
47
-
Because we are optimizing for personal computing scale, and focused on the product look/feel, focus on SHIPPING.
57
+
Because we are optimizing for personal computing scale, and focused on the
58
+
product look/feel, focus on SHIPPING.
48
59
49
-
Don't be clever. Use boring off-the-shelf technology. Don't worry about optimizing for performance.
60
+
Don't be clever. Use boring off-the-shelf technology. Don't worry about
61
+
optimizing for performance.
50
62
51
63
Just ship it.
52
64
53
-
# Design
65
+
##Design
54
66
55
-
The idea here is that we have a single root API, which only handles ROUTING and LOGGING. Then each individual API will register and mount itself and a distinct URL endpoint.
67
+
The idea here is that we have a single root API, which only handles ROUTING and
68
+
LOGGING. Then each individual API will register and mount itself and a distinct
69
+
URL endpoint.
56
70
57
71
The URL root prefix is `/api`
58
72
59
73
For example, here are some potential endpoints we may want:
60
74
61
-
- /api/profile (queryable user profile knowledge graph)
62
-
- /api/ai/llm (planning server)
63
-
- /api/ai/img (flux)
64
-
- /api/ai/url2text (jina reader)
65
-
- /api/ai/transcribe (incredibly fast whisper)
66
-
- /api/data (synopsys db proxy)
75
+
-`/api/profile` (queryable user profile knowledge graph)
76
+
-`/api/ai/llm` (planning server)
77
+
-`/api/ai/img` (flux)
78
+
-`/api/ai/url2text` (jina reader)
79
+
-`/api/ai/transcribe` (incredibly fast whisper)
80
+
-`/api/data` (synopsys db proxy)
67
81
68
-
In the future, it's possible we will want to refactor or completely rewrite an individual endpoint, for example, the `/ai/llm` endpoint could require breaking changes to improve. During such a rewrite, if you are breaking the interface, you should add an additional version path to the URL.
82
+
In the future, it's possible we will want to refactor or completely rewrite an
83
+
individual endpoint, for example, the `/ai/llm` endpoint could require breaking
84
+
changes to improve. During such a rewrite, if you are breaking the interface,
85
+
you should add an additional version path to the URL.
69
86
70
87
`/api/profile/v2/`
71
88
72
-
This allows us to aggressively pursue new breaking ideas, while also supporting old still-functional API endpoints.
89
+
This allows us to aggressively pursue new breaking ideas, while also supporting
We still have some unknowns around how, exactly, we want to deploy things into secure enclaves. This is a very handwavy collection of thoughts.
112
-
113
-
One such option that's been talked about a lot is using kubernetes in conjunction with [Constellation from Edgeless](https://docs.edgeless.systems/constellation). The big downside with Constellation, is that we need to actually run our own kubernetes controlplane, we can't rely on AKS/EKS/GKE. This makes it significantly less attractive to me from a daily operations perspective, as it adds a large amount of complextiy and operational upkeep.
114
-
115
-
Instead, I think we would be more well suited if we had some sort of custom controlplane that manages quickly spinning up, and monitoring instances of Toolshed. What I mean by this is we can build Toolshed, sign it, then create a bare cloud VM image that contains little more than the toolshed binary. When a new user signs up, and we need to spin up their sandbox/vm, we can schedule the creation of a cloud instance using the latest VM image. When the vm image comes online, it will have everything it needs to support and expose remote attestation. This setup also gives us a straightforward path to exposing metrics from a VM to the cloud orchestrator. Since we can't see inside the vm, we will want to expose some metrics export capability so the orchestrator understands "is the toolshed running?", "is the toolshed performing?", all without ever having to peek inside or see user data.
116
-
117
-
This is sort of an unclear brain dump as I rush to get this out of my brain, but it's clear to me that this general direction will give us operational clarity and the ability to quickly iterate and improve our infrastructure.
127
+
## Deployment
128
+
129
+
We still have some unknowns around how, exactly, we want to deploy things into
130
+
secure enclaves. This is a very handwavy collection of thoughts.
131
+
132
+
One such option that's been talked about a lot is using kubernetes in
133
+
conjunction with [Constellation from
134
+
Edgeless](https://docs.edgeless.systems/constellation). The big downside with
135
+
Constellation, is that we need to actually run our own kubernetes controlplane,
136
+
we can't rely on AKS/EKS/GKE. This makes it significantly less attractive to me
137
+
from a daily operations perspective, as it adds a large amount of complextiy and
138
+
operational upkeep.
139
+
140
+
Instead, I think we would be more well suited if we had some sort of custom
141
+
controlplane that manages quickly spinning up, and monitoring instances of
142
+
Toolshed. What I mean by this is we can build Toolshed, sign it, then create a
143
+
bare cloud VM image that contains little more than the toolshed binary. When a
144
+
new user signs up, and we need to spin up their sandbox/vm, we can schedule the
145
+
creation of a cloud instance using the latest VM image. When the vm image comes
146
+
online, it will have everything it needs to support and expose remote
147
+
attestation. This setup also gives us a straightforward path to exposing metrics
148
+
from a VM to the cloud orchestrator. Since we can't see inside the vm, we will
149
+
want to expose some metrics export capability so the orchestrator understands
150
+
"is the toolshed running?", "is the toolshed performing?", all without ever
151
+
having to peek inside or see user data.
152
+
153
+
This is sort of an unclear brain dump as I rush to get this out of my brain, but
154
+
it's clear to me that this general direction will give us operational clarity
155
+
and the ability to quickly iterate and improve our infrastructure.
0 commit comments