Agenda​
- DISCUSSION: wasmbus modules and 1.0
- DISCUSSION: wasmcloud-ollama
- DISCUSSION: Progress on the 1.0 roadmap
Meeting Notes​
Welcome to new community members Markus Eicher and Eric Gregory, Eric is also a new member of the Cosmonic team 🎉.
DISCUSSION: wasmbus modules and 1.0
- As we move forward to the 1.0 milestone, we’ve been taking a look at the support we have in wasmcloud for our legacy wasmbus method for doing rpc with complex types. The approach we’re working on now, with wRPC, enables component-native communication over the lattice. We want this to feel like composing components at runtime.
- In this world, there are a lot of new concepts, including Wasm components — we’ve had support for a while, but this is still new. One aspect is unidirection links, or interface links, where we’re linking on a WASI interface.
- Over time we’ve realized that the wasmbus approach doesn’t really map to the wRPC approach.
- As we’re looking toward 1.0, this promises a certain degree of support. So we’re talking about launching 1.0 with only wRPC support, dropping wasmbus support. We expect that going forward, users will want to use components for their interoperability.
- Questions:
- Markus: Yes. If I’m starting from fresh now using components, this doesn’t affect me, right?
- Brooks: Exactly right. All of the component examples in our Quickstart will be carried forward, essentially.
- Bailey: One of the coolest things about the wRPC change is that you can build components with your own API, your own custom interfaces. You can focus on your business logic.
- Colin Murphy: So the “Rust” example in the quickstart refers to the old approach?
- Brooks: Right, with 1.0 we’ll drop that and drop the “component” suffix on the others.
- Markus: Yes. If I’m starting from fresh now using components, this doesn’t affect me, right?
- Brooks: One of our core tenets is that you shouldn’t have to recompile your business logic. We’ve supported the same API since about 2021 — you can take examples written then and run them in the Rust host we released a couple of months ago. So it’s a big deal that on this call I’m saying, “You should recompile your app into a component.” It’s for the good of the ecosystem, but we want to be really clear about that.
- wasmCloud 0.82 is coming later today and will include all of the bindings to WASI 0.2 APIs that were stabilized and made official, along with the new Wasmtime version.
DISCUSSION:Â wasmcloud-ollama
- iceber has been working on a project called wasmcloud-ollama that we want to highlight—hopefully we can get a demo in a future call!
- This project allows LLMs to be deployed in a distributed manner via wasmCloud — heavyweight inferencing can run where it makes sense.
- Questions:
- Colin: What’s the general state of LLMs in wasmCloud?
- Brooks: When it comes to the broader state of LLMs in wasmCloud, we don’t have any official support in the ecosystem. A capability provider can implement an interface like this and a component can consume that. It would be cool to have other providers as well.
- Colin: How does wasi-nn tie into this?
- Bailey: wasi-nn stands for wasi-neural-network, and Andrew Brown is working on this. We have a WIT definition in the proposal. It hasn’t moved through the phase process in the subgroup yet, but I kind of expect it to move through. An important thing to recognize is that it has an API that says “This is the type of tensor and driver I need”—in a lot of ways, you could say it’s not portable. I also feel like the breadth of software you have to install…you want sandboxing for it, but none of that compiles to WebAssembly. So, we’ve been having conversations about host plugins that can help manage that kind of thing, so as to not expand the trusted compute base — let’s not bake in Tensorflow inside of Wasmtime, for example. Today we don’t have wasi-nn support in wasmCloud, so you would write a provider that implements it.
- Brooks: I think iceber has worked on a wasi-nn adapter. To be super explicit, I think this is a really cool example for using components and an LLM, but it’s not to end-all-be-all recommendation for how to do things. I appreciate how iceber has proposed things with a concrete example.
- Colin: What’s the general state of LLMs in wasmCloud?
DISCUSSION: Progress on the 1.0 roadmap
- wasmCloud 1.0 roadmap
- We’ve been making great progress toward the 1.0 milestone. We officially have more in the completed column than to-do!
- The biggest remaining piece is integrating the wRPC protocol.
- Taylor has a PR out for formalizing the control interface.
- Joonas opened a PR for adding metrics support. We’re wrapping up the effort toward making wasmCloud completely OTEL-observable. We’ll export tracing, logging, and metrics. That’s a demo I’d like to do at the beginning of one of our next calls. This is all really awesome and just plugs into existing cloud native infrastructure.
- We need to pin WIT definitions, drop deprecated
wasmbus.evt.{latticeID}subject, and remove capability claim signing. We’ll be dropping capability claim signing from our examples and wash CLI.
DISCUSSION: Heads up on forthcoming OCI implementation
- Taylor: This is me wearing my CNCF hat to let the wasmCloud community know: we’re working on something to standardize Wasm components with OCI. So you’ll be able to use OCI registries to store components. After KubeCon, a couple white papers will be landing.
- Also, wasi-cloud is being actively worked on. First drafts will be in a useable state soon.
- Bailey: How is this new OCI approach different from wasmCloud’s current approach?
- Taylor: The current approach is single layer and differs across platforms. Spin does this, we do this, wasmEdge maybe. There’s no consistency. The idea is that these components will be shareable. The first piece is standardized metadata. If you were working with Wasmtime and then want to run on wasmCloud, you don’t have to repackage it.
The second piece is that components can be composed of components that are themselves composed of components. So we want to have an “exploded view” where you can store these in layers and de-duplicate the same way you do with containers. The goal is to have this done in the next month or two.
- wasmCloud users shouldn’t even notice a difference in standard usage — the previous OCI model should work as before, and we’ll also support the new one.
Where we’ll be…
Check out the Arm Developer Podcast where Bailey and Liam discussed the intersection of Wasm and GPU technologies.
Cosmonic CTO Bailey Hayes met Chris Matteson (Fermyon) and Oscar Spencer (F5 NGINX) on the panel at the latest Kubernetes meetup in New York. They explored the Wasm what’s, why’s, standards and considerations when adoption. The recording is live!
Listen in to the last WasmEdge community meeting where Bailey Hayes talks all things WASI 0.2.0 and we hear from the students of the University of Tokyo on some cool projects.
February 22 Bailey will be a guest on the Rancher Live podcast with Divya Mohan. Tune in for a deep dive into WASI 0.2.0!
March 14 - 15 We’ll be at WASM I/O in Barcelona which really is becoming a highlight event. Be sure to say Hi if you’re there and get your tickets early.
March 19 - 22 We’ll also be in Paris for KubeCon + CloudNativeCon EU 2024 and Cloud Native Wasm Day plus a host of other co-located events. The schedules are live!